Newton for nonlinear systems

• more complicated with multiple variables and equations
• hard problem to tackle in the general case
• based on Taylor series: linear part of function near x plus higher order term

\[ \mathbf{f}(\mathbf{x} + \mathbf{h}) = \mathbf{f}(\mathbf{x}) + \mathbf{J}(x)\mathbf{h}+O(||\mathbf{h}||^2) \]

Multidimensional Newton’s method:

Given \(\mathbf{f}\) and a starting value \(\mathbf{x}_1\), for each \(k = 1, 2, 3, ...\):

1. Compute \(\mathbf{y}_k = \mathbf{f}(\mathbf{x}_k)\) and \(\mathbf{A}_k = \mathbf{f}'(\mathbf{x}_k)\)
2. Solve linear system \(\mathbf{A}_k\mathbf{s}_k = -\mathbf{y}_k\) for the Newton step \(\mathbf{s}_k\)
3. Let \(\mathbf{x}_{k+1} = \mathbf{x}_k + \mathbf{s}_k\)