Newton for nonlinear systems

• more complicated with multiple variables and equations
• hard problem to tackle in the general case
• based on Taylor series: linear part of function near x plus higher order term

$$\mathbf{f}(\mathbf{x} + \mathbf{h}) = \mathbf{f}(\mathbf{x}) + \mathbf{J}(x)\mathbf{h}+O(||\mathbf{h}||^2)$$

Multidimensional Newton’s method:

Given $\mathbf{f}$ and a starting value $\mathbf{x}_1$, for each $k = 1, 2, 3, ...$:

1. Compute $\mathbf{y}_k = \mathbf{f}(\mathbf{x}_k)$ and $\mathbf{A}_k = \mathbf{f}'(\mathbf{x}_k)$
2. Solve linear system $\mathbf{A}_k\mathbf{s}_k = -\mathbf{y}_k$ for the Newton step $\mathbf{s}_k$
3. Let $\mathbf{x}_{k+1} = \mathbf{x}_k + \mathbf{s}_k$