Nonlinear least squares

Gauss-Newton method:

Given $\mathbf{f}$ and a starting value $\mathbf{x}_1$, for each $k = 1, 2, 3, ...$:

1. Compute $\mathbf{y}_k = \mathbf{f}(\mathbf{x}_k)$ and $\mathbf{A}_k$ at $\mathbf{x}_k$
2. Solve linear least squares $||\mathbf{A}_k\mathbf{s}_k + \mathbf{y}_k ||_2$ for $\mathbf{s}_k$
3. Let $\mathbf{x}_{k+1} = \mathbf{x}_k + \mathbf{s}_k$