Nonlinear least squares

Gauss-Newton method:

Given \(\mathbf{f}\) and a starting value \(\mathbf{x}_1\), for each \(k = 1, 2, 3, ...\):

1. Compute \(\mathbf{y}_k = \mathbf{f}(\mathbf{x}_k)\) and \(\mathbf{A}_k\) at \(\mathbf{x}_k\)
2. Solve linear least squares \(||\mathbf{A}_k\mathbf{s}_k + \mathbf{y}_k ||_2\) for \(\mathbf{s}_k\)
3. Let \(\mathbf{x}_{k+1} = \mathbf{x}_k + \mathbf{s}_k\)