Pseudoinverse and definiteness

The normal equations are a square \(n\times n\) linear system to solve for \(\mathbf{x}\) which leads to the defintion of the pseudoinverse as a formal solution:

\[ \mathbf{A}^+ = (\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T \]

In practice this is not used for the same reason that the ordinary inverse is not used. But conceptually the \ operator is mathematically equivalent to left multiplying by the inverse (square matrix) or pseudoinverse (rectangular).

The matrix \(\mathbf{A}^T\mathbf{A}\) has some important properties:

1. \(\mathbf{A}^T\mathbf{A}\) is symmetric

2. \(\mathbf{A}^T\mathbf{A}\) is singular only if the columns of \(\mathbf{A}\) or linearly dependant.

3. If \(\mathbf{A}^T\mathbf{A}\) is nonsingular, that it is positive definate.