Pseudoinverse and definiteness
The normal equations are a square \(n\times n\) linear system to solve for \(\mathbf{x}\) which leads to the defintion of the pseudoinverse as a formal solution:
\[ \mathbf{A}^+ = (\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T \]
In practice this is not used for the same reason that the ordinary inverse is not used. But conceptually the \
operator is mathematically equivalent to left multiplying by the inverse (square matrix) or pseudoinverse (rectangular).
The matrix \(\mathbf{A}^T\mathbf{A}\) has some important properties:
\(\mathbf{A}^T\mathbf{A}\) is symmetric
\(\mathbf{A}^T\mathbf{A}\) is singular only if the columns of \(\mathbf{A}\) or linearly dependant.
If \(\mathbf{A}^T\mathbf{A}\) is nonsingular, that it is positive definate.