7.3 Singular Value Decomposition

Key theory:

  • unlike EVD, exists for any matrix
  • singular values are unique, values are not
  • eigenvalues of A\(^*\)A are squares of singular values of A
    • eigenvectors are related to right singular vectors
  • for \(m \times n\) matrices with \(m > n\), the last \(m-n\) singular values are 0 and gives a thin form
  • 2-norm of a matrix is the first singular value
  • condition number is ratio between first singular value and \(\min \{m,n\}\)th singular value
knitr::include_graphics("images/decomp_table.png")

Exercise 7.3.3

Let x be a vector of 1000 equally spaced points between 0 and 1. Suppose A\(_n\) is the \(1000 \times n\) matrix whose \((i,j)\) entry is \(x_i^{j-1}\) for \(j = 1,...,n\)

  1. Print out singular values of A\(_1\), A\(_2\), and A\(_3\)
x = range(0, 1.0, length = 1000);
A1 = [x[i]^(j-1) for i = 1:1000, j = 1];
A2 = [x[i]^(j-1) for i = 1:1000, j = 1:2];
A3 = [x[i]^(j-1) for i = 1:1000, j = 1:3];

svdvals(A1)
svdvals(A2)
svdvals(A3)
  1. Make a log-linear plot of the singular values of A\(_40\)
A40 = [x[i]^(j-1) for i = 1:1000, j = 1:40];
s = svdvals(A40);

plot(s, leg=:none, yaxis=(:log10,"Log Singular Values"))
  1. Repeat b after converting x to type Float32
A40 = [Float32(x[i])^(j-1) for i = 1:1000, j = 1:40];
s = svdvals(A40);

plot(s, leg=:none, yaxis=(:log10,"Log Singular Values"),
     title = "Single Precision")
  1. Having seen (c), which singular values of (b) do you suspect may be incorrect?