7.3 Singular Value Decomposition

Key theory:

  • unlike EVD, exists for any matrix
  • singular values are unique, values are not
  • eigenvalues of AA are squares of singular values of A
    • eigenvectors are related to right singular vectors
  • for m×n matrices with m>n, the last mn singular values are 0 and gives a thin form
  • 2-norm of a matrix is the first singular value
  • condition number is ratio between first singular value and minth singular value
knitr::include_graphics("images/decomp_table.png")

Exercise 7.3.3

Let x be a vector of 1000 equally spaced points between 0 and 1. Suppose A_n is the 1000 \times n matrix whose (i,j) entry is x_i^{j-1} for j = 1,...,n

  1. Print out singular values of A_1, A_2, and A_3
x = range(0, 1.0, length = 1000);
A1 = [x[i]^(j-1) for i = 1:1000, j = 1];
A2 = [x[i]^(j-1) for i = 1:1000, j = 1:2];
A3 = [x[i]^(j-1) for i = 1:1000, j = 1:3];

svdvals(A1)
svdvals(A2)
svdvals(A3)
  1. Make a log-linear plot of the singular values of A_40
A40 = [x[i]^(j-1) for i = 1:1000, j = 1:40];
s = svdvals(A40);

plot(s, leg=:none, yaxis=(:log10,"Log Singular Values"))
  1. Repeat b after converting x to type Float32
A40 = [Float32(x[i])^(j-1) for i = 1:1000, j = 1:40];
s = svdvals(A40);

plot(s, leg=:none, yaxis=(:log10,"Log Singular Values"),
     title = "Single Precision")
  1. Having seen (c), which singular values of (b) do you suspect may be incorrect?