Unit 10: The Singular Value Decomposition and Pseudoinverse
Any matrix at all, even a non-square one, unpacked into a rotation, a stretch along perpendicular axes, and another rotation.
7 topics · 26 questions
Each topic is a path, not a menu: work down it in order and every question builds on the one above. Click any of them to watch the answer draw itself.
The Idea of the SVD
- How it worksHow does the SVD write any matrix A as U Sigma V transpose using two orthogonal matrices and one diagonal one?
- How it worksHow does tracking where the SVD sends the unit circle reveal both the singular values and the directions U and V point in?
- Worked exampleFind U, Sigma, and V for the diagonal matrix [[3, 0], [0, 1]], where the SVD is easiest to see directly.
- Common mistakeWhy isn't the SVD just another version of diagonalization that happens to work on every matrix?
- Going furtherWhat happens to the SVD of a matrix that's already symmetric and positive definite?
Your turn1 of 2
Find the product of the two singular values of the diagonal matrix [[-3, 0], [0, 6]].Computing a Singular Value Decomposition
- How it worksHow do the eigenvectors of A transpose A become the right singular vectors packed into V?
- Worked exampleFind the singular values of the matrix [[3, 0], [0, 2]].
- Worked exampleFind the full singular value decomposition of the matrix [[2, 0], [0, -3]].
- Common mistakeWhy doesn't every matrix's singular values turn out equal to its eigenvalues?
Your turn1 of 2
Find the sum of the singular values of the matrix [[-3, 0], [0, 6]].Singular Values and the Unit Circle Becoming an Ellipse
- How it worksHow do the singular values measure exactly how long the resulting ellipse's axes turn out to be?
- Worked exampleFind the lengths of the semi-axes of the ellipse that the matrix [[2, 0], [0, 5]] maps the unit circle onto.
- Common mistakeWhy doesn't the largest singular value have to equal the largest entry sitting inside the matrix?
Your turn1 of 2
Find the length of the longer semi-axis of the ellipse that the matrix [[8, 0], [0, 3]] maps the unit circle onto.The Pseudoinverse
- How it worksHow does the pseudoinverse invert the singular values that aren't zero and leave everything else alone?
- Worked exampleFind the pseudoinverse of the matrix [[2, 0], [0, 0]].
- Common mistakeWhy doesn't multiplying a matrix by its pseudoinverse always give back the identity matrix?
Your turn1 of 2
Find the (1,1) entry of the pseudoinverse of the diagonal matrix [[7, 0], [0, 0]].Least Squares via the Pseudoinverse
- How it worksHow does multiplying b by the pseudoinverse of A produce the same answer the normal equations do?
- Worked exampleFind the least squares solution to the system x = 1, x = 3 using the pseudoinverse.
- Ties togetherHow does the pseudoinverse's solution connect back to projecting b onto the column space of A?
Your turn1 of 2
Find the least squares solution to the system x = 4, x = 3 using the pseudoinverse.Rank and the SVD
- How it worksHow does keeping only the largest few singular values build the best possible lower-rank approximation of a matrix?
- Worked exampleFind the rank of a matrix whose singular values are 5, 3, 0, and 0.
- Common mistakeWhy doesn't a tiny nonzero singular value always deserve to be treated as truly nonzero in practice?
- Going furtherWhat happens to an image compressed by keeping only its top 10 singular values instead of all of them?
Your turn1 of 2
A matrix has 7 singular values, of which 2 are nonzero. Find its rank.The SVD and the Four Fundamental Subspaces
- How it worksHow do the columns of U and V split into orthonormal bases for the column space, left null space, row space, and null space?
- Worked exampleFind which columns of U and V correspond to the null space of a matrix with singular values 4, 2, and 0.
- Common mistakeWhy doesn't every column of V correspond to a nonzero singular value?
- Ties togetherHow does the SVD's picture of four subspaces connect back to the orthogonality proved earlier in the course?
Your turn1 of 2
A matrix has 7 rows, 7 columns, and 3 nonzero singular values (its rank). Find the number of columns of V that correspond to the null space (the zero singular values).