Lab 16 — SVD: The Matrix Microscope

Explore singular values, singular vectors, rank-one layers, low-rank approximation, compression, denoising, PCA intuition, and hidden factors.

Big idea

The singular value decomposition writes a matrix as A = UΣVᵀ. It says that every matrix has important input directions, output directions, and strengths. It also writes a matrix as a sum of rank-one layers ordered from strongest to weakest.

input directionsoutput directionssingular valuesrank-one layerslow-rank approximation

1. A 2D matrix sends the unit circle to an ellipse

Move the matrix sliders. The unit circle maps to an ellipse or collapses to a line. The semi-axis lengths are the singular values.

2. Rank-one layers build a matrix

A rank-one layer is a simple outer product pattern. Here a synthetic image-like matrix is built from three layers. Turn layers on and off.

The first layer usually captures broad structure. Later layers add detail.

3. Low-rank approximation

The original matrix below has several hidden layers plus noise. Choose how many layers to keep.

4. Denoising intuition

When signal is concentrated in the top layers and noise is spread out, truncated SVD can improve the image. But if k is too small, important details disappear.

5. PCA intuition from SVD

For centered data, SVD finds principal directions. Adjust the spread and rotation of the cloud.

Reflection prompts

  1. Why is the SVD a “microscope” rather than just a formula?
  2. What does a large singular value mean in an image? In a rating matrix?
  3. Why can small singular values make inverse problems unstable?
  4. How does PCA use SVD?
  5. When might low-rank approximation remove meaningful information?
Chapter 16 interactive lab — SVD: The Matrix Microscope