Modern interactive page · based on uploaded QMD

Singular Value Decomposition

SVD explains how every matrix rotates, stretches, compresses, approximates, denoises, and reveals low-rank structure.

Opening idea

Eigenvalues and eigenvectors reveal the special directions of a square matrix. But many important matrices are not square. A data matrix may have thousands of rows and hundreds of columns. An image matrix may be rectangular. A design matrix in least squares is often tall. A transformation may map one dimension into another. So we need a factorization that works for every matrix. The singular value decomposition, or SVD, does exactly that. For any real $m\times n$ matrix $A$, the SVD writes where: - $U$ has orthonormal columns, - $V$ has orthon...

Central formula

$$A=U\Sigma V^T,\qquad A\,v_i=\sigma_i u_i,\qquad A\approx\sum_{i=1}^r\sigma_i u_iv_i^T$$

Learning goals

Condensed from the chapter source.

  1. Define singular values and singular vectors.
  2. Explain the singular value decomposition $A=U\Sigma V^T$.
  3. Connect the SVD to eigenvalues of $A^TA$ and $AA^T$.
  4. Interpret singular values as stretching factors.
  5. Understand the four fundamental subspaces through the SVD.
  6. Compute the SVD using Python.
  7. Use the SVD to find rank and numerical rank.
  8. Construct low-rank approximations.
  9. Apply the SVD to image compression.
  10. Connect the SVD to PCA, least squares, data science, and AI.

Chapter map

Key ideas to keep visible while exploring the interactive tools.

Geometry

Unit circle to ellipse

SVD finds principal input and output directions.

Numerics

Singular values

Rank, condition number, and stability.

Data

Low-rank approximation

Compression and denoising keep the largest singular values.

2×2 SVD explorer

See how $A=U\Sigma V^T$ rotates, stretches, and rotates/refects again.

Matrix $A$

Unit circle mapped to ellipse

Rank, conditioning, and energy

Singular values measure directions of strength. Small singular values mean information is weak or lost.

Singular value spectrum

Energy bars

Synthetic image compression

A low-rank approximation keeps only the strongest outer-product patterns.

Rank kept2

Quick check

Geometric meaning of SVD.

Question. What do singular values measure?

They measure how much $A$ stretches special orthonormal input directions, the right singular vectors.

Chapter summary

The main takeaways in one place.

  • SVD exists for every real matrix, including rectangular matrices.
  • Right singular vectors are input directions; left singular vectors are output directions.
  • Singular values are stretching factors and reveal rank, conditioning, and information strength.
  • Low-rank approximation, PCA, compression, denoising, and pseudoinverses are all SVD ideas.
Generated as a standalone interactive HTML page. Equations use MathJax.