Chapter 13 · Linear Algebra Computation

Orthogonal Projection

Closest vectors, shadows, errors, and projection matrices

Projection onto a line Projection matrices Normal equations Closest point

Opening idea

Suppose a vector $b$ does not lie on a line or inside a subspace. We may still want the closest vector in that line or subspace. This is the idea of projection. Projection answers questions such as: - What is the shadow of a vector on a line? - What is the closest point in a subspace? - How can we split a vector into useful and error parts? - How do we approximate inconsistent systems?

Big question. Closest vectors, shadows, errors, and projection matrices

Learning goals

Define orthogonal projection onto a line.
Derive the projection formula onto a line.
Decompose a vector into parallel and perpendicular parts.
Interpret projection as the closest point problem.
Define projection onto a subspace.
Use an orthonormal basis to compute projection.
Build projection matrices.
Understand the normal equations.
Relate projection to least squares.
Use Python to visualize and compute projections.

Core formulas

Line projection

$$\operatorname{proj}_{\operatorname{span}\{a\}}(b)=\frac{a^Tb}{a^Ta}a$$

Parallel plus perpendicular

$$b=\operatorname{proj}_V(b)+(b-\operatorname{proj}_V(b)),\quad b-\operatorname{proj}_V(b)\perp V$$

Orthonormal columns

$$P=QQ^T,\qquad \operatorname{proj}_V(b)=QQ^Tb$$

Full column rank

$$P=A(A^TA)^{-1}A^T,\qquad A^T(b-A\hat x)=0$$

Chapter map from the uploaded QMD

The interactive page follows the chapter sequence and turns key computational ideas into live tools.

13.1 Projection onto a line
13.2 Geometry of projection
13.3 Example: projection onto a line in R^2
13.4 Python plot: projection onto a line
13.5 Parallel plus perpendicular decomposition
13.6 Projection using a unit vector
13.7 Projection matrix onto a line
13.8 Example: projection matrix onto a line
13.9 Properties of projection matrices
13.10 Closest point interpretation
13.11 Projection onto a subspace with an orthonormal basis
13.12 Matrix formula with orthonormal columns
13.13 Example: projection onto a plane in R^3
13.14 Python plot: projection onto the xy-plane
13.15 Projection onto a subspace with a nonorthonormal basis
13.16 Projection matrix for a full-rank matrix

Interactive Lab 1: Projection onto a line

Move the direction vector $a$ and the target vector $b$. The page computes the shadow $p=\operatorname{proj}_L(b)$ and the perpendicular error $e=b-p$.

$a_1$

$a_2$

$b_1$

$b_2$

Live computation

distance0

$a^Te$0

coefficient0

Interactive Lab 2: Projection matrix

For a line $L=\operatorname{span}\{a\}$, the projection matrix is $P=aa^T/(a^Ta)$.

Interactive Lab 3: Closest point scan

Choose a point $xa$ on the line. The distance curve is minimized at $x=(a^Tb)/(a^Ta)$.

test coefficient $x$

Algorithmic summary

1Choose a spanning matrix $A$ for the subspace.
2If columns are orthonormal, use $P=QQ^T$.
3If columns are independent but not orthonormal, solve $A^TA\hat x=A^Tb$.
4Compute $p=A\hat x$ and residual $e=b-p$.
5Check $A^Te=0$.

Common mistakes to avoid

Using $AA^T$ when columns are not orthonormal.
Forgetting that the residual, not the projection, is orthogonal to the subspace.
Computing an explicit inverse when a solver is more stable.
Assuming projection error is always zero.

AI-age connection. Projection is the geometric engine behind least squares, regression, denoising, embeddings, recommender systems, and many optimization algorithms in AI.

Quick checks

What is the main computational object in this chapter?

Use the formulas and live labs above: identify the matrix/vector operation that turns the concept into computation.

What geometric picture should you keep in mind?

Follow the canvas visualization: a vector, subspace, transformed square, fitted line, or special direction depending on the chapter.

What should be checked after computation?

Check the defining property: orthogonality, $Q^TQ=I$, residual condition, determinant sign/size, or $Av=\lambda v$.