Chapter 14 · Linear Algebra Computation

Gram--Schmidt, QR Factorization, and Numerical Orthogonality

Turning messy bases into orthonormal coordinates and QR factorizations

Gram–Schmidt Orthonormal basis QR factorization Numerical stability

Opening idea

In the previous chapters, orthogonality made geometry simpler. Projection onto a line was easy when the direction vector was known. Projection onto a subspace was especially simple when the basis was orthonormal: $$ \operatorname{proj}_W(b)=\sum_{j=1}^k (b\cdot q_j)q_j. $$ So a natural question appears: > Given a set of independent vectors, can we replace them with an orthonormal set that spans the same subspace? The answer is yes.

Big question. Turning messy bases into orthonormal coordinates and QR factorizations

Learning goals

Explain why orthonormal bases simplify projection.
Apply the Gram--Schmidt process to a set of independent vectors.
Construct an orthogonal basis from a given basis.
Normalize an orthogonal basis to obtain an orthonormal basis.
Understand the matrix factorization $A=QR$.
Compute a QR factorization by Gram--Schmidt.
Use QR factorization to compute projections.
Use QR factorization to solve least-squares problems.
Understand why numerical orthogonality matters.
Use Python to compute and check QR factorizations.

Core formulas

First vector

$$q_1=\frac{v_1}{\|v_1\|}$$

Remove previous shadows

$$u_k=v_k-\sum_{j=1}^{k-1}(v_k^Tq_j)q_j$$

Normalize

$$q_k=\frac{u_k}{\|u_k\|}$$

$$A=QR,\qquad R=Q^TA$$

Chapter map from the uploaded QMD

The interactive page follows the chapter sequence and turns key computational ideas into live tools.

14.1 Why orthonormal bases are valuable
14.2 The problem
14.3 The first step
14.4 The second step
14.5 The general Gram--Schmidt process
14.6 Example in R^2
14.7 Python plot: Gram--Schmidt in R^2
14.8 Example in R^3
14.9 Python implementation of Gram--Schmidt
14.10 From Gram--Schmidt to QR factorization
14.11 How to read A=QR
14.12 Reduced and full QR factorization
14.13 QR and projection
14.14 QR and least squares
14.15 Example: least squares by QR
14.16 Python: least squares by QR

Interactive Lab 1: Gram--Schmidt in $\mathbb R^2$

Change two input vectors. The tool subtracts the shadow of $v_2$ on $q_1$ to produce an orthogonal direction.

$v_{1x}$

$v_{1y}$

$v_{2x}$

$v_{2y}$

QR output

Interactive Lab 2: QR least squares

Fit $y=\beta_0+\beta_1x$ using QR on a tiny dataset.

Interactive Lab 3: Near-dependence stress test

As $\varepsilon$ gets small, the second vector becomes almost parallel to the first.

$\varepsilon$

Algorithmic summary

1Start with independent vectors $v_1,\dots,v_k$.
2Set $u_1=v_1$ and normalize to get $q_1$.
3For each new vector, subtract its projections onto all previous $q_j$.
4Normalize the remaining vector to get $q_k$.
5Collect $q_j$ as columns of $Q$ and compute $R=Q^TA$.

Common mistakes to avoid

Subtracting only one previous projection instead of all previous projections.
Forgetting to normalize after orthogonalization.
Confusing $Q^TQ=I$ with $QQ^T=I$ for rectangular $Q$.
Ignoring numerical instability when vectors are nearly dependent.

AI-age connection. QR factorization is a reliable computational backbone for least squares, regression, numerical optimization, and stable feature preprocessing.

Quick checks

What is the main computational object in this chapter?

Use the formulas and live labs above: identify the matrix/vector operation that turns the concept into computation.

What geometric picture should you keep in mind?

Follow the canvas visualization: a vector, subspace, transformed square, fitted line, or special direction depending on the chapter.

What should be checked after computation?

Check the defining property: orthogonality, $Q^TQ=I$, residual condition, determinant sign/size, or $Av=\lambda v$.