In earlier chapters, we studied spans, linear independence, bases, and dimension. These ideas naturally lead to one of the most important objects in linear algebra:
a subspace.
A subspace is a set of vectors that is closed under the operations of linear algebra. If we add vectors in the set, we stay in the set. If we multiply a vector in the set by a scalar, we stay in the set.
Subspaces are the natural habitats of linear algebra.
They appear as:
lines and planes through the origin,
solution sets of homogeneous systems,
spans of vectors,
column spaces of matrices,
nullspaces of matrices,
row spaces of matrices,
spaces of possible outputs,
spaces of invisible inputs.
The central question of this chapter is:
What kinds of sets behave like vector spaces inside \(\mathbb{R}^n\)?
10.2 Learning goals
After studying this chapter, you should be able to:
Define a subspace of \(\mathbb{R}^n\).
Test whether a subset of \(\mathbb{R}^n\) is a subspace.
Recognize spans as subspaces.
Recognize solution sets of homogeneous systems as subspaces.
Understand why nonhomogeneous solution sets are usually not subspaces.
Define the column space, nullspace, and row space of a matrix.
Find bases for column spaces using pivot columns.
Find bases for nullspaces using row reduction.
Use rank and nullity to describe dimensions.
Visualize subspaces in \(\mathbb{R}^2\) and \(\mathbb{R}^3\) using Python.
10.3 8.1 What is a subspace?
NoteDefinition: subspace of \(\mathbb{R}^n\)
A subset \(W\) of \(\mathbb{R}^n\) is a subspace of \(\mathbb{R}^n\) if:
\(0\in W\).
If \(u,v\in W\), then \(u+v\in W\).
If \(u\in W\) and \(c\in\mathbb{R}\), then \(cu\in W\).
The three conditions say:
the zero vector is included,
the set is closed under vector addition,
the set is closed under scalar multiplication.
These are exactly the operations used to form linear combinations.
Therefore, if \(W\) is a subspace and \(u,v\in W\), then every linear combination
\[
cu+dv
\]
also belongs to \(W\).
10.4 8.2 First examples of subspaces
10.4.1 The zero subspace
The set
\[
\{0\}
\]
is a subspace of \(\mathbb{R}^n\).
It contains the zero vector. Adding zero to zero gives zero. Multiplying zero by any scalar gives zero.
Its dimension is \(0\).
10.4.2 The whole space
The set
\[
\mathbb{R}^n
\]
is a subspace of itself.
Its dimension is \(n\).
10.4.3 Lines through the origin
A line through the origin in \(\mathbb{R}^2\) or \(\mathbb{R}^3\) is a subspace.
The span of any set of vectors is always a subspace.
10.10 8.8 Homogeneous systems create subspaces
A homogeneous system has the form
\[
Ax=0.
\]
The set of all solutions is a subspace of \(\mathbb{R}^n\).
Indeed, if \(u\) and \(v\) solve \(Ax=0\), then
\[
Au=0
\]
and
\[
Av=0.
\]
Then
\[
A(u+v)=Au+Av=0+0=0.
\]
So \(u+v\) is also a solution.
If \(c\) is a scalar, then
\[
A(cu)=cAu=c0=0.
\]
So \(cu\) is also a solution.
Therefore the solution set of a homogeneous system is a subspace.
This subspace is called the nullspace of \(A\).
10.11 8.9 Nonhomogeneous systems usually do not create subspaces
A nonhomogeneous system has the form
\[
Ax=b,
\]
where \(b\ne 0\).
Its solution set is usually not a subspace.
Why?
A subspace must contain the zero vector. But if \(b\ne 0\), then
\[
A0=0\ne b.
\]
So \(0\) is not a solution.
Therefore the solution set of \(Ax=b\) is not a subspace when \(b\ne 0\).
It may be a shifted subspace, also called an affine subspace. For example, a line not through the origin is not a subspace, but it is a shifted copy of a one-dimensional subspace.
10.12 8.10 Column space
NoteDefinition: column space
The column space of a matrix \(A\) is the span of the columns of \(A\).
10.20 8.18 Python: visualizing subspaces in \(\mathbb{R}^2\)
A line through the origin is a subspace of \(\mathbb{R}^2\).
import numpy as npimport matplotlib.pyplot as pltv = np.array([2, 1], dtype=float)t = np.linspace(-3, 3, 200)line = np.outer(t, v)plt.figure(figsize=(6, 5))plt.plot(line[:, 0], line[:, 1], label=r"$\operatorname{span}\{(2,1)\}$")plt.arrow(0, 0, v[0], v[1], head_width=0.1, length_includes_head=True)plt.text(v[0] +0.1, v[1], "$v$")plt.scatter([0], [0])plt.axhline(0, linewidth=0.8)plt.axvline(0, linewidth=0.8)plt.grid(True)plt.axis("equal")plt.title("A line through the origin is a subspace")plt.legend()plt.show()
A line not through the origin is not a subspace.
x = np.linspace(-3, 3, 200)y = x +1plt.figure(figsize=(6, 5))plt.plot(x, y, label="$y=x+1$")plt.scatter([0], [0], label="origin")plt.axhline(0, linewidth=0.8)plt.axvline(0, linewidth=0.8)plt.grid(True)plt.axis("equal")plt.title("A line not through the origin is not a subspace")plt.legend()plt.show()
10.21 8.19 Python: visualizing a plane subspace in \(\mathbb{R}^3\)
The equation
\[
x+2y-z=0
\]
defines a plane through the origin, hence a subspace.
Solve for \(z\):
\[
z=x+2y.
\]
from mpl_toolkits.mplot3d import Axes3Dx = np.linspace(-2, 2, 20)y = np.linspace(-2, 2, 20)X, Y = np.meshgrid(x, y)Z = X +2*Yfig = plt.figure(figsize=(7, 6))ax = fig.add_subplot(111, projection="3d")ax.plot_surface(X, Y, Z, alpha=0.5)ax.scatter([0], [0], [0])ax.set_xlabel("$x$")ax.set_ylabel("$y$")ax.set_zlabel("$z$")ax.set_title("The plane x + 2y - z = 0 is a subspace")plt.show()
The equation
\[
x+2y-z=1
\]
defines a parallel plane not passing through the origin. It is not a subspace.
10.22 8.20 Python: column space and nullspace with SymPy
Why are subspaces the natural setting for linear algebra?
How does a column space describe possible outputs?
How does a nullspace describe invisible inputs?
Why is rank-nullity a conservation law for dimension?
How do subspaces help organize data and computation?
10.28 AI-age connection
Subspaces are essential in data science and AI because high-dimensional data often lives near lower-dimensional structure. A collection of images, signals, or feature vectors may appear to occupy a huge ambient space, but much of its meaningful variation may lie in a smaller subspace or near a low-dimensional manifold. Column spaces describe what a model can produce. Nullspaces describe what a model cannot detect. Rank and nullity measure visible and invisible dimensions. These ideas prepare the way for least squares, projections, PCA, and the SVD.