Opening question
In Chapter 6, we learned that span describes what vectors can be built from a collection of vectors.
But a spanning set may contain redundancy.
For example,
\[
\left\{
\begin{bmatrix}1\\0\end{bmatrix},
\begin{bmatrix}0\\1\end{bmatrix},
\begin{bmatrix}1\\1\end{bmatrix}
\right\}
\]
spans \(\mathbb{R}^2\) , but the third vector is not really needed because
\[
\begin{bmatrix}1\\1\end{bmatrix}
=
\begin{bmatrix}1\\0\end{bmatrix}
+
\begin{bmatrix}0\\1\end{bmatrix}.
\]
This chapter studies how to remove redundancy without losing span.
The key ideas are:
linear independence : no vector is redundant,basis : a minimal spanning set,dimension : the number of directions needed.
The central question is:
What is the smallest number of vectors needed to describe a space completely?
Learning goals
After studying this chapter, you should be able to:
Define linear dependence and linear independence.
Test whether vectors are linearly independent using row reduction.
Interpret dependence geometrically.
Understand the relationship between span and independence.
Define a basis for a subspace of \(\mathbb{R}^n\) .
Find bases from spanning sets.
Find bases for column spaces using pivot columns.
Define dimension.
Use the basis theorem.
Use Python and SymPy to test independence and extract bases.
7.1 Redundancy in a spanning set
Consider the vectors
\[
v_1=
\begin{bmatrix}
1\\
0
\end{bmatrix},
\qquad
v_2=
\begin{bmatrix}
0\\
1
\end{bmatrix},
\qquad
v_3=
\begin{bmatrix}
1\\
1
\end{bmatrix}.
\]
The set \(\{v_1,v_2,v_3\}\) spans \(\mathbb{R}^2\) , but \(v_3\) is redundant because
\[
v_3=v_1+v_2.
\]
So any linear combination involving \(v_3\) can be rewritten using only \(v_1\) and \(v_2\) .
For example,
\[
2v_1+3v_2+5v_3
=
2v_1+3v_2+5(v_1+v_2)
=
7v_1+8v_2.
\]
The vector \(v_3\) adds no new direction.
This is the motivation for linear independence.
7.2 Linear dependence
Vectors \(v_1,v_2,\ldots,v_k\) are linearly dependent if there exist scalars \(c_1,c_2,\ldots,c_k\) , not all zero, such that
\[
c_1v_1+c_2v_2+\cdots+c_kv_k=0.
\]
The phrase not all zero is essential. The equation
\[
0v_1+0v_2+\cdots+0v_k=0
\]
is always true and gives no information.
A dependence relation is a nontrivial way to combine the vectors and get the zero vector.
Example
Let
\[
v_1=
\begin{bmatrix}
1\\
0
\end{bmatrix},
\qquad
v_2=
\begin{bmatrix}
0\\
1
\end{bmatrix},
\qquad
v_3=
\begin{bmatrix}
1\\
1
\end{bmatrix}.
\]
Then
\[
v_1+v_2-v_3=0.
\]
The coefficients are
\[
1,\quad 1,\quad -1,
\]
which are not all zero. Therefore the vectors are linearly dependent.
7.3 Linear independence
Vectors \(v_1,v_2,\ldots,v_k\) are linearly independent if the equation
\[
c_1v_1+c_2v_2+\cdots+c_kv_k=0
\]
has only the trivial solution
\[
c_1=c_2=\cdots=c_k=0.
\]
This means no vector in the set can be built from the others.
Linear independence means:
no redundancy.
Example
The vectors
\[
e_1=
\begin{bmatrix}
1\\
0
\end{bmatrix},
\qquad
e_2=
\begin{bmatrix}
0\\
1
\end{bmatrix}
\]
are linearly independent.
If
\[
c_1e_1+c_2e_2=0,
\]
then
\[
c_1
\begin{bmatrix}
1\\
0
\end{bmatrix}
+
c_2
\begin{bmatrix}
0\\
1
\end{bmatrix}
=
\begin{bmatrix}
c_1\\
c_2
\end{bmatrix}
=
\begin{bmatrix}
0\\
0
\end{bmatrix}.
\]
Therefore
\[
c_1=0,\qquad c_2=0.
\]
So the only solution is the trivial solution.
7.4 Testing independence using a matrix
To test whether vectors \(v_1,\ldots,v_k\) are linearly independent, place them as columns of a matrix:
\[
A=\begin{bmatrix}v_1&v_2&\cdots&v_k\end{bmatrix}.
\]
Then solve the homogeneous system
\[
Ac=0.
\]
Here
\[
c=
\begin{bmatrix}
c_1\\
c_2\\
\vdots\\
c_k
\end{bmatrix}.
\]
The vectors are linearly independent if and only if the homogeneous system has only the trivial solution.
They are linearly dependent if and only if the homogeneous system has a nontrivial solution.
The columns of \(A\) are linearly independent if and only if
\[
Ax=0
\]
has only the zero solution.
7.5 Row-reduction criterion
Row reduction gives a practical test.
The columns of a matrix \(A\) are linearly independent if and only if every column of \(A\) is a pivot column.
Equivalently:
if every variable is a pivot variable, the columns are independent,
if at least one variable is free, the columns are dependent.
Why? A free variable in \(Ax=0\) creates a nonzero solution, which gives a dependence relation among the columns.
7.6 Example: independent vectors in \(\mathbb{R}^3\)
Determine whether
\[
v_1=
\begin{bmatrix}
1\\
0\\
1
\end{bmatrix},
\qquad
v_2=
\begin{bmatrix}
0\\
1\\
1
\end{bmatrix},
\qquad
v_3=
\begin{bmatrix}
1\\
1\\
0
\end{bmatrix}
\]
are linearly independent.
Place them as columns:
\[
A=
\begin{bmatrix}
1&0&1\\
0&1&1\\
1&1&0
\end{bmatrix}.
\]
Solve
\[
Ac=0.
\]
The augmented matrix is
\[
\left[
\begin{array}{ccc|c}
1&0&1&0\\
0&1&1&0\\
1&1&0&0
\end{array}
\right].
\]
Row reduction gives
\[
\left[
\begin{array}{ccc|c}
1&0&0&0\\
0&1&0&0\\
0&0&1&0
\end{array}
\right].
\]
Every column is a pivot column. Therefore the vectors are linearly independent.
7.7 Example: dependent vectors in \(\mathbb{R}^3\)
Determine whether
\[
v_1=
\begin{bmatrix}
1\\
2\\
3
\end{bmatrix},
\qquad
v_2=
\begin{bmatrix}
2\\
4\\
6
\end{bmatrix},
\qquad
v_3=
\begin{bmatrix}
1\\
0\\
1
\end{bmatrix}
\]
are linearly independent.
Since
\[
v_2=2v_1,
\]
the vectors are linearly dependent.
A dependence relation is
\[
2v_1-v_2+0v_3=0.
\]
The coefficients are not all zero, so the set is dependent.
The matrix with these vectors as columns is
\[
A=
\begin{bmatrix}
1&2&1\\
2&4&0\\
3&6&1
\end{bmatrix}.
\]
Since the second column is twice the first, the second column cannot be a pivot column.
7.8 Geometry of independence in \(\mathbb{R}^2\)
In \(\mathbb{R}^2\) :
one nonzero vector is independent,
two vectors are independent if they are not on the same line through the origin,
three or more vectors are always dependent.
Why are three vectors in \(\mathbb{R}^2\) always dependent?
Because \(\mathbb{R}^2\) has only two independent directions. A third vector must be built from the first two if they already span the plane, or it lies in the same line if they do not.
import numpy as npimport matplotlib.pyplot as pltdef draw_vectors(vectors, labels, title):= (6 , 6 ))for v, label in zip (vectors, labels):0 , 0 , v[0 ], v[1 ], head_width= 0.08 , length_includes_head= True )0 ] + 0.08 , v[1 ] + 0.08 , label)0 , linewidth= 0.8 )0 , linewidth= 0.8 )True )"equal" )- 1 , 4 )- 1 , 4 )= np.array([1 , 2 ])= np.array([2 , 1 ])"$v_1$" , "$v_2$" ], "Two independent vectors in R^2" )= np.array([1 , 2 ])= np.array([2 , 4 ])"$w_1$" , "$w_2=2w_1$" ], "Two dependent vectors in R^2" )
7.9 Geometry of independence in \(\mathbb{R}^3\)
In \(\mathbb{R}^3\) :
one nonzero vector is independent,
two vectors are independent if they are not multiples of each other,
three vectors are independent if they do not lie in the same plane through the origin,
four or more vectors are always dependent.
Three independent vectors in \(\mathbb{R}^3\) provide three independent directions and span all of \(\mathbb{R}^3\) .
Three dependent vectors lie in a plane through the origin or have more obvious redundancy.
from mpl_toolkits.mplot3d import Axes3D= np.array([1 , 0 , 0 ], dtype= float )= np.array([0 , 1 , 0 ], dtype= float )= np.array([0 , 0 , 1 ], dtype= float )= plt.figure(figsize= (7 , 6 ))= fig.add_subplot(111 , projection= "3d" )for v, label in [(v1, "$e_1$" ), (v2, "$e_2$" ), (v3, "$e_3$" )]:0 , 0 , 0 , v[0 ], v[1 ], v[2 ], length= 1 , normalize= False )0 ], v[1 ], v[2 ], label)0 , 1.2 )0 , 1.2 )0 , 1.2 )"$x$" )"$y$" )"$z$" )"Three independent directions in R^3" )
7.10 Too many vectors guarantee dependence
A set of \(k\) vectors in \(\mathbb{R}^n\) is automatically linearly dependent if
\[
k>n.
\]
Why?
When testing independence, we solve
\[
Ac=0,
\]
where \(A\) is an \(n\times k\) matrix.
If \(k>n\) , then there are more variables than equations. Therefore there must be at least one free variable, so there is a nontrivial solution.
Thus the columns are dependent.
Any set of more than \(n\) vectors in \(\mathbb{R}^n\) is linearly dependent.
7.11 Sets containing the zero vector
Any set containing the zero vector is linearly dependent.
Suppose
\[
v_1=0.
\]
Then
\[
1v_1+0v_2+\cdots+0v_k=0.
\]
This is a nontrivial dependence relation because the first coefficient is \(1\) .
Therefore the set is dependent.
A set containing the zero vector is always linearly dependent.
7.12 Basis
Span and independence are two complementary ideas.
Span means the vectors are enough to build the space.
Independence means none of the vectors are redundant.
A basis has both properties.
A basis for a subspace \(W\) of \(\mathbb{R}^n\) is a set of vectors in \(W\) that:
spans \(W\) ,
is linearly independent.
A basis is a coordinate system for a space.
It gives a way to describe every vector in the space using exactly one set of coefficients.
7.13 The standard basis
The standard basis of \(\mathbb{R}^n\) is
\[
e_1=
\begin{bmatrix}
1\\
0\\
\vdots\\
0
\end{bmatrix},
\quad
e_2=
\begin{bmatrix}
0\\
1\\
\vdots\\
0
\end{bmatrix},
\quad
\ldots,
\quad
e_n=
\begin{bmatrix}
0\\
0\\
\vdots\\
1
\end{bmatrix}.
\]
Every vector
\[
x=
\begin{bmatrix}
x_1\\
x_2\\
\vdots\\
x_n
\end{bmatrix}
\]
can be written as
\[
x=x_1e_1+x_2e_2+\cdots+x_ne_n.
\]
The representation is unique.
Therefore
\[
\{e_1,e_2,\ldots,e_n\}
\]
is a basis for \(\mathbb{R}^n\) .
7.14 Nonstandard bases
A basis does not have to be the standard basis.
For example,
\[
v_1=
\begin{bmatrix}
1\\
1
\end{bmatrix},
\qquad
v_2=
\begin{bmatrix}
1\\
-1
\end{bmatrix}
\]
form a basis of \(\mathbb{R}^2\) .
They span \(\mathbb{R}^2\) because they point in different directions.
They are independent because neither vector is a scalar multiple of the other.
Every vector in \(\mathbb{R}^2\) can be written uniquely as a combination
\[
c_1v_1+c_2v_2.
\]
For example, to write
\[
b=
\begin{bmatrix}
3\\
1
\end{bmatrix}
\]
in this basis, solve
\[
c_1
\begin{bmatrix}
1\\
1
\end{bmatrix}
+
c_2
\begin{bmatrix}
1\\
-1
\end{bmatrix}
=
\begin{bmatrix}
3\\
1
\end{bmatrix}.
\]
This gives
\[
c_1+c_2=3,
\]
\[
c_1-c_2=1.
\]
Solving,
\[
c_1=2,\qquad c_2=1.
\]
Therefore
\[
\begin{bmatrix}
3\\
1
\end{bmatrix}
=
2
\begin{bmatrix}
1\\
1
\end{bmatrix}
+
1
\begin{bmatrix}
1\\
-1
\end{bmatrix}.
\]
7.15 Dimension
The dimension of a subspace \(W\) is the number of vectors in any basis for \(W\) .
This definition makes sense because all bases of the same subspace have the same number of vectors.
Examples:
A line through the origin has dimension \(1\) .
A plane through the origin in \(\mathbb{R}^3\) has dimension \(2\) .
\(\mathbb{R}^n\) has dimension \(n\) .The zero subspace \(\{0\}\) has dimension \(0\) .
Dimension counts the number of independent directions in a space.
7.16 The basis theorem
Let \(W\) be a subspace of dimension \(p\) .
Any linearly independent set of \(p\) vectors in \(W\) is a basis for \(W\) .
Any spanning set of \(p\) vectors for \(W\) is a basis for \(W\) .
In \(\mathbb{R}^n\) , this means:
\(n\) independent vectors automatically form a basis of \(\mathbb{R}^n\) ,\(n\) vectors that span \(\mathbb{R}^n\) automatically form a basis of \(\mathbb{R}^n\) .
This theorem is powerful because it reduces the work needed to prove that a set is a basis.
7.17 Finding a basis from pivot columns
Suppose
\[
A=
\begin{bmatrix}
1&2&0&3\\
0&1&1&1\\
1&3&1&4
\end{bmatrix}.
\]
The columns are
\[
a_1,a_2,a_3,a_4.
\]
To find a basis for the column space, row reduce \(A\) .
Suppose the rref is
\[
\operatorname{rref}(A)=
\begin{bmatrix}
1&0&-2&1\\
0&1&1&1\\
0&0&0&0
\end{bmatrix}.
\]
The pivot columns are columns \(1\) and \(2\) .
Then the corresponding original columns
\[
a_1=
\begin{bmatrix}
1\\
0\\
1
\end{bmatrix},
\qquad
a_2=
\begin{bmatrix}
2\\
1\\
3
\end{bmatrix}
\]
form a basis for \(\operatorname{Col}(A)\) .
Use the pivot columns of the original matrix, not the pivot columns of the rref matrix, as a basis for the column space.
7.18 Why original pivot columns?
Row operations change the columns of a matrix, so the column space usually changes during row reduction.
However, row reduction identifies which original columns are independent and which original columns are redundant.
Therefore:
row reduce to find pivot column positions,
go back to the original matrix,
select the original columns in those positions.
Those original pivot columns form a basis for the column space.
7.19 Coordinates relative to a basis
If
\[
\mathcal{B}=\{v_1,v_2,\ldots,v_p\}
\]
is a basis for a subspace \(W\) , then every vector \(w\in W\) can be written uniquely as
\[
w=c_1v_1+c_2v_2+\cdots+c_pv_p.
\]
The scalars
\[
c_1,c_2,\ldots,c_p
\]
are the coordinates of \(w\) relative to the basis \(\mathcal{B}\) .
We write
\[
[w]_{\mathcal{B}}=
\begin{bmatrix}
c_1\\
c_2\\
\vdots\\
c_p
\end{bmatrix}.
\]
Coordinates will be studied in detail in Chapter 9.
7.20 Python: testing independence
import sympy as sp= sp.Matrix([1 , 0 , 1 ],0 , 1 , 1 ],1 , 1 , 0 ]
(Matrix([
[1, 0, 0],
[0, 1, 0],
[0, 0, 1]]),
(0, 1, 2))
If every column is a pivot column, the columns are independent.
Now test a dependent set:
= sp.Matrix([1 , 2 , 1 ],2 , 4 , 0 ],3 , 6 , 1 ]
(Matrix([
[1, 2, 0],
[0, 0, 1],
[0, 0, 0]]),
(0, 2))
Since there is a non-pivot column, the columns are dependent.
7.21 Python: finding pivot columns
= sp.Matrix([1 , 2 , 0 , 3 ],0 , 1 , 1 , 1 ],1 , 3 , 1 , 4 ]= A.rref()
(Matrix([
[1, 0, -2, 1],
[0, 1, 1, 1],
[0, 0, 0, 0]]),
(0, 1))
The tuple pivots gives the pivot column indices using Python’s zero-based indexing.
If pivots returns (0, 1), then the pivot columns are columns \(1\) and \(2\) in mathematical indexing.
To extract the original pivot columns:
= [A[:, j] for j in pivots]
[Matrix([
[1],
[0],
[1]]),
Matrix([
[2],
[1],
[3]])]
These columns form a basis for the column space of \(A\) .
7.22 Python: visualizing a basis in \(\mathbb{R}^2\)
import numpy as npimport matplotlib.pyplot as plt= np.array([1 , 1 ], dtype= float )= np.array([1 , - 1 ], dtype= float )= np.linspace(- 2 , 2 , 9 )= []for c1 in coeffs:for c2 in coeffs:* v1 + c2* v2)= np.array(points)= (6 , 6 ))0 ], points[:, 1 ], s= 20 )0 , 0 , v1[0 ], v1[1 ], head_width= 0.08 , length_includes_head= True )0 , 0 , v2[0 ], v2[1 ], head_width= 0.08 , length_includes_head= True )0 ] + 0.1 , v1[1 ], "$v_1$" )0 ] + 0.1 , v2[1 ], "$v_2$" )0 , linewidth= 0.8 )0 , linewidth= 0.8 )True )"equal" )"A nonstandard basis for R^2" )
7.23 Algorithmic summary
To test whether \(v_1,\ldots,v_k\) are linearly independent:
Form the matrix
\[
A=\begin{bmatrix}v_1&v_2&\cdots&v_k\end{bmatrix}.
\]
Row reduce \(A\) .
If every column is a pivot column, the vectors are independent.
If some column is not a pivot column, the vectors are dependent.
To find a basis for \(\operatorname{Col}(A)\) :
Row reduce \(A\) .
Identify the pivot column positions.
Select the corresponding columns of the original matrix \(A\) .
Those original pivot columns form a basis for \(\operatorname{Col}(A)\) .
7.24 Common mistakes
Mistake 1: confusing span and independence
A set can span but still be dependent.
A set can be independent but not span the whole space.
A basis must do both.
Mistake 2: forgetting the nontrivial condition
Linear dependence requires coefficients not all zero.
The all-zero coefficients always produce the zero vector.
Mistake 3: using rref columns as a column-space basis
The pivot columns of the original matrix form a basis for the column space.
Do not use the columns of the rref matrix as a basis for the original column space.
Mistake 4: thinking more vectors is always better
More vectors can create redundancy. A basis is efficient because it has no unnecessary vectors.
Mistake 5: thinking dimension depends on the basis
Different bases may look different, but they always have the same number of vectors.
That number is the dimension.
7.25 Chapter summary
Vectors \(v_1,\ldots,v_k\) are linearly dependent if there is a nontrivial relation
\[
c_1v_1+\cdots+c_kv_k=0.
\]
They are linearly independent if the only solution to
\[
c_1v_1+\cdots+c_kv_k=0
\]
is
\[
c_1=\cdots=c_k=0.
\]
To test independence, place the vectors as columns of a matrix and row reduce.
A basis is a set of vectors that spans a space and is linearly independent.
Dimension is the number of vectors in a basis.
The pivot columns of the original matrix form a basis for the column space.
A basis provides a minimal, nonredundant coordinate system for a space.
Exercises
A. Dependence relations
For each set of vectors, determine whether the set is linearly dependent. If dependent, give a dependence relation.
\[
\left\{
\begin{bmatrix}1\\0\end{bmatrix},
\begin{bmatrix}0\\1\end{bmatrix},
\begin{bmatrix}1\\1\end{bmatrix}
\right\}
\]
\[
\left\{
\begin{bmatrix}1\\2\end{bmatrix},
\begin{bmatrix}2\\4\end{bmatrix}
\right\}
\]
\[
\left\{
\begin{bmatrix}1\\0\\1\end{bmatrix},
\begin{bmatrix}0\\1\\1\end{bmatrix},
\begin{bmatrix}1\\1\\0\end{bmatrix}
\right\}
\]
\[
\left\{
\begin{bmatrix}1\\2\\3\end{bmatrix},
\begin{bmatrix}2\\4\\6\end{bmatrix},
\begin{bmatrix}1\\0\\1\end{bmatrix}
\right\}
\]
B. Independence by row reduction
Determine whether the columns of each matrix are linearly independent.
\[
\begin{bmatrix}
1&0\\
0&1
\end{bmatrix}
\]
\[
\begin{bmatrix}
1&2\\
2&4
\end{bmatrix}
\]
\[
\begin{bmatrix}
1&0&1\\
0&1&1\\
1&1&0
\end{bmatrix}
\]
\[
\begin{bmatrix}
1&2&0&3\\
0&1&1&1\\
1&3&1&4
\end{bmatrix}
\]
C. Bases of \(\mathbb{R}^n\)
Determine whether each set is a basis for the indicated space.
\[
\left\{
\begin{bmatrix}1\\0\end{bmatrix},
\begin{bmatrix}0\\1\end{bmatrix}
\right\}
\]
for \(\mathbb{R}^2\) .
\[
\left\{
\begin{bmatrix}1\\1\end{bmatrix},
\begin{bmatrix}1\\-1\end{bmatrix}
\right\}
\]
for \(\mathbb{R}^2\) .
\[
\left\{
\begin{bmatrix}1\\2\end{bmatrix},
\begin{bmatrix}2\\4\end{bmatrix}
\right\}
\]
for \(\mathbb{R}^2\) .
\[
\left\{
\begin{bmatrix}1\\0\\0\end{bmatrix},
\begin{bmatrix}0\\1\\0\end{bmatrix},
\begin{bmatrix}0\\0\\1\end{bmatrix}
\right\}
\]
for \(\mathbb{R}^3\) .
D. Column-space bases
Find a basis for the column space of each matrix.
\[
A=
\begin{bmatrix}
1&2&0&3\\
0&1&1&1\\
1&3&1&4
\end{bmatrix}
\]
\[
B=
\begin{bmatrix}
1&2&3\\
2&4&6\\
1&1&1
\end{bmatrix}
\]
\[
C=
\begin{bmatrix}
1&0&2&1\\
0&1&3&-1\\
0&0&0&0
\end{bmatrix}
\]
E. Dimension
Give the dimension of each space.
A line through the origin in \(\mathbb{R}^3\) .
A plane through the origin in \(\mathbb{R}^3\) .
\(\mathbb{R}^5\) .The zero subspace \(\{0\}\) .
The span of
\[
\left\{
\begin{bmatrix}1\\2\\3\end{bmatrix},
\begin{bmatrix}2\\4\\6\end{bmatrix}
\right\}.
\]
F. Coordinates in a basis
Let
\[
\mathcal{B}=
\left\{
\begin{bmatrix}1\\1\end{bmatrix},
\begin{bmatrix}1\\-1\end{bmatrix}
\right\}.
\]
Find the coordinate vector \([b]_{\mathcal{B}}\) for each vector.
\[
b=
\begin{bmatrix}
3\\
1
\end{bmatrix}
\]
\[
b=
\begin{bmatrix}
4\\
0
\end{bmatrix}
\]
\[
b=
\begin{bmatrix}
0\\
2
\end{bmatrix}
\]
G. Python exercises
Use SymPy to test whether the columns of
\[
A=
\begin{bmatrix}
1&0&1\\
0&1&1\\
1&1&0
\end{bmatrix}
\]
are independent.
Use SymPy to test whether the columns of
\[
B=
\begin{bmatrix}
1&2&1\\
2&4&0\\
3&6&1
\end{bmatrix}
\]
are independent.
Use SymPy to find pivot columns of
\[
C=
\begin{bmatrix}
1&2&0&3\\
0&1&1&1\\
1&3&1&4
\end{bmatrix}.
\]
Extract the original pivot columns and explain why they form a basis for \(\operatorname{Col}(C)\) .
Plot two nonstandard basis vectors in \(\mathbb{R}^2\) and generate a grid of their linear combinations.
H. Reflection
Write a short paragraph answering each question.
Why does linear independence mean no redundancy?
Why does a basis need both span and independence?
Why is dimension the number of independent directions?
Why are pivot columns useful for finding bases?
How does a basis act like a coordinate system?
AI-age connection
Linear independence and bases are the mathematical language of removing redundancy. In data science and AI, high-dimensional data often contains repeated, correlated, or unnecessary features. A basis gives an efficient set of directions for representing information. Later, QR factorization, eigenvectors, PCA, and the SVD will build on the same idea: find useful directions, remove redundancy, and represent data with fewer but more meaningful coordinates.
