Most vectors change direction when a matrix acts on them.
For example, if
\[
A=
\begin{bmatrix}
2&1\\
1&2
\end{bmatrix},
\]
then a typical vector \(x\) is transformed into a new vector \(Ax\) pointing in a different direction.
But some special vectors keep their direction. They are only stretched, compressed, or reversed.
These special vectors are eigenvectors.
If
\[
Ax=\lambda x,
\]
then the matrix \(A\) acts on \(x\) in the simplest possible way: it only multiplies \(x\) by a scalar.
The scalar \(\lambda\) is the eigenvalue.
The central question of this chapter is:
Which directions does a matrix preserve?
Eigenvalues and eigenvectors reveal the natural directions of a linear transformation. They are essential for dynamical systems, diagonalization, Markov chains, stability, graph ranking, PCA, spectral clustering, and the SVD.
19.2 Learning goals
After studying this chapter, you should be able to:
Define eigenvalues and eigenvectors.
Check whether a vector is an eigenvector.
Interpret eigenvectors geometrically.
Derive the characteristic equation.
Compute eigenvalues of \(2\times 2\) matrices.
Find eigenspaces by solving homogeneous systems.
Understand algebraic and geometric multiplicity.
Recognize when a matrix may be diagonalizable.
Use Python to compute and visualize eigenvectors.
Connect eigenvalues to dynamics, stability, and data.
19.3 17.1 The eigenvector equation
Let \(A\) be an \(n\times n\) matrix.
A nonzero vector \(v\in\mathbb{R}^n\) is an eigenvector of \(A\) if there is a scalar \(\lambda\) such that
\[
Av=\lambda v.
\]
The scalar \(\lambda\) is called the corresponding eigenvalue.
NoteDefinition: eigenvalue and eigenvector
For an \(n\times n\) matrix \(A\), a nonzero vector \(v\) is an eigenvector with eigenvalue \(\lambda\) if
\[
Av=\lambda v.
\]
The vector \(v\) must be nonzero. The zero vector satisfies
\[
A0=\lambda 0
\]
for every \(\lambda\), so it is not allowed as an eigenvector.
19.4 17.2 Geometric meaning
The equation
\[
Av=\lambda v
\]
means that \(Av\) lies on the same line as \(v\).
The matrix does not rotate \(v\) away from its own direction. It only scales that direction.
There are three common cases:
If \(\lambda>1\), the vector is stretched.
If \(0<\lambda<1\), the vector is compressed.
If \(\lambda<0\), the vector is reversed and scaled.
If \(\lambda=0\), the vector is collapsed to zero.
So an eigenvector is a direction preserved by the linear transformation.
for j inrange(len(eigenvalues)): lam = eigenvalues[j] v = eigenvectors[:, j]print("lambda =", lam)print("A v =", A @ v)print("lambda v =", lam * v)print()
lambda = 3.0
A v = [2.12132034 2.12132034]
lambda v = [2.12132034 2.12132034]
lambda = 1.0
A v = [-0.70710678 0.70710678]
lambda v = [-0.70710678 0.70710678]
Because eigenvectors can be scaled, NumPy may return unit eigenvectors rather than the simple integer vectors.
For a \(2\times 2\) matrix, the eigenvalues satisfy:
\[
\lambda_1+\lambda_2=\operatorname{tr}(A),
\]
and
\[
\lambda_1\lambda_2=\det(A).
\]
More generally, for an \(n\times n\) matrix, the determinant is the product of eigenvalues and the trace is the sum of eigenvalues, counted with multiplicity.
Geometrically, this makes sense: a \(90^\circ\) rotation has no real direction that stays on the same line.
In this book, we will often focus on real eigenvalues and real eigenvectors, but complex eigenvalues are important in rotations, oscillations, and differential equations.
Power iteration is a simple algorithm for approximating a dominant eigenvector.
A = np.array([[2, 1], [1, 2]], dtype=float)x = np.array([1, 0], dtype=float)history = []for k inrange(10): x = A @ x x = x / np.linalg.norm(x) history.append(x.copy())np.array(history)
For each \(2\times 2\) matrix, compute trace, determinant, eigenvalues, and verify
\[
\lambda_1+\lambda_2=\operatorname{tr}(A),
\]
and
\[
\lambda_1\lambda_2=\det(A).
\]
\[
\begin{bmatrix}
2&1\\
1&2
\end{bmatrix}
\]
\[
\begin{bmatrix}
1&2\\
0&3
\end{bmatrix}
\]
\[
\begin{bmatrix}
4&1\\
2&3
\end{bmatrix}
\]
19.33.6 F. Dynamics
Let
\[
A=
\begin{bmatrix}
2&1\\
1&2
\end{bmatrix}.
\]
Find eigenvalues and eigenvectors.
Write
\[
x_0=
\begin{bmatrix}
2\\
0
\end{bmatrix}
\]
as a linear combination of eigenvectors. 3. Compute a formula for \(A^kx_0\). 4. Describe the dominant direction as \(k\) grows.
19.33.7 G. Python exercises
Use NumPy to compute eigenvalues and eigenvectors of several \(2\times 2\) matrices.
Use SymPy to compute exact characteristic polynomials.
Plot eigenvectors and their images under a matrix.
Use power iteration to approximate a dominant eigenvector.
Test a rotation matrix and observe its complex eigenvalues.
For a symmetric matrix, check that eigenvectors for distinct eigenvalues are orthogonal.
19.33.8 H. Reflection
Write a short paragraph answering each question.
Why are eigenvectors natural directions of a matrix?
Why does \(\det(A-\lambda I)=0\) find eigenvalues?
What is the difference between an eigenvector and an eigenspace?
Why do eigenvalues control repeated multiplication?
Why are eigenvalues important in data science and AI?
19.34 AI-age connection
Eigenvalues and eigenvectors are central to modern computation. They explain stability in dynamical systems, long-term behavior in Markov chains, ranking in networks, modes of vibration, principal directions in data, and the geometry of symmetric matrices. Spectral methods in graph learning, PCA, matrix factorization, and many AI algorithms all begin with the idea that a matrix has special directions that reveal its structure.
