Linear Algebra in the AI Age: Geometry, Computation, and Data

Book for: MATH 5110 Applied Linear Algebra and Matrix Analysis

Author

Affiliation

He Wang

Department of Mathematics, Northeastern University.

Welcome

Welcome to MATH 5110 Applied Linear Algebra and Matrix Analysis.

This online book is an applied, story-driven introduction to linear algebra and matrix analysis for graduate students in MS Applied Mathematics and MS Statistics at Northeastern University. It is designed for graduate students studying applied linear algebra, matrix analysis, computation, data science, and artificial intelligence. The goal is to present linear algebra not only as a collection of techniques, but also as a language for understanding modern data, algorithms, and mathematical modeling.

Course Information

Course: MATH 5110 Applied Linear Algebra and Matrix Analysis
Instructor: He Wang
Department: Department of Mathematics
University: Northeastern University
Personal Website: He Wang’s Website
Course Website: MATH 5110 Course Website
Github: Github Account

About the Author

He Wang is an Associate Teaching Professor in the Department of Mathematics at Northeastern University. He serves as Program Director for several graduate programs, including MS Applied Mathematics, MS Statistics, and MS Mathematics.

His teaching and academic interests include algebraic topology, mathematical foundations of data science and AI, optimization, and computational mathematics.

About This Book

This book develops linear algebra from foundational ideas to modern applications. Topics include systems of linear equations, matrix algebra, vector spaces, linear transformations, orthogonality, eigenvalues, singular value decomposition, optimization, Markov chains, and applications to data science and machine learning.

The book includes:

• Lecture-style chapters
• Independent study labs
• Python computation examples in each chapter (MATLAB examples will be added in the future.)
• Interactive HTML activities
• Conceptual explanations
• Applications to data, computation, and AI

This book is based on my teaching materials of MATH 5110 for graduate students, motivated by MATH 7243 and MATH 7339 Machine Learning and Statistical Learning Theory 1 and 2. The ebook was written with the help of ChatGPT, Claude, and Gemini, and built with Quarto

Overview:

This book is an applied, story-driven introduction to linear algebra and matrix analysis for graduate students. The guiding idea is simple:

Linear algebra is the language of structure, computation, geometry, data, and modern intelligent systems.

We begin with the concrete problem of solving linear equations, then gradually uncover the abstract objects behind the computations: vector spaces, linear transformations, bases, coordinates, eigenvectors, inner products, projections, factorizations, spectral theory, and modern applications such as Markov chains, least squares, PCA, Fourier analysis, wavelets, graph theory, optimization, probability, and matrix calculus.

The course is designed to move back and forth between four viewpoints:

1. Algebraic: equations, matrices, subspaces, transformations, determinants, eigenvalues.
2. Geometric: dimension, coordinates, orthogonality, projections, distance, angles.
3. Computational: row reduction, LU, QR, Schur, SVD, FFT, sparse matrices, algorithms.
4. Applied/data-driven: least squares, compression, PCA, Markov chains, graph networks, optimization, Topological Data Analysis, probability, and AI.

A conceptual map

flowchart TD
  X[Set and Maps]
  X --> Y[Monoid, Group, Ring, Fiesds]
  Y --> C[Vector spaces and linear transformations]
  C --> Z[Subspace, Sum, Quotient]
  E --> U[Topological Data Analysis]
  E --> B[Matrix algebra]
  B --> D[Row operations, LU, determinant, rank]
  Z --> E[Basis, coordinates, dimension]
  E --> F[Eigenvalues and eigenspaces]
  F --> B
  D --> A[Linear systems] 
  A --> W[Differential Equations]
  F --> G[Diagonalization and Jordan form]
  G --> H[Dynamical systems, Markov chains, PageRank]
  E --> I[Inner Product Spaces - Geometry] 
  F --> I
  I --> J[Orthogonal projection and orthogonal bases]
  J --> K[Least squares]
  J --> L[QR and Schur]
  J --> M[Spectral theorem]
  M --> N[SVD and PCA]
  J --> O[FFT and wavelets]
  G --> Q[Spectral graph theory]
  I --> R[Hilbert spaces]
  E --> S[Multilinear algebra]
  D --> T[Optimization]
  D --> P[matrix calculus]
  D --> V[Random Variables]

flowchart TD
  X[Set and Maps]
  X --> Y[Monoid, Group, Ring, Fiesds]
  Y --> C[Vector spaces and linear transformations]
  C --> Z[Subspace, Sum, Quotient]
  E --> U[Topological Data Analysis]
  E --> B[Matrix algebra]
  B --> D[Row operations, LU, determinant, rank]
  Z --> E[Basis, coordinates, dimension]
  E --> F[Eigenvalues and eigenspaces]
  F --> B
  D --> A[Linear systems] 
  A --> W[Differential Equations]
  F --> G[Diagonalization and Jordan form]
  G --> H[Dynamical systems, Markov chains, PageRank]
  E --> I[Inner Product Spaces - Geometry] 
  F --> I
  I --> J[Orthogonal projection and orthogonal bases]
  J --> K[Least squares]
  J --> L[QR and Schur]
  J --> M[Spectral theorem]
  M --> N[SVD and PCA]
  J --> O[FFT and wavelets]
  G --> Q[Spectral graph theory]
  I --> R[Hilbert spaces]
  E --> S[Multilinear algebra]
  D --> T[Optimization]
  D --> P[matrix calculus]
  D --> V[Random Variables]

History of Linear Algebra:

Timeline page: Timeline.html

Course learning arc

Part I. The computational beginning

The course begins with the practical question:

When does a system of linear equations have no solution, one solution, or infinitely many solutions?

This leads naturally to matrices, row operations, rank, null spaces, determinants, and matrix factorizations.

Part II. The language of vector spaces

A linear system is not only a calculation. It reveals the deeper language of vector spaces: subspaces, span, independence, bases, dimension, coordinates, linear transformations, kernels, images, and quotient spaces.

Part III. Spectral structure

Eigenvalues and eigenvectors reveal directions that a transformation preserves. This is the gateway to diagonalization, Jordan form, matrix powers, dynamical systems, Markov chains, Perron–Frobenius theory, and PageRank-type ideas.

Part IV. Geometry from inner products

Inner products give length, angle, distance, and orthogonality. This leads to projections, Gram–Schmidt, QR factorization, least squares, adjoints, symmetric/Hermitian matrices, Schur decomposition, spectral decomposition, and positive definiteness.

Part V. Data, signals, networks, and applications

The later chapters show how the same core ideas power modern applications: SVD and PCA, image compression, Haar wavelets, FFT, Grassmannians, numerical eigenvalue computation, multilinear algebra, Hilbert spaces, spectral graph theory, differential equations, optimization, probability, and matrix calculus.

Chapter roadmap

Chapter	Topic	Main ideas	Chapter link	Lab link
1	Linear Systems	linear systems, row reduction, existence and uniqueness	chapter-01	lab-01
2	Matrix Algebra	matrix products, inverses, transpose, LU, rank	chapter-02	lab-02
3	Linear Spaces, Subspaces, Linear Maps	vector spaces, span, direct sums, kernels, images, quotients	chapter-03	lab-03
4	Basis, Dimension, Rank-Nullity	basis, coordinates, dimension, rank-nullity	chapter-04	lab-04
5	Coordinates, Matrices, Change of Basis	coordinate vectors, matrix representations, similarity	chapter-05	lab-05
6	Determinants	area/volume, invertibility, row operations, characteristic polynomial	chapter-06	lab-06
7	Eigenvalues and Diagonalization	eigenvectors, eigenspaces, diagonalization, powers	chapter-07	lab-07
8	Jordan Canonical Form	generalized eigenvectors, Jordan blocks, Cayley-Hamilton	chapter-08	lab-08
9	Dynamical Systems and Markov Chains	linear iteration, stochastic matrices, Perron-Frobenius	chapter-09	lab-09
10	Inner Products, Projections, QR, Adjoints	geometry, orthogonality, Gram-Schmidt, QR	chapter-10	lab-10
11	Complex Inner Products and Schur	Hermitian inner product, unitary matrices, Schur decomposition	chapter-11	lab-11
12	Least Squares and Data Fitting	normal equations, projections, regression, data fitting	chapter-12	lab-12
13	Symmetric and Hermitian Matrices	spectral theorem, quadratic forms, positive definiteness	chapter-13	lab-13
14	Singular Value Decomposition	SVD, low-rank approximation, image compression, PCA	chapter-14	lab-14
15	Haar Bases and Wavelets	orthogonal bases, multiscale signals, compression	chapter-15	lab-15
16	Fast Fourier Transforms	Fourier series, DFT, FFT, signals	chapter-16	lab-16
17	Grassmannians and Subspace Distances	principal angles, subspace geometry, data subspaces	chapter-17	lab-17
18	Computational Complexity and Eigenvalues	Big-O, sparse matrices, power method, QR algorithm	chapter-18	lab-18
19	Applications of Multilinear Algebra	tensor products, exterior products, multilinear maps	chapter-19	lab-19
20	Hilbert Spaces and Applications	metric/normed/inner product spaces, projection theorem	chapter-20	lab-20
21	Topological Data Analysis	chain complexes, homology, persistence, barcodes	chapter-21	lab-21
22	Duality and Cohomology	dual spaces, cochains, coboundaries, cohomology	chapter-22	lab-22
23	Spectral Graph Theory	graphs as matrices, Laplacian, spectrum, clustering	chapter-23	lab-23
24	Linear Algebra and Differential Equations	matrix exponential, systems of ODEs, stability	chapter-24	lab-24
25	Linear Algebra and Optimization	convexity, gradients, least squares, constraints	chapter-25	lab-25
26	Linear Algebra and Probability	random vectors, covariance, Gaussian distributions, random matrices	chapter-26	lab-26
27	Matrix Calculus	gradients, Hessians, Jacobians, optimization and ML	chapter-27	lab-27

How to use this book

Each chapter is written as an independent learning unit. Most chapters include:

• a story-driven opening motivation;
• precise definitions, theorems, and examples;
• Python computations and numerical experiments;
• challenge questions;
• practice problems;
• hide/show proofs and solutions;
• AI companion activities for guided exploration.

Each lab is designed for independent study. The labs usually include a Quarto lab page, an interactive HTML exploration, and a Jupyter notebook with guided solutions.

Mathematical habits for the course

The course repeatedly practices the following habits.

Translate among representations

A single idea may appear as an equation, a matrix, a linear map, a geometric object, or a dataset.

For example, a linear transformation can be viewed as:

• a function \(T:V\to W\);
• a matrix \(A\) after choosing bases;
• a geometric machine acting on vectors;
• a data transformation acting on features;
• an operator whose structure is revealed by eigenvalues or singular values.

Ask the invariant question

Many computations are coordinate-dependent, but the important ideas are often invariant:

• dimension of a vector space;
• rank and nullity of a linear map;
• eigenvalues under similarity;
• singular values under orthogonal/unitary changes of coordinates;
• subspace angles on Grassmannians;
• connected components from Laplacian zero eigenvalues.

Connect proof and computation

A theorem is most useful when it becomes a computational tool. For example:

• the rank-nullity theorem gives computation of Betti numbers in TDA;
• the spectral theorem explains quadratic forms and PCA;
• the projection theorem explains least squares;
• the SVD explains low-rank approximation;
• the Perron–Frobenius theorem explains long-term behavior of positive systems;
• the Laplacian quadratic form explains smoothness and clustering on graphs.

AI companion prompts

Use AI as a study partner, not as a replacement for mathematical reasoning.

1. Concept check. Explain the difference between a matrix, a linear transformation, and a coordinate representation.
2. Proof coach. Give me hints for proving the rank-nullity theorem without giving the full solution first.
3. Computation coach. Generate a Python example where a matrix is diagonalizable and another where it is not.
4. Application bridge. Explain how projections lead to least squares, and how least squares leads to data fitting.
5. Synthesis. Make a concept map connecting QR, Schur, spectral decomposition, SVD, and PCA.

Suggested independent-study routine

For each chapter:

1. Read the story and definitions.
2. Work through examples by hand.
3. Run the Python computations.
4. Try the interactive lab.
5. Complete the practice problems before opening solutions.
6. Ask an AI companion to generate one similar problem and then solve it independently.

The interactive overview page is included with this file as: linear_algebra_interactive_overview