Computational Linear Algebra

A First Course in Matrices, Algorithms, Geometry, and Data

Author

He Wang

1 Welcome

Computational Linear Algebra in \(\mathbb{R}^n\) is a first course in linear algebra written for students who want to understand matrices as both mathematical objects and computational tools.

The book begins with systems of linear equations and gradually builds toward the modern language of data, algorithms, and artificial intelligence. Along the way, vectors become data points, matrices become transformations, subspaces become model spaces, projections become best approximations, eigenvectors become natural directions, and singular vectors become hidden patterns in data.

This book is designed around one guiding idea:

Linear algebra is the language of structure, computation, geometry, and data.

1.1 What this book is about

Linear algebra is often introduced as a collection of procedures:

  • solve systems,
  • reduce matrices,
  • compute inverses,
  • find determinants,
  • calculate eigenvalues.

Those procedures are important, but they are only the beginning.

This book emphasizes the ideas behind the procedures:

  • a linear system describes constraints,
  • a matrix represents a transformation,
  • a basis gives a coordinate language,
  • a subspace describes possible outputs or hidden degrees of freedom,
  • orthogonality gives geometry,
  • projection gives best approximation,
  • least squares turns inconsistent systems into useful models,
  • eigenvectors reveal invariant directions,
  • the SVD reveals low-rank structure in data.

The goal is to help students see linear algebra as a coherent story.

1.2 Audience

This book is written for:

  • students taking a first course in linear algebra,
  • students preparing for applied mathematics, statistics, machine learning, data science, or AI,
  • instructors who want a computational and geometric linear algebra text,
  • readers who want to connect classical linear algebra to modern applications.

The main setting is \(\\mathbb{R}^n\). Complex vector spaces, abstract vector spaces, and infinite-dimensional spaces are not the focus. The goal is to build a strong foundation in finite-dimensional real linear algebra with computation and applications.

1.3 Mathematical and computational style

This book uses three complementary viewpoints.

1.3.1 Algebraic viewpoint

We study equations, matrices, row reduction, inverses, determinants, eigenvalues, and factorizations.

1.3.2 Geometric viewpoint

We interpret vectors as arrows and points, matrices as transformations, subspaces as geometric objects, projections as shadows, determinants as volume scaling, and eigenvectors as invariant directions.

1.3.3 Computational viewpoint

We use Python to compute, visualize, simulate, and experiment. The computational examples are not added at the end; they are part of how the book explains the ideas.

NoteRecommended workflow

For each chapter:

  1. Read the mathematical story.
  2. Run the Python examples.
  3. Draw or visualize the geometry.
  4. Work through the exercises.
  5. Connect the ideas to data and computation.

1.4 Chapter roadmap

1.4.1 Part I. Linear systems and matrices

Chapter Topic Main idea
1 Linear Systems Linear equations describe constraints, intersections, and balance laws.
2 Matrices and Vectors Matrices organize data and systems; vectors carry numerical meaning.
3 Row Reduction Gaussian elimination reveals pivots, free variables, consistency, and solution structure.
4 Matrix Algebra Matrix operations describe combinations and compositions of linear actions.
5 Inverses Invertible matrices represent reversible transformations.

1.4.2 Part II. Vector spaces inside \(\\mathbb{R}^n\)

Chapter Topic Main idea
6 Span Linear combinations generate lines, planes, and higher-dimensional subspaces.
7 Bases and Dimension A basis is an efficient coordinate system for a space.
8 Subspaces Column spaces, null spaces, and row spaces organize matrix structure.
9 Coordinates and Change of Basis Coordinates depend on the basis; changing basis changes the language but not the vector.

1.4.3 Part III. Linear transformations and geometry

Chapter Topic Main idea
10 Linear Transformations Matrices move vectors while preserving linear combinations.
11 Composition and Inverses Matrix products represent composition of transformations.
12 Orthogonality Dot products define length, distance, angle, and perpendicularity.
13 Orthogonal Projection Projection finds the closest point in a subspace.
14 Gram–Schmidt and QR Orthogonal bases lead to stable matrix computations.
15 Least Squares Inconsistent systems become best-approximation problems.

1.4.4 Part IV. Spectral structure

Chapter Topic Main idea
16 Determinants Determinants measure signed volume scaling and invertibility.
17 Eigenvalues and Eigenvectors Eigenvectors are directions preserved by a matrix.
18 Diagonalization and Dynamics An eigenvector basis separates a system into independent directions.
19 Orthogonal and Symmetric Matrices Orthogonal matrices preserve geometry; symmetric matrices have spectral structure.
20 Quadratic Forms Symmetric matrices define landscapes, curvature, and optimization geometry.
21 Singular Value Decomposition The SVD reveals stretching directions, rank, and low-dimensional structure.

1.4.5 Part V. Linear algebra for data and AI

Chapter Topic Main idea
22 Linear Algebra for Data and AI Data matrices, embeddings, PCA, low-rank approximation, graphs, and neural-network layers all use linear algebra.

1.5 The central story

The book follows a gradual path.

We begin with a system:

\[ Ax=b. \]

Then we ask:

  • When does a solution exist?
  • How many solutions are there?
  • What does the solution set look like?
  • What does the matrix \(A\) do to vectors?
  • What structure is hidden in the columns and rows of \(A\)?
  • What if no exact solution exists?
  • Which directions does \(A\) preserve?
  • Which directions does \(A\) stretch the most?
  • How does this help us understand data?

By the end of the book, the same equation \(Ax=b\) has become part of a much larger language:

\[ \text{linear algebra} = \text{systems} + \text{geometry} + \text{computation} + \text{data}. \]

1.6 Why computation matters

Linear algebra is not only a pencil-and-paper subject. It is also the foundation of scientific computing and machine learning.

This book uses Python to help students:

  • visualize lines, planes, projections, and transformations,
  • compute row-reduced forms and bases,
  • solve least-squares problems,
  • fit data,
  • simulate dynamical systems,
  • compute eigenvalues and singular values,
  • compress and denoise matrices,
  • explore PCA and low-rank structure.

The Python examples are intended to support mathematical understanding, not replace it.

1.7 What students should learn to see

After completing the book, students should be able to look at a matrix and ask:

  • What are its columns telling me?
  • What is its column space?
  • What is its null space?
  • Is the transformation reversible?
  • Does it preserve geometry?
  • Does it collapse dimensions?
  • What are its special directions?
  • What are its dominant singular values?
  • What model or data structure does it represent?

These questions are more important than memorizing formulas. They are the habits of linear algebraic thinking.

1.8 Suggested use in a course

A one-semester course can emphasize the core chapters:

  • Chapters 1–5: systems and matrices,
  • Chapters 6–9: subspaces, basis, and coordinates,
  • Chapters 10–15: transformations, orthogonality, projection, QR, and least squares,
  • Chapters 16–21: determinants, eigenvalues, diagonalization, spectral theorem, quadratic forms, and SVD,
  • Chapter 22: synthesis for data and AI.

Instructors can choose applications according to the audience:

  • engineering: systems, transformations, least squares, eigenvalues,
  • statistics: projections, least squares, covariance, PCA,
  • data science: SVD, low-rank approximation, embeddings,
  • applied mathematics: subspaces, spectral theory, optimization geometry,
  • AI: matrix layers, similarity, attention, dimensionality reduction.

1.9 How to read the book

This book is cumulative. Later chapters depend on earlier ones.

A good reading pattern is:

  1. Understand the main question of the chapter.
  2. Work through the examples by hand.
  3. Run the Python code.
  4. Interpret the pictures.
  5. Solve exercises without looking back too quickly.
  6. Revisit the chapter summary.
  7. Ask how the idea appears in data, computation, or AI.

1.10 Companion computational materials

The book is designed to support computational labs and interactive pages.

Suggested companion materials include:

  • Jupyter notebooks for chapter labs,
  • interactive HTML demonstrations,
  • Python reference notes,
  • visualization exercises,
  • data and AI mini-projects.

These materials can help students move from formulas to experiments.

1.11 Book repository

Repository:

https://github.com/wanghemath/Book-ComputationalLinearAlgebra

Book site:

https://wanghemath.github.io/Book-ComputationalLinearAlgebra/

1.12 Final invitation

Linear algebra is a language.

At first, it describes rows, columns, equations, and matrices. But with practice, it begins to describe much more: geometry, data, algorithms, models, images, networks, and learning systems.

The purpose of this book is to help students learn that language carefully, computationally, and visually.