2  Introduction

2.1 Why another linear algebra book?

Linear algebra is one of the most useful languages in modern mathematics, science, engineering, statistics, data science, and artificial intelligence. It appears whenever we organize information into vectors, transform one set of numbers into another, solve many equations at once, approximate noisy data, or search for hidden structure in a matrix.

Yet many students first meet linear algebra as a long list of procedures:

• solve a system by elimination,
• compute a matrix product,
• find an inverse,
• calculate a determinant,
• find eigenvalues,
• apply a formula for least squares.

These procedures are important, but linear algebra is much more than procedure. It is a way of seeing structure.

This book is written to help students build that way of seeing.

The main goal is not only to teach how to compute. The goal is to explain what the computations mean.

A matrix is not only an array of numbers. It can be a system of equations, a transformation of space, a data table, a model, a graph, an image, or a layer in a neural network.

A vector is not only a column of numbers. It can be a point, an arrow, a feature representation, a signal, an embedding, or a state of a dynamical system.

A subspace is not only a set closed under addition and scalar multiplication. It can be a model space, a space of possible outputs, a set of hidden constraints, or a lower-dimensional structure inside high-dimensional data.

This book develops linear algebra as a coherent story connecting algebra, geometry, computation, and data.

2.2 The central theme

The central theme of the book is:

Linear algebra is the language of structure, transformation, and approximation.

The book begins with linear systems,

\[ Ax=b, \]

because systems of equations are one of the most concrete ways to enter the subject. But as the book progresses, the meaning of this equation expands.

The matrix \(A\) becomes a transformation.

The vector \(b\) becomes a target.

The equation \(Ax=b\) becomes a question about whether \(b\) lies in the column space of \(A\).

If no exact solution exists, the problem becomes a projection problem.

Projection leads to least squares.

Least squares leads to data fitting.

Data fitting leads to optimization.

Eigenvalues reveal the directions a transformation preserves.

Singular values reveal the directions a transformation stretches most.

By the end of the book, the original equation

\[ Ax=b \]

has become a gateway to the larger language of modern computation.

2.3 A book in \(\mathbb{R}^n\)

This book focuses on linear algebra in real finite-dimensional space:

\[ \mathbb{R}^n. \]

This choice is intentional.

The setting of \(\mathbb{R}^n\) is concrete enough for visualization and computation, but rich enough to contain the essential ideas of linear algebra:

• vectors,
• matrices,
• systems of equations,
• subspaces,
• bases,
• coordinates,
• linear transformations,
• orthogonality,
• projection,
• least squares,
• determinants,
• eigenvalues,
• diagonalization,
• symmetric matrices,
• quadratic forms,
• singular value decomposition.

Abstract vector spaces and complex vector spaces are beautiful and important, but they are not the main focus here. The goal is to build a strong computational and geometric foundation first.

2.4 Three viewpoints

The book uses three viewpoints throughout.

2.4.1 The algebraic viewpoint

The algebraic viewpoint studies symbols and equations.

We ask:

• How do we solve a system?
• What does row reduction reveal?
• When does an inverse exist?
• How do matrix products work?
• How do we compute determinants, eigenvalues, and singular values?

This viewpoint gives precision and algorithms.

2.4.2 The geometric viewpoint

The geometric viewpoint studies shapes and transformations.

We ask:

• What does a vector represent?
• What does a matrix do to space?
• What is the geometry of a solution set?
• What is a subspace?
• What does it mean to project onto a subspace?
• How does a determinant measure volume?
• Why are eigenvectors special directions?

This viewpoint gives intuition.

2.4.3 The computational viewpoint

The computational viewpoint studies implementation and experimentation.

We ask:

• How can Python help us compute?
• How can plots reveal geometry?
• How do numerical algorithms behave?
• Why are orthogonal matrices stable?
• Why is QR often better than normal equations?
• What do singular values reveal about data?

This viewpoint connects mathematics to practice.

NoteThree ways to understand a concept

For every major idea, try to understand it algebraically, geometrically, and computationally.

A formula tells you how to compute.

A picture tells you what is happening.

A program lets you experiment.

2.5 The role of computation

Computation is not treated as an optional add-on in this book.

Python examples are included because many linear algebra ideas become clearer when students can compute and visualize them.

For example:

• row reduction reveals pivots and free variables,
• plots show how a matrix transforms a grid,
• projection diagrams show closest-point geometry,
• least-squares examples show how a model fits data,
• eigenvalue simulations show long-term dynamics,
• SVD experiments show compression and low-rank structure.

The purpose of computation is not to replace mathematical reasoning. The purpose is to make reasoning visible.

Students should still learn to work by hand. But they should also learn how modern linear algebra is actually used: with algorithms, numerical methods, visualization, and data.

2.6 The role of examples

The book uses many small examples.

Small examples matter because they allow us to see structure clearly. A \(2\times 2\) matrix can show rotation, reflection, scaling, shear, determinant, eigenvectors, diagonalization, and singular values.

A \(3\times 3\) system can show pivot variables, free variables, column spaces, null spaces, and rank.

A small data set can show least squares, projection, residuals, and model fitting.

After the idea is clear in small dimensions, Python allows us to move to larger examples.

2.7 The role of proof

This book includes proofs and derivations, but the purpose of proof is explanation.

A proof should answer the question:

Why is this true?

For example:

• Why does row reduction preserve solution sets?
• Why does projection error have to be orthogonal?
• Why do the normal equations appear in least squares?
• Why are eigenvectors for distinct eigenvalues independent?
• Why does a symmetric matrix have orthogonal eigenvectors?
• Why does the SVD reveal rank and low-rank structure?

Proofs are not separate from computation or geometry. They connect the pieces.

2.8 The story of the book

The book is organized as a path from equations to data.

2.8.1 1. Linear systems

We begin with systems of equations. A system is a collection of constraints. Solving a system means finding all points satisfying those constraints.

This leads to matrices, augmented matrices, row operations, pivots, consistency, and solution sets.

2.8.2 2. Vectors and matrices

Vectors organize numbers with meaning. Matrices organize systems and transformations. Matrix multiplication becomes a way of combining information and composing actions.

2.8.3 3. Subspaces and bases

The column space, null space, and row space reveal the structure of a matrix. Bases and dimension give efficient descriptions of subspaces.

Coordinates show that the same vector can be described in different languages.

2.8.4 4. Linear transformations

A matrix becomes a function:

\[ x\mapsto Ax. \]

This viewpoint connects algebra to geometry. Matrices stretch, rotate, reflect, shear, project, and collapse space.

2.8.5 5. Orthogonality and projection

The dot product introduces length, distance, and angle.

Orthogonality leads to projection, which solves closest-point problems.

Projection leads naturally to least squares, one of the most important applications of linear algebra.

2.8.6 6. Spectral structure

Determinants measure volume scaling.

Eigenvectors reveal invariant directions.

Diagonalization explains repeated multiplication and dynamical systems.

Symmetric matrices give orthogonal eigenvector bases.

Quadratic forms describe curvature and optimization landscapes.

2.8.7 7. Singular values and data

The SVD works for every matrix.

It reveals stretching directions, rank, low-rank approximation, image compression, PCA, denoising, and the geometry of data.

2.8.8 8. Linear algebra for AI

The final chapter shows how the ideas of the book appear in data science and artificial intelligence: data matrices, embeddings, similarity, regression, PCA, graphs, recommendation systems, neural-network layers, and attention-like matrix operations.

2.9 A map of important equations

Several equations appear again and again.

2.9.1 Linear systems

\[ Ax=b. \]

This is the starting point. It asks whether \(b\) can be produced as a linear combination of the columns of \(A\).

2.9.2 Homogeneous systems

\[ Ax=0. \]

This reveals the null space of \(A\), the directions that \(A\) sends to zero.

2.9.3 Matrix transformations

\[ T(x)=Ax. \]

This interprets a matrix as a function from one vector space to another.

2.9.4 Projection

\[ \operatorname{proj}_W(b) \]

is the closest vector in a subspace \(W\) to the vector \(b\).

2.9.5 Least squares

\[ A^TA\hat x=A^Tb. \]

These normal equations appear when \(Ax=b\) has no exact solution and we seek the best approximation.

2.9.6 Eigenvalues

\[ Av=\lambda v. \]

This says that \(v\) is a direction preserved by \(A\).

2.9.7 Diagonalization

\[ A=PDP^{-1}. \]

This says that \(A\) becomes diagonal in an eigenvector basis.

2.9.8 Spectral theorem

\[ A=QDQ^T. \]

For symmetric matrices, the eigenvector basis can be chosen orthonormally.

2.9.9 Quadratic forms

\[ q(x)=x^TAx. \]

This describes energy, curvature, ellipses, saddles, and optimization geometry.

2.9.10 Singular value decomposition

\[ A=U\Sigma V^T. \]

This reveals the input directions, output directions, and stretching factors of any matrix.

2.10 What makes linear algebra powerful?

Linear algebra is powerful because the same structures appear everywhere.

A matrix may represent:

• a system of equations,
• a data table,
• a linear transformation,
• an image,
• a graph,
• a covariance matrix,
• a design matrix,
• a neural-network layer,
• a recommendation model.

A vector may represent:

• an unknown solution,
• a point in space,
• a row of data,
• a signal,
• a word embedding,
• a probability distribution,
• a state of a system.

A subspace may represent:

• all possible outputs,
• all solutions of a homogeneous system,
• a model class,
• a low-dimensional approximation,
• a space of hidden features.

This reuse of ideas is what makes linear algebra a language.

2.11 How to study this book

Linear algebra is cumulative. Each chapter uses ideas from earlier chapters.

A productive way to study is:

1. Read the opening question of the chapter.
2. Identify the main objects: vectors, matrices, subspaces, transformations.
3. Work through the examples slowly.
4. Draw pictures whenever possible.
5. Run the Python code.
6. Change the numbers and observe what changes.
7. Work exercises by hand.
8. Summarize the chapter in your own words.

Do not try to memorize every formula at once. Instead, ask what each formula means.

For example, the projection formula is not only a formula. It says that the best approximation is found by making the error orthogonal.

The normal equations are not only equations. They say that the residual is orthogonal to the model space.

The eigenvalue equation is not only algebra. It says that a matrix has special directions it does not turn.

The SVD is not only a factorization. It says that every matrix has natural input and output directions.

2.12 What students should be able to do

After completing this book, students should be able to:

• solve and interpret linear systems,
• use row reduction to find pivots, free variables, and solution sets,
• understand matrices as transformations,
• find bases for column spaces, null spaces, and row spaces,
• change coordinates between bases,
• use dot products to compute lengths, distances, angles, and projections,
• solve least-squares problems,
• compute and interpret determinants,
• find eigenvalues and eigenvectors,
• diagonalize appropriate matrices,
• use eigenvalues to analyze dynamics,
• apply the spectral theorem to symmetric matrices,
• classify quadratic forms,
• compute and interpret the SVD,
• use low-rank approximation for data,
• connect linear algebra to modern data and AI.

2.13 A note on notation

Vectors are usually written as column vectors.

Matrices are denoted by capital letters such as \(A\), \(B\), \(Q\), \(R\), \(U\), \(V\), and \(D\).

The transpose of \(A\) is written as \(A^T\).

The identity matrix is written as \(I\).

The column space of \(A\) is written as \(\operatorname{Col}(A)\).

The null space of \(A\) is written as \(\operatorname{Null}(A)\).

The rank of \(A\) is written as \(\operatorname{rank}(A)\).

The determinant of a square matrix is written as \(\det(A)\).

The norm of a vector is written as \(\|x\|\).

The dot product of \(x\) and \(y\) is written as \(x\cdot y\) or \(x^Ty\).

The notation is standard, but the meaning should always be connected to the geometry.

2.14 A note on Python

The Python examples use common scientific computing tools:

• numpy for arrays and linear algebra,
• matplotlib for visualization,
• occasionally sympy for exact symbolic computation.

The examples are meant to be readable and modifiable.

Students are encouraged to change matrices, vectors, and data sets to see what happens.

Linear algebra becomes much more concrete when formulas can be tested.

2.15 The promise of the subject

Linear algebra has a special feature: simple ideas combine into very powerful tools.

The dot product leads to projection.

Projection leads to least squares.

Least squares leads to regression.

Orthogonality leads to QR factorization.

Eigenvectors lead to diagonalization.

Symmetric matrices lead to the spectral theorem.

The spectral theorem leads to quadratic forms and PCA.

The SVD leads to compression, denoising, and low-rank structure.

These are not separate topics. They are one connected language.

The purpose of this book is to help you learn that language.

2.16 Final perspective

Linear algebra is not only about calculating answers. It is about learning to see patterns:

• a solution set as a geometric object,
• a matrix as a transformation,
• a basis as a coordinate language,
• a projection as a best approximation,
• an eigenvector as a natural direction,
• a singular value as a measure of importance,
• a data matrix as a structured object.

Once these ideas become familiar, linear algebra becomes a way to think about the modern world of computation and data.