Matrices, Vectors, and Augmented Systems
Chapter 1 solved systems by rewriting equations. Chapter 2 introduces the compressed language: matrices store the coefficients, vectors store unknowns and outputs, and matrix-vector multiplication turns a whole system into one equation.
A matrix stores a linear system, and matrix-vector multiplication turns a system into one compact equation: $Ax=b$.
Example system
$$\begin{cases}x_1-3x_2-5x_3=1,\\x_1-x_2-2x_3=0,\\3x_1-x_2+x_3=3\end{cases}$$
Augmented matrix
$$\left[\begin{array}{ccc|c}1&-3&-5&1\\1&-1&-2&0\\3&-1&1&3\end{array}\right]$$
Learning goals
2.1–2.2 Matrix anatomy
An $m\times n$ matrix is a rectangular array with $m$ rows and $n$ columns. The entry $a_{ij}$ is in row $i$, column $j$.
Build a matrix
Memory rule
The notation $a_{ij}$ means: go to row $i$, then column $j$.
For a $3\times 4$ matrix, there are 3 horizontal rows and 4 vertical columns. A common mistake is to reverse the order.
2.3 Special matrices
Square, diagonal, zero, identity, and symmetric matrices appear throughout linear algebra.
Explore examples
Quick definitions
Square same number of rows and columns.
Diagonal square, off-diagonal entries are zero.
Identity diagonal entries are 1, all others 0.
Symmetric $A^T=A$.
2.4–2.6 Vectors in $\mathbb{R}^n$
A vector in $\mathbb{R}^n$ is an ordered list of $n$ real numbers. In this book, vectors are usually column vectors.
Draw a vector in $\mathbb{R}^2$
Column vector form
The same vector may represent a point, displacement, velocity, signal, feature vector, or embedding.
2.7–2.10 Matrix operations and transpose
Add and scale matrices
For matrices of the same size, addition is component-by-component. Scalar multiplication multiplies every entry.
Transpose a matrix
Edit this $2\times 3$ matrix. Rows become columns.
2.11 and 2.18 From systems to matrices
A linear system can be stored by its coefficient matrix $A$, unknown vector $x$, right-hand side $b$, and augmented matrix $[A\mid b]$.
Do not change the variable order from row to row. If the order is $x_1,x_2,x_3$, then every row must follow that order. Missing variables get coefficient 0.
2.12 The compact equation $Ax=b$
The expression $Ax=b$ represents many equations at once:
$$\begin{bmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\a_{21}&a_{22}&\cdots&a_{2n}\\\vdots&\vdots&\ddots&\vdots\\a_{m1}&a_{m2}&\cdots&a_{mn}\end{bmatrix}\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}=\begin{bmatrix}b_1\\b_2\\\vdots\\b_m\end{bmatrix}.$$
Matrix $A$
Size $m\times n$.
Stores coefficients.
Unknown vector $x$
Size $n\times 1$.
Stores unknowns.
Output vector $b$
Size $m\times 1$.
Stores right-hand sides.
2.13–2.14 Matrix-vector multiplication
Each component of $Ax$ is a dot product between a row of $A$ and the vector $x$.
Compute $Ax$
Edit the $2\times 3$ matrix and the vector in $\mathbb{R}^3$.
Dimension compatibility
If $A$ is $m\times n$, then $x$ must have $n$ entries and $Ax$ has $m$ entries.
2.15 The column-combination view
If $A=[a_1\ a_2\ \cdots\ a_n]$, then
$$Ax=x_1a_1+x_2a_2+\cdots+x_na_n.$$
Build $b$ from columns
Use $A=\begin{bmatrix}1&1\\2&-1\end{bmatrix}$, so $a_1=(1,2)$ and $a_2=(1,-1)$.
2.16 Row picture versus column picture
The same equation can be read in two different ways. The row picture asks where equations intersect. The column picture asks how to build $b$ from the columns of $A$.
Row picture
For the same matrix, the system is
The solution point is $(x_1,x_2)$.
2.20–2.21 First Python computations
NumPy gives a compact language for matrices, vectors, transpose, and matrix-vector multiplication.
import numpy as np
A = np.array([
[1, -3, -5],
[1, -1, -2],
[3, -1, 1]
], dtype=float)
b = np.array([1, 0, 3], dtype=float)
x = np.array([0, -2, 1], dtype=float)
A.shape # (3, 3)
A @ x # matrix-vector product
A.T # transposePython starts counting at 0. The mathematical entry $a_{11}$ is stored as A[0, 0].
2.22–2.24 Summary and quick quiz
Chapter summary
A matrix is a rectangular array of numbers. An $m\times n$ matrix has $m$ rows and $n$ columns.
A vector in $\mathbb{R}^n$ is usually written as a column. A linear system can be written as $Ax=b$.
The augmented matrix $[A\mid b]$ stores both the coefficients and the right-hand side.
Matrix-vector multiplication can be computed row by row or understood as a linear combination of columns.