Central idea
input vector → matrix machine → output vector
If A is an m × n matrix, then it maps vectors from n-dimensional input space to m-dimensional output space.
Rows describe output recipes. Columns describe building blocks. The product Ax is a linear combination of the columns of A.
1. Build a matrix and transform one vector
Blue is the input vector. The second vector is the output Ax. The transformed basis directions Ae₁ and Ae₂ are also shown from the origin.
2. The grid test
A matrix transforms the whole plane, not only one vector. Move the sliders above and watch the grid change.
Try this: set b = 0 and c = 0. You get pure coordinate scaling. Then set a = 1, d = 1, c = 0, and move b. You get a shear.
3. Standard matrix machines
4. Feature mixer
This small data machine turns three raw features into two summary features.
Raw student vector: [study hours, sleep hours, practice problems]
A warning: if one feature has a much larger scale, it can dominate the output. This is why scaling and standardization matter in data science.
5. Information loss
A projection matrix can collapse a whole plane onto a line. Once this happens, many different inputs have the same output.
Question: after projection onto the x-axis, can you recover the original y-coordinate? Why not?
6. Image as points, matrix as visual transformer
The letter below is a cloud of points. The same matrix sliders from Section 1 transform it.
7. Reflection prompts
- What is the difference between the row view and the column view of matrix-vector multiplication?
- Why do the columns of a matrix tell us where the standard basis vectors go?
- Give one example where a matrix preserves information and one example where it loses information.
- How is a neural network layer related to the matrix machine idea?