1. Embedding search
Move the query vector. The words are toy embeddings. Ranking is by cosine similarity.
2. Neural layer
A layer computes h = ReLU(Wx+b). Change the input and watch the hidden activations.
3. Softmax classifier
Scores become probabilities. Increase the temperature to make probabilities flatter.
Softmax converts raw dot-product scores into a probability distribution.
4. Attention matrix
Each row is a token choosing a weighted average of value vectors. Adjust similarity strength.
5. Low-rank hidden factors
A large table can be approximated by a few hidden factors. Choose rank .
6. High-dimensional geometry
Random vectors become nearly orthogonal as dimension grows.
Reflection
Write a short explanation: Which operation seems most central to AI in this lab: dot product, matrix multiplication, softmax, SVD, or optimization?
Big idea: AI systems often look magical because they are large, but their core grammar is built from linear algebra operations you can inspect.