Central formula
A neural network layer computes
h = σ(Wx + b)
The matrix W mixes features, the bias b shifts thresholds, and the activation σ bends the computation.
1. One ReLU neuron
Move the inputs and weights. The neuron computes z = w·x + b and h = max(0,z).
2. Activation functions
Compare ReLU, sigmoid, and tanh.
3. One layer as many neurons
This layer maps R² to R³.
W = [[1,0],[0,1],[1,-1]], b = [0,0,1]
4. Decision boundary
A ReLU neuron changes behavior across the line w·x+b=0.
5. Softmax classifier
Adjust raw scores and see probabilities.
6. XOR needs hidden features
The XOR pattern cannot be separated by one line in input space. A hidden layer can make the structure easier.
7. Gradient descent
Train a line y = wx+b on synthetic data. Click several times.
8. Parameter counter
For a dense layer from n inputs to m outputs, parameters = mn + m.
Reflection
Write a short answer: In what sense is a neural network a matrix machine, and in what sense is it more than a matrix machine?