Chapter 4 Interactive: Matrix Algebra and Matrix Multiplication

Learning goals

This chapter develops the algebra of matrices: matrix-vector multiplication, matrix-matrix multiplication, row-column computation, column interpretation, matrix powers, diagonal and triangular matrices, and the noncommutative nature of matrix multiplication.

Check dimensions

Decide whether products are defined and determine the output size.

Compute products

Use row-column multiplication and the column interpretation.

Understand order

Interpret products as composition of linear processes.

The dimension rule

$$\underbrace{A}_{m\times n}\underbrace{B}_{r\times p}\quad\text{is defined exactly when}\quad n=r.$$

$$\text{If }A\text{ is }m\times n\text{ and }B\text{ is }n\times p,\text{ then }AB\text{ is }m\times p.$$

Interactive dimension checker

A rows $m$

A columns $n$

B rows $r$

B columns $p$

Common habit. Always write the dimensions before multiplying. Many matrix mistakes disappear once the inner dimensions are checked.

Matrix product calculator

Use this lab to compute $AB$ and compare it with $BA$. Entries may be integers, decimals, or fractions.

Choose a preset

Matrix A

Matrix B

Use one row per line, for example 1 2 then 3 4.

Column interpretation

If $B=[b_1\;b_2\;\cdots\;b_p]$, then

$$AB=\begin{bmatrix}Ab_1&Ab_2&\cdots&Ab_p\end{bmatrix}.$$

The product $AB$ is built by applying $A$ to every column of $B$.

Matrix multiplication is not usually commutative

For real numbers, $ab=ba$. For matrices, usually $AB\ne BA$. Sometimes both products are defined but unequal. Sometimes one product is defined and the other is not.

Both defined but unequal

$$\begin{bmatrix}1&2\\3&4\end{bmatrix}\begin{bmatrix}2&1\\0&-1\end{bmatrix}\ne\begin{bmatrix}2&1\\0&-1\end{bmatrix}\begin{bmatrix}1&2\\3&4\end{bmatrix}.$$

One defined, one not

$$(2\times3)(3\times1)\text{ is defined, but }(3\times1)(2\times3)\text{ is not.}$$

Meaning. Different order means different sequence of processes.

Identity and zero matrices

Identity matrices

If $A$ is $m\times n$, then

$$I_mA=A,\qquad AI_n=A.$$

The identity size on the left and right may be different.

Zero matrices

If sizes are compatible, multiplying by a zero matrix gives a zero matrix.

$$A0_{n\times p}=0_{m\times p}.$$

The output size is determined by the outside dimensions.

Matrix powers

If $A$ is square, then $A^2=AA$, $A^3=AAA$, and $A^k$ means repeated application of the same process.

Diagonal matrices

$$D=\begin{bmatrix}2&0\\0&3\end{bmatrix}\quad\Rightarrow\quad D^k=\begin{bmatrix}2^k&0\\0&3^k\end{bmatrix}.$$

Each coordinate is scaled independently.

Repeated dynamics

$$x_{t+1}=Ax_t\quad\Rightarrow\quad x_k=A^kx_0.$$

This idea appears in Markov chains, ranking algorithms, and dynamical systems.

Diagonal and triangular matrices

Diagonal

Only diagonal entries may be nonzero. Easy to multiply and power.

Upper triangular

All entries below the diagonal are zero. Use back substitution.

Lower triangular

All entries above the diagonal are zero. Use forward substitution.

$$\begin{cases}x_1+2x_2-x_3=4,\\x_2+3x_3=5,\\x_3=1\end{cases}\quad\Rightarrow\quad (x_1,x_2,x_3)=(1,2,1).$$

Composition visualizer: projection and rotation

The product $ABx$ means $B$ acts first, then $A$ acts. Here $P$ is projection onto the $x$-axis and $Q$ is counterclockwise rotation by $90^\circ$. Compare $PQx$ with $QPx$.

The original vector $x$ is shown along with the two results.

$$P=\begin{bmatrix}1&0\\0&0\end{bmatrix},\qquad Q=\begin{bmatrix}0&-1\\1&0\end{bmatrix}.$$

$x_1$1

$x_2$2

Interactive repeated dynamics

Explore $x_{t+1}=Ax_t$. The default matrix is from the chapter's repeated-dynamics example.

Matrix A Initial vector x0 Number of steps

Python connection

Matrix multiplication

import numpy as np

A = np.array([[1, 2], [3, 4]], dtype=float)
B = np.array([[2, 1], [0, -1]], dtype=float)

A @ B
B @ A

Powers and dynamics

A = np.array([[1, 1], [1, 0]], dtype=float)
np.linalg.matrix_power(A, 5)

x = np.array([100, 0], dtype=float)
for k in range(10):
    x = A @ x

Common mistakes

Multiplying entry by entry

Matrix multiplication is row-by-column, not entrywise multiplication.

Ignoring dimensions

Check $(m\times n)(n\times p)=(m\times p)$ before computing.

Assuming $AB=BA$

Matrix multiplication is usually not commutative.

Reversing composition order

In $ABx$, $B$ acts first and $A$ acts second.

Quick check

1. What is the size of $(2\times3)(3\times4)$?

It is defined and the result has size $2\times4$.

2. In $ABx$, which matrix acts on $x$ first?

$B$ acts first. Then $A$ acts on the result.

3. When is $A^2$ defined?

$A^2$ is defined only when $A$ is square.