Linear algebra began in this book with systems of equations.
Then matrices became transformations.
Vectors became data points.
Subspaces became model spaces.
Orthogonality became geometry.
Eigenvectors and singular vectors became hidden directions.
Now we bring these ideas together.
Modern data science and artificial intelligence are built on linear algebra. Data sets are matrices. Images are arrays. Text is represented by vectors. Similarity is measured by dot products. Models use matrix multiplication. Least squares fits parameters. PCA and the SVD find low-dimensional structure. Neural networks are built from linear maps and nonlinear activations.
The central question of this final chapter is:
How do the ideas of linear algebra become the language of data and AI?
This chapter is a synthesis. It reviews the key ideas of the course through modern computational examples.
After studying this chapter, you should be able to:
A data set with \(m\) observations and \(n\) features can be represented as a matrix
\[ X= \begin{bmatrix} --- x_1^T ---\\ --- x_2^T ---\\ \vdots\\ --- x_m^T --- \end{bmatrix}. \]
Here each row
\[ x_i^T \]
is one observation, and each column is one feature.
For example, if we measure height, weight, and age for \(m\) people, then
\[ X= \begin{bmatrix} \text{height}_1&\text{weight}_1&\text{age}_1\\ \text{height}_2&\text{weight}_2&\text{age}_2\\ \vdots&\vdots&\vdots\\ \text{height}_m&\text{weight}_m&\text{age}_m \end{bmatrix}. \]
Thus a data set is not only a table. It is a collection of vectors in \(\mathbb{R}^n\).
Each data point is a point in feature space.
The two viewpoints of a data matrix are both important.
Rows are data points.
Each row vector
\[ x_i\in\mathbb{R}^n \]
describes one observation using \(n\) features.
This viewpoint supports:
Columns are features.
Each column vector records one variable across all observations.
This viewpoint supports:
Linear algebra lets us move between these viewpoints.
Before analyzing data, we often center each feature by subtracting its mean.
If \(X\) is a data matrix, the centered data matrix is
\[ X_c=X-\mathbf{1}\mu^T, \]
where \(\mu\) is the vector of column means.
Centering makes each feature have mean zero.
We may also scale each feature by its standard deviation, giving standardized data.
Centering and scaling are linear-algebraic preprocessing steps.
They change the coordinate system in which data geometry is measured.
import numpy as np
import matplotlib.pyplot as plt
rng = np.random.default_rng(0)
# Synthetic data with two features
X = rng.multivariate_normal(
mean=[3, 5],
cov=[[4, 1.5], [1.5, 1]],
size=200
)
mu = X.mean(axis=0)
X_centered = X - mu
std = X_centered.std(axis=0, ddof=1)
X_standardized = X_centered / std
mu, std(array([3.18878164, 5.09467867]), array([1.97691788, 0.9654219 ]))plt.figure(figsize=(6, 6))
plt.scatter(X[:, 0], X[:, 1], s=15, alpha=0.6, label="original")
plt.scatter(X_centered[:, 0], X_centered[:, 1], s=15, alpha=0.6, label="centered")
plt.axhline(0, linewidth=0.8)
plt.axvline(0, linewidth=0.8)
plt.grid(True)
plt.axis("equal")
plt.legend()
plt.title("Centering moves the data cloud to the origin")
plt.show()
The dot product measures alignment.
For vectors \(x,y\in\mathbb{R}^n\),
\[ x\cdot y=x^Ty. \]
If \(x\cdot y\) is large and positive, the vectors point in similar directions.
If \(x\cdot y=0\), the vectors are orthogonal.
If \(x\cdot y\) is negative, the vectors point in opposing directions.
In data science, dot products appear in:
When magnitude is less important than direction, we use cosine similarity:
\[ \operatorname{cosine}(x,y) = \frac{x\cdot y}{\|x\|\|y\|}. \]
This is the cosine of the angle between the vectors.
Cosine similarity lies between \(-1\) and \(1\).
For nonzero vectors \(x\) and \(y\),
\[ \operatorname{cosine}(x,y) = \frac{x^Ty}{\|x\|\|y\|}. \]
def cosine_similarity(x, y):
return np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y))
x = np.array([1, 2, 3], dtype=float)
y = np.array([2, 4, 6], dtype=float)
z = np.array([3, -2, 1], dtype=float)
cosine_similarity(x, y), cosine_similarity(x, z)(1.0, 0.14285714285714285)The first value is \(1\) because \(y=2x\).
The second value is closer to \(0\) because \(z\) is less aligned with \(x\).
A linear model predicts an output using a dot product.
For a feature vector
\[ x= \begin{bmatrix} x_1\\ \vdots\\ x_n \end{bmatrix}, \]
a linear prediction has the form
\[ \hat y=w^Tx+b. \]
Here:
If we include a constant feature \(1\), then we can write the model as one dot product:
\[ \hat y=\beta^T\tilde x, \]
where
\[ \tilde x= \begin{bmatrix} 1\\ x_1\\ \vdots\\ x_n \end{bmatrix}, \qquad \beta= \begin{bmatrix} b\\ w_1\\ \vdots\\ w_n \end{bmatrix}. \]
For many observations, the linear model becomes
\[ A\beta\approx y. \]
The least-squares solution minimizes
\[ \|y-A\beta\|^2. \]
The normal equations are
\[ A^TA\hat\beta=A^Ty. \]
Computationally, we usually use QR or SVD rather than explicitly forming \(A^TA\).
Linear regression is the basic example of learning from data by minimizing error.
rng = np.random.default_rng(1)
t = np.linspace(0, 10, 50)
y = 1.5 + 0.8*t + rng.normal(scale=1.0, size=len(t))
A = np.column_stack([np.ones_like(t), t])
beta = np.linalg.lstsq(A, y, rcond=None)[0]
y_hat = A @ beta
plt.figure(figsize=(6, 5))
plt.scatter(t, y, s=20, label="data")
plt.plot(t, y_hat, label="least-squares fit")
plt.xlabel("$t$")
plt.ylabel("$y$")
plt.grid(True)
plt.legend()
plt.title("Linear regression as least squares")
plt.show()
beta
array([1.58556853, 0.77567068])The fitted line is
\[ \hat y=\hat\beta_0+\hat\beta_1t. \]
A model may be nonlinear in the original input but linear in transformed features.
For example, polynomial regression uses features
\[ 1,\ t,\ t^2,\ldots,t^d. \]
The model
\[ \hat y=\beta_0+\beta_1t+\beta_2t^2 \]
is nonlinear in \(t\) but linear in the parameters \(\beta\).
The design matrix is
\[ A= \begin{bmatrix} 1&t_1&t_1^2\\ 1&t_2&t_2^2\\ \vdots&\vdots&\vdots\\ 1&t_m&t_m^2 \end{bmatrix}. \]
Feature engineering often means choosing a useful vector representation of data.
Once data is represented as vectors, linear algebra can be applied.
rng = np.random.default_rng(2)
t = np.linspace(-3, 3, 80)
y = 1 - 0.5*t + 0.8*t**2 + rng.normal(scale=1.5, size=len(t))
A = np.column_stack([np.ones_like(t), t, t**2])
beta = np.linalg.lstsq(A, y, rcond=None)[0]
t_grid = np.linspace(-3.2, 3.2, 200)
A_grid = np.column_stack([np.ones_like(t_grid), t_grid, t_grid**2])
y_fit = A_grid @ beta
plt.figure(figsize=(6, 5))
plt.scatter(t, y, s=18, alpha=0.7, label="data")
plt.plot(t_grid, y_fit, label="quadratic fit")
plt.xlabel("$t$")
plt.ylabel("$y$")
plt.grid(True)
plt.legend()
plt.title("Polynomial regression is linear algebra with transformed features")
plt.show()
beta
array([ 1.13020673, -0.48494825, 0.77831217])High-dimensional data often contains lower-dimensional structure.
For example, data in \(\mathbb{R}^{1000}\) may approximately lie near a low-dimensional subspace.
Dimensionality reduction seeks a simpler representation:
\[ \mathbb{R}^n \to \mathbb{R}^k, \qquad k<n. \]
Linear dimensionality reduction means projecting data onto a lower-dimensional subspace.
The most important linear method is PCA.
PCA finds directions that preserve as much variance as possible.
Let \(X_c\) be a centered data matrix.
The covariance matrix is
\[ C=\frac{1}{m-1}X_c^TX_c. \]
This matrix is symmetric positive semidefinite.
Its eigenvectors give principal directions.
Its eigenvalues give variance along those directions.
If
\[ Cv_i=\lambda_i v_i, \]
then \(v_i\) is a principal component direction and \(\lambda_i\) is the variance explained in that direction.
PCA can also be computed using the SVD.
Let
\[ X_c=U\Sigma V^T. \]
Then
\[ X_c^TX_c=V\Sigma^T\Sigma V^T. \]
Therefore the right singular vectors in \(V\) are eigenvectors of the covariance matrix.
The variances are
\[ \frac{\sigma_i^2}{m-1}. \]
Thus PCA is an SVD of the centered data matrix.
For centered data \(X_c\), the principal component directions are the right singular vectors of \(X_c\).
rng = np.random.default_rng(3)
X = rng.multivariate_normal(
mean=[0, 0],
cov=[[5, 2], [2, 1]],
size=300
)
Xc = X - X.mean(axis=0)
U, s, Vt = np.linalg.svd(Xc, full_matrices=False)
V = Vt.T
explained_variance = s**2 / (len(Xc) - 1)
plt.figure(figsize=(6, 6))
plt.scatter(Xc[:, 0], Xc[:, 1], s=12, alpha=0.45)
for j in range(2):
direction = V[:, j]
length = np.sqrt(explained_variance[j])
plt.arrow(0, 0, length*direction[0], length*direction[1],
head_width=0.08, length_includes_head=True)
plt.text(length*direction[0], length*direction[1], f"$v_{j+1}$")
plt.axhline(0, linewidth=0.8)
plt.axvline(0, linewidth=0.8)
plt.grid(True)
plt.axis("equal")
plt.title("PCA directions from the SVD")
plt.show()
explained_variance
array([5.9726404, 0.1623369])If \(V_k\) contains the first \(k\) principal directions, then the low-dimensional coordinates are
\[ Z=X_cV_k. \]
The reconstruction is
\[ \hat X=ZV_k^T. \]
This is projection onto the subspace spanned by the first \(k\) principal components.
For \(k=1\), each data point is represented by one number.
For larger \(k\), more structure is preserved.
V1 = V[:, :1]
Z = Xc @ V1
X_reconstructed = Z @ V1.T
plt.figure(figsize=(6, 6))
plt.scatter(Xc[:, 0], Xc[:, 1], s=12, alpha=0.35, label="centered data")
plt.scatter(X_reconstructed[:, 0], X_reconstructed[:, 1], s=12, alpha=0.65, label="rank-1 reconstruction")
plt.axhline(0, linewidth=0.8)
plt.axvline(0, linewidth=0.8)
plt.grid(True)
plt.axis("equal")
plt.legend()
plt.title("Projection onto the first principal component")
plt.show()
The reconstructed points lie on the best one-dimensional subspace.
Many data matrices are approximately low rank.
This means that most information can be captured by a small number of singular values and singular vectors.
The truncated SVD
\[ X_k=U_k\Sigma_kV_k^T \]
is the best rank-\(k\) approximation to \(X\).
This gives:
rng = np.random.default_rng(4)
m, n = 80, 50
u1 = rng.normal(size=m)
v1 = rng.normal(size=n)
u2 = rng.normal(size=m)
v2 = rng.normal(size=n)
X_true = 3*np.outer(u1, v1) + 1.5*np.outer(u2, v2)
X_noisy = X_true + 2.0*rng.normal(size=(m, n))
U, s, Vt = np.linalg.svd(X_noisy, full_matrices=False)
def rank_k_approx(U, s, Vt, k):
return (U[:, :k] * s[:k]) @ Vt[:k, :]
X_rank2 = rank_k_approx(U, s, Vt, 2)
np.linalg.norm(X_noisy - X_true), np.linalg.norm(X_rank2 - X_true)(125.66104708574623, 30.439091236936875)The rank-\(2\) approximation can be closer to the true structure than the noisy matrix.
plt.figure(figsize=(6, 4))
plt.plot(s, marker="o")
plt.xlabel("index")
plt.ylabel("singular value")
plt.grid(True)
plt.title("Singular values reveal low-rank structure")
plt.show()
A grayscale image is a matrix.
Each entry is a pixel intensity.
A color image can be represented by three matrices: red, green, and blue channels.
Linear algebra operations on images include:
The SVD compresses an image by keeping only the largest singular values.
n = 140
x = np.linspace(-1, 1, n)
y = np.linspace(-1, 1, n)
Xgrid, Ygrid = np.meshgrid(x, y)
image = (
np.exp(-10*((Xgrid+0.35)**2 + (Ygrid-0.2)**2))
+ 0.8*np.exp(-18*((Xgrid-0.25)**2 + (Ygrid+0.3)**2))
+ 0.25*np.sin(8*Xgrid)*np.cos(5*Ygrid)
)
U, s, Vt = np.linalg.svd(image, full_matrices=False)
ranks = [1, 5, 15, 40]
fig, axes = plt.subplots(1, len(ranks)+1, figsize=(15, 4))
axes[0].imshow(image, cmap="gray")
axes[0].set_title("original")
axes[0].set_xticks([])
axes[0].set_yticks([])
for ax, k in zip(axes[1:], ranks):
img_k = rank_k_approx(U, s, Vt, k)
ax.imshow(img_k, cmap="gray")
ax.set_title(f"rank {k}")
ax.set_xticks([])
ax.set_yticks([])
plt.tight_layout()
plt.show()
This is a visual example of the SVD as compression.
Text can also be represented by vectors.
A simple representation counts word frequencies.
Suppose we have documents and a vocabulary of words.
The document-term matrix is
\[ X_{ij}= \text{number of times word }j\text{ appears in document }i. \]
Then:
More advanced embeddings also represent words, sentences, or documents as vectors, but the geometric ideas are the same: similarity, distance, projection, and dimension.
Suppose the vocabulary is:
\[ \{\text{linear},\ \text{matrix},\ \text{data},\ \text{model}\}. \]
A document might be represented as
\[ x= \begin{bmatrix} 3\\ 2\\ 0\\ 1 \end{bmatrix}, \]
meaning:
Another document is another vector.
Cosine similarity compares their word-use patterns.
docs = np.array([
[3, 2, 0, 1],
[2, 1, 0, 0],
[0, 0, 3, 2],
[1, 1, 2, 2]
], dtype=float)
names = ["linear algebra note", "matrix exercise", "data model note", "mixed document"]
similarities = np.zeros((len(docs), len(docs)))
for i in range(len(docs)):
for j in range(len(docs)):
similarities[i, j] = cosine_similarity(docs[i], docs[j])
similaritiesarray([[1. , 0.95618289, 0.14824986, 0.59160798],
[0.95618289, 1. , 0. , 0.42426407],
[0.14824986, 0. , 1. , 0.87705802],
[0.59160798, 0.42426407, 0.87705802, 1. ]])plt.figure(figsize=(5, 4))
plt.imshow(similarities)
plt.colorbar(label="cosine similarity")
plt.xticks(range(len(names)), names, rotation=45, ha="right")
plt.yticks(range(len(names)), names)
plt.title("Document similarity matrix")
plt.tight_layout()
plt.show()
The similarity matrix is a Gram matrix of normalized document vectors.
A simple recommendation system can be represented by a user-item matrix.
Rows are users.
Columns are items.
Entries are ratings.
For example,
\[ R_{ij} \]
might be the rating user \(i\) gives item \(j\).
Usually many entries are missing.
The goal is to predict missing ratings.
A common idea is that the rating matrix is approximately low rank:
\[ R\approx UV^T. \]
Here:
This is another appearance of matrix factorization.
If user \(i\) has latent vector \(u_i\) and item \(j\) has latent vector \(v_j\), then the predicted rating is
\[ \hat R_{ij}=u_i\cdot v_j. \]
So recommendation is built from dot products in a latent feature space.
The model learns hidden coordinates that explain preferences.
This is the same geometric idea as projecting data into a useful lower-dimensional space.
Graphs can also be represented by matrices.
For a graph with \(n\) nodes, the adjacency matrix is
\[ A_{ij}= \begin{cases} 1, & \text{if node }i\text{ connects to node }j,\\ 0, & \text{otherwise}. \end{cases} \]
For an undirected graph, the adjacency matrix is symmetric.
Another important graph matrix is the degree matrix \(D\), a diagonal matrix whose entries count the number of edges incident to each node.
The graph Laplacian is
\[ L=D-A. \]
The Laplacian is symmetric positive semidefinite for undirected graphs.
Eigenvectors of graph matrices reveal clusters, connectivity, diffusion, and ranking.
# A small graph with two clusters
A_graph = np.array([
[0, 1, 1, 0, 0, 0],
[1, 0, 1, 0, 0, 0],
[1, 1, 0, 1, 0, 0],
[0, 0, 1, 0, 1, 1],
[0, 0, 0, 1, 0, 1],
[0, 0, 0, 1, 1, 0]
], dtype=float)
degrees = A_graph.sum(axis=1)
D_graph = np.diag(degrees)
L = D_graph - A_graph
eigvals, eigvecs = np.linalg.eigh(L)
eigvalsarray([-1.30552502e-16, 4.38447187e-01, 3.00000000e+00, 3.00000000e+00,
3.00000000e+00, 4.56155281e+00])The smallest eigenvalue of the graph Laplacian is \(0\).
The next eigenvector often reveals a meaningful split in the graph.
fiedler = eigvecs[:, 1]
plt.figure(figsize=(6, 4))
plt.bar(np.arange(len(fiedler)), fiedler)
plt.axhline(0, linewidth=0.8)
plt.xlabel("node")
plt.ylabel("Fiedler vector value")
plt.title("A Laplacian eigenvector suggests a graph partition")
plt.grid(True)
plt.show()
A basic neural-network layer has the form
\[ z=Wx+b, \]
followed by a nonlinear activation
\[ h=\phi(z). \]
The matrix \(W\) is a linear transformation.
The vector \(b\) is a translation.
The activation \(\phi\) is nonlinear.
Thus a neural-network layer is:
\[ \text{affine map} \quad + \quad \text{nonlinearity}. \]
Without nonlinearities, a sequence of layers would collapse into one affine map.
For example,
\[ W_2(W_1x+b_1)+b_2 = (W_2W_1)x+(W_2b_1+b_2). \]
The nonlinear activation is what allows neural networks to represent more complex functions.
def relu(z):
return np.maximum(0, z)
W = np.array([
[1.0, -0.5],
[0.3, 1.2],
[-1.0, 0.7]
])
b = np.array([0.2, -0.1, 0.5])
x = np.array([2.0, 1.0])
z = W @ x + b
h = relu(z)
z, h(array([ 1.7, 1.7, -0.8]), array([1.7, 1.7, 0. ]))The matrix multiplication creates linear combinations of input features.
The activation changes the geometry by introducing nonlinear behavior.
Suppose a network had only linear layers and no activation functions.
Then
\[ x\mapsto W_1x \]
followed by
\[ x\mapsto W_2x \]
followed by
\[ x\mapsto W_3x \]
would be
\[ x\mapsto W_3W_2W_1x. \]
This is still just one linear transformation.
Even with biases, a stack of affine transformations is still affine.
Therefore deep learning needs nonlinear activations to go beyond linear algebra.
But the linear parts remain fundamental. They carry most of the parameters and computations.
Many modern sequence models use attention-like operations built from matrix products.
A simplified attention computation uses matrices:
\[ Q=XW_Q, \]
\[ K=XW_K, \]
\[ V=XW_V. \]
Then scores are computed by
\[ S=QK^T. \]
The entry \(S_{ij}\) is a dot product between a query vector and a key vector.
After normalization, the output is a weighted combination of value vectors.
The key linear algebra ideas are:
This shows again that vector geometry is central to AI.
An embedding maps an object to a vector.
Objects may be:
Once objects are vectors, we can compare them using:
\[ \|x-y\| \]
or
\[ \frac{x^Ty}{\|x\|\|y\|}. \]
The goal of embedding is to place similar objects near each other in a meaningful vector space.
Linear algebra gives the geometry of that space.
A linear classifier separates data using a hyperplane.
For binary classification, a simple rule is
\[ \hat y= \begin{cases} 1, & w^Tx+b\ge 0,\\ 0, & w^Tx+b<0. \end{cases} \]
The set
\[ w^Tx+b=0 \]
is a hyperplane.
The vector \(w\) is normal to the hyperplane.
The sign of \(w^Tx+b\) tells which side of the hyperplane the point lies on.
This is another direct use of dot products and geometry.
rng = np.random.default_rng(5)
X1 = rng.normal(loc=[1.5, 1.5], scale=0.6, size=(60, 2))
X2 = rng.normal(loc=[-1.5, -1.0], scale=0.6, size=(60, 2))
w = np.array([1.0, 1.0])
b0 = 0.0
xx = np.linspace(-3, 3, 100)
yy = -(w[0]*xx + b0) / w[1]
plt.figure(figsize=(6, 6))
plt.scatter(X1[:, 0], X1[:, 1], label="class 1")
plt.scatter(X2[:, 0], X2[:, 1], label="class 0")
plt.plot(xx, yy, label="$w^Tx+b=0$")
plt.arrow(0, 0, w[0], w[1], head_width=0.08, length_includes_head=True)
plt.text(w[0]+0.1, w[1]+0.1, "$w$")
plt.axhline(0, linewidth=0.8)
plt.axvline(0, linewidth=0.8)
plt.grid(True)
plt.axis("equal")
plt.legend()
plt.title("A linear classifier uses a hyperplane")
plt.show()
The same ideas appear repeatedly:
| Linear algebra idea | Data/AI meaning |
|---|---|
| Vector | data point, embedding, feature representation |
| Matrix | data set, transformation, model layer |
| Dot product | similarity, prediction, attention score |
| Norm | length, error, regularization |
| Projection | approximation, PCA, least squares |
| Orthogonality | independent directions, stable coordinates |
| Eigenvectors | principal directions, graph modes, steady states |
| Singular values | strength of latent components |
| Low rank | compression, denoising, latent structure |
| Quadratic form | loss curvature, covariance geometry |
Linear algebra is not just a prerequisite for AI.
It is the grammar of many AI computations.
We now combine several ideas:
rng = np.random.default_rng(6)
X = rng.multivariate_normal(
mean=[2, -1],
cov=[[6, 2.5], [2.5, 1.5]],
size=300
)
Xc = X - X.mean(axis=0)
U, s, Vt = np.linalg.svd(Xc, full_matrices=False)
V1 = Vt[:1, :].T
Z = Xc @ V1
Xc_hat = Z @ V1.T
error = np.linalg.norm(Xc - Xc_hat, "fro")
plt.figure(figsize=(6, 6))
plt.scatter(Xc[:, 0], Xc[:, 1], s=12, alpha=0.35, label="centered data")
plt.scatter(Xc_hat[:, 0], Xc_hat[:, 1], s=12, alpha=0.7, label="rank-1 reconstruction")
plt.axhline(0, linewidth=0.8)
plt.axvline(0, linewidth=0.8)
plt.grid(True)
plt.axis("equal")
plt.legend()
plt.title("PCA as projection and low-rank approximation")
plt.show()
error
10.76500220638757This one example uses:
This book developed a sequence of ideas:
Together, these ideas form the computational language of linear algebra.
Given data:
Given a data matrix \(X\):
\[ X_c=X-\mathbf{1}\mu^T. \]
\[ X_c=U\Sigma V^T. \]
\[ Z=X_cV_k. \]
\[ \hat X=ZV_k^T. \]
Given features and responses:
\[ A\beta\approx y \]
in the least-squares sense. 3. Use QR or SVD for numerical stability. 4. Predict with
\[ \hat y=A\hat\beta. \]
Given a matrix \(X\):
\[ X=U\Sigma V^T. \]
\[ X_k=U_k\Sigma_kV_k^T. \]
A data matrix is also a geometric object. Rows are points, columns are features, and matrix operations reveal structure.
Distances and dot products depend on units. Centering and scaling can change the geometry dramatically.
Linear algebra can reveal patterns and structure, but interpretation requires context and careful reasoning.
In high dimensions, distances can behave unexpectedly. Dimension reduction and normalization often matter.
Low-rank approximation requires judgment. Too few components lose signal; too many may preserve noise.
Neural networks rely heavily on linear algebra. Matrix multiplication is the main computational engine.
Data can be represented as a matrix.
Rows are observations, columns are features.
Dot products measure alignment and support prediction, similarity, and attention.
Norms measure length and error.
Least squares fits linear models by projecting data onto a model space.
PCA uses eigenvectors or the SVD to find directions of maximum variance.
The SVD gives low-rank approximations for compression and denoising.
Images, text, recommendation systems, graphs, and neural-network layers can all be described using vectors and matrices.
Linear algebra provides the grammar of data and AI:
\[ \text{vectors}+\text{matrices}+\text{geometry}+\text{optimization}. \]
Suppose
\[ X= \begin{bmatrix} 1&2&3\\ 2&3&5\\ 3&4&7\\ 4&5&9 \end{bmatrix}. \]
For
\[ x= \begin{bmatrix} 1\\2\\3 \end{bmatrix}, \qquad y= \begin{bmatrix} 2\\4\\6 \end{bmatrix}, \qquad z= \begin{bmatrix} 3\\-2\\1 \end{bmatrix}, \]
Given data points
\[ (0,1),\ (1,2),\ (2,2),\ (3,4), \]
\[ y=\beta_0+\beta_1t. \]
Generate or use a two-dimensional data set.
Create a matrix \(X\) or use a small image matrix.
For the adjacency matrix
\[ A= \begin{bmatrix} 0&1&1&0\\ 1&0&1&0\\ 1&1&0&1\\ 0&0&1&0 \end{bmatrix}, \]
Let
\[ W= \begin{bmatrix} 1&-1\\ 2&0\\ 0&1 \end{bmatrix}, \qquad b= \begin{bmatrix} 1\\ 0\\ -1 \end{bmatrix}. \]
For
\[ x= \begin{bmatrix} 2\\ 3 \end{bmatrix}, \]
\[ h_i=\max(0,z_i). \]
np.linalg.lstsq.Write a short paragraph answering each question.
Linear algebra is a language of structure.
It begins with equations, but it grows into a way of seeing:
In the age of data and AI, linear algebra is not merely a technical prerequisite. It is a way to translate problems into vectors, matrices, transformations, and optimization. Once a problem is expressed in that language, computation can begin.