Computational Linear Algebra · Chapter 1

From Equations to Linear Systems

Linear algebra begins with a simple question: when several relationships must be true at the same time, how do we organize them, solve them, and understand the answer geometrically?

Opening question

Several unknowns, several relationships

Suppose several quantities are unknown, but we know relationships among them: traffic flows, nutritional constraints, balancing forces, market prices, or measurements in a data model. A linear system is the organized language for these relationships.

Model

Turn words, diagrams, or data into equations.

Solve

Use elimination to reduce a complicated system to a simpler one.

Interpret

Understand solutions as intersections of lines, planes, or hyperplanes.

Learning goals

Conceptual

  • Recognize linear and nonlinear equations.
  • Explain one solution, no solution, and infinitely many solutions.
  • Interpret systems geometrically.
  • Understand free parameters.

Computational

  • Write systems from verbal or network problems.
  • Solve small systems by substitution and elimination.
  • Check answers by substitution.
  • Use Python to solve and visualize systems.

1. Linear equations

Definition. A linear equation in variables \(x_1,x_2,\ldots,x_n\) has the form \[ a_1x_1+a_2x_2+\cdots+a_nx_n=b, \] where the coefficients \(a_1,\ldots,a_n\) and the right-hand side \(b\) are real numbers.

Linear examples

\[ 3x+2=0,\qquad 2x_1+3x_2=1,\qquad 2x_1+3x_2-x_3=1. \]

Nonlinear examples

\[ x^2+y=1,\qquad xy=3,\qquad \sin(x)+y=0. \]
Quick test: variables may appear only to the first power, and variables cannot multiply each other.

2. Systems of linear equations

A linear system is a collection of linear equations involving the same variables. A solution is a choice of variable values that makes every equation true at the same time.

\[ \begin{cases} x_1-3x_2-5x_3=1,\\ x_1-x_2-2x_3=0,\\ 3x_1-x_2+x_3=3. \end{cases} \]

3. Solving by elimination

Elimination replaces a system by a simpler system with the same solutions. The goal is to remove variables step by step until the answer becomes clear.

Interactive stepper: solve the first example

Click through the elimination process for

\[ \begin{cases} x_1-3x_2-5x_3=1,\\ x_1-x_2-2x_3=0,\\ 3x_1-x_2+x_3=3. \end{cases} \]
Step 1

Always check: substituting \((x_1,x_2,x_3)=(0,-2,1)\) gives \(1\), \(0\), and \(3\) on the three right-hand sides.

4. Geometry of solution sets

A linear equation in two variables is a line. Solving two equations means finding where two lines intersect.

SituationGeometryAlgebraic meaning
One solutionNonparallel lines meet once.Exactly one ordered pair satisfies both equations.
No solutionParallel distinct lines never meet.Elimination produces a contradiction such as \(0=1\).
Infinitely manyThe two equations describe the same line.Elimination produces a repeated equation such as \(0=0\).

Interactive line-system explorer

Change the coefficients in

\[ a_1x+b_1y=c_1,\qquad a_2x+b_2y=c_2. \]

Planes and hyperplanes

A linear equation in three variables, \(ax+by+cz=d\), is a plane in \(\mathbb R^3\). In \(n\) variables, a linear equation is a hyperplane in \(\mathbb R^n\).

Big geometric idea: solving a linear system means finding an intersection: lines in \(\mathbb R^2\), planes in \(\mathbb R^3\), and hyperplanes in \(\mathbb R^n\).

5. Parameters and infinitely many solutions

When a system has fewer independent equations than unknowns, some variables may be chosen freely. These are free variables, and they become parameters.

\[ x_1-2x_2+3x_3=0 \quad\Longrightarrow\quad x_1=2x_2-3x_3. \] Let \(x_2=s\) and \(x_3=t\). Then \[ \begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix} =s\begin{bmatrix}2\\1\\0\end{bmatrix} +t\begin{bmatrix}-3\\0\\1\end{bmatrix}. \]

Parameter explorer

Move \(s\) and \(t\) to generate different solutions of \(x_1-2x_2+3x_3=0\).

Common mistake: a free variable is not automatically zero. It can be any real number unless the application adds constraints.

6. Inconsistent and homogeneous systems

Inconsistent system

If elimination produces a false equation such as \(0=1\), the system has no solution.

\[ \begin{cases} x_1-2x_2+3x_3=0,\\ -2x_1+4x_2-6x_3=1 \end{cases} \]

Homogeneous system

A system with right-hand side zero has the form \(Ax=0\). It always has the trivial solution \(x=0\).

\[ Ax=0 \]

7. Modeling example: traffic flow

Linear systems often come from conservation laws. At each intersection, use flow in = flow out.

\[ \begin{aligned} 53+x_1&=96+x_4,\\ x_4+121&=135+x_3,\\ x_3+45&=x_2+78,\\ 152+x_2&=62+x_1. \end{aligned} \] Hence \[ (x_1,x_2,x_3,x_4)=(t+43,t-47,t-14,t). \]

Traffic-flow slider

The algebraic solution has one free parameter \(t=x_4\). The real traffic interpretation requires all flows to be nonnegative.

53 x₁ 152 62 135 121 x₃ 78 45 x₄ x₂
Modeling lesson: algebra may give a family of solutions, but applications often add constraints such as nonnegative traffic, positive mass, or probabilities between 0 and 1.

8. Row picture and column picture

Consider

\[ \begin{cases} 3x_1+x_2=7,\\ x_1+2x_2=4. \end{cases} \]

Row picture

Each equation is a line. The solution is the intersection of the two lines.

\[ (x_1,x_2)=(2,1). \]

Column picture

The right-hand side is built as a linear combination of columns:

\[ 2\begin{bmatrix}3\\1\end{bmatrix} +1\begin{bmatrix}1\\2\end{bmatrix} =\begin{bmatrix}7\\4\end{bmatrix}. \]

9. A first Python computation

For numerical computation, represent the system as \(Ax=b\), then solve it with NumPy.

import numpy as np

A = np.array([
    [1, -3, -5],
    [1, -1, -2],
    [3, -1,  1]
], dtype=float)

b = np.array([1, 0, 3], dtype=float)

x = np.linalg.solve(A, b)
print(x)          # [ 0. -2.  1.]
print(A @ x)      # checks that Ax equals b
Numerical note: exact symbolic elimination and floating-point computation are related but not identical. Later, matrix algorithms will help us understand both.

10. Chapter summary

Algebra

  • A linear equation has the form \(a_1x_1+\cdots+a_nx_n=b\).
  • A linear system is a collection of such equations.
  • A solution satisfies all equations simultaneously.
  • Elimination is the first algorithm of linear algebra.

Geometry

  • In two variables, equations are lines.
  • In three variables, equations are planes.
  • In \(n\) variables, equations are hyperplanes.
  • Solution sets are intersections.

Exercises

A. Recognizing linear equations
  1. \(3x_1-2x_2+7x_3=5\)
  2. \(x_1x_2+x_3=1\)
  3. \(4x-5y=0\)
  4. \(x^2+y=3\)
  5. \(\sin(x)+y=1\)
  6. \(2x_1+3x_2-8=0\)
B. Solving small systems
  1. \(\begin{cases}x+y=5,\\2x-y=1.\end{cases}\)
  2. \(\begin{cases}x-2y=3,\\2x-4y=6.\end{cases}\)
  3. \(\begin{cases}x-2y=3,\\2x-4y=8.\end{cases}\)
  4. \(\begin{cases}x_1+x_2+x_3=2,\\x_1-x_2+2x_3=1,\\2x_1+x_2+3x_3=4.\end{cases}\)
C. Geometry
  1. Explain why two equations in two variables can have one solution, no solution, or infinitely many solutions.
  2. Give an example of two equations whose solution set is the line \(y=2x+1\).
  3. Describe geometrically the solution set of \(z=1\) in variables \(x,y,z\).
  4. Describe what it means for two planes in \(\mathbb R^3\) to have infinitely many common points.
D. Parameters
  1. Write \(x_1-2x_2+3x_3=0\) in parametric vector form.
  2. Solve \(\begin{cases}x_1+x_2+x_3=1,\\2x_1+2x_2+2x_3=2.\end{cases}\)
  3. Classify \(\begin{cases}x_1+x_2-x_3=0,\\2x_1+2x_2-2x_3=1.\end{cases}\)
E. Modeling and Python
  1. Set up and solve a three-flow conservation system.
  2. Use np.linalg.solve to solve the first chapter system.
  3. Check the answer by computing A @ x - b.
  4. Plot \(x+y=5\) and \(2x-y=1\).

AI-age connection

Linear systems are the first computational language of linear algebra. In data science and AI, exact systems are often replaced by approximate systems with many more equations than unknowns. That leads naturally to least squares, regression, projections, QR factorization, and the singular value decomposition.