Computer Labs

The Mathematics of Modern Computer Vision

Computer Labs

This page collects the Google Colab notebooks for The Mathematics of Modern Computer Vision. The book repository is:

https://github.com/wanghemath/Book-MathCV

Each lab is designed for independent study. The notebooks include the necessary mathematical background before programming, guided examples, visualizations, exercises, AI-assisted reflection prompts, and a mini-project.

How to use the labs

There are two standard ways to open a lab.

1. Click Open in Colab in the table below.
2. In Colab, choose File \(\to\) Save a copy in Drive before editing.

The direct Colab link pattern is:

https://colab.research.google.com/github/wanghemath/Book-MathCV/blob/main/labs/<notebook-name>.ipynb

For example:

https://colab.research.google.com/github/wanghemath/Book-MathCV/blob/main/labs/lab-01-what-is-computer-vision.ipynb

Lab sequence

Lab	Title	Main focus	Colab	Notebook
1	What Is Computer Vision?	Computer vision as inverse problem, prediction problem, and representation problem.	Open in Colab	lab-01-what-is-computer-vision.ipynb
2	Pixels, Coordinates, and Color Spaces	Pixel grids, image coordinates, RGB channels, color spaces, quantization, histograms, and normalization.	Open in Colab	lab-02-pixels-coordinates-color.ipynb
3	Images as Functions on Domains	Images as sampled functions, interpolation, resizing, aliasing, gradients, warping, and finite differences.	Open in Colab	lab-03-images-as-functions.ipynb
4	Linear Algebra of Images	Vectorization, norms, inner products, SVD compression, PCA, projections, and patch embeddings.	Open in Colab	lab-04-linear-algebra-of-images.ipynb
5	Fourier Analysis and Frequency	Fourier bases, 1D/2D DFT, magnitude and phase, frequency filters, compression, and aliasing.	Open in Colab	lab-05-fourier-analysis-frequency.ipynb
6	Convolution, Filtering, and Local Operators	Convolution/correlation, local filters, separability, derivative filters, stride, dilation, and CNN-style filter banks.	Open in Colab	lab-06-convolution-filtering-local-operators.ipynb
7	Edges, Corners, and Local Features	Gradients, Canny-style edges, structure tensor, Harris/Shi–Tomasi corners, descriptors, and matching.	Open in Colab	lab-07-edges-corners-local-features.ipynb
8	Image Segmentation as Optimization	Masks, Otsu thresholding, IoU/Dice, Gaussian mixtures, unary terms, smoothness, and segmentation losses.	Open in Colab	lab-08-segmentation-as-optimization.ipynb
9	Morphology and Shape	Dilation, erosion, opening, closing, connected components, distance transforms, skeletons, and shape descriptors.	Open in Colab	lab-09-morphology-and-shape.ipynb
10	Curves, Surfaces, and Level Sets	Parametric/implicit curves, curvature, signed distance functions, active contours, and 3D level sets.	Open in Colab	lab-10-curves-surfaces-level-sets.ipynb
11	Optical Flow and Motion	Brightness constancy, aperture problem, Lucas–Kanade flow, Horn–Schunck regularization, warping, and flow metrics.	Open in Colab	lab-11-optical-flow-and-motion.ipynb
12	Cameras as Maps	Pinhole cameras, homogeneous coordinates, projection matrices, back-projection rays, vanishing points, and distortion.	Open in Colab	lab-12-cameras-as-maps.ipynb
13	Camera Calibration	Checkerboards, homographies, intrinsic/extrinsic recovery, distortion, reprojection error, and calibration uncertainty.	Open in Colab	lab-13-camera-calibration.ipynb
14	Epipolar Geometry	Fundamental/essential matrices, epipolar lines, eight-point estimation, Sampson error, RANSAC, and pose ambiguity.	Open in Colab	lab-14-epipolar-geometry.ipynb
15	Stereo Vision and Depth	Rectified stereo, disparity, depth uncertainty, matching costs, cost volumes, subpixel disparity, and photometric loss.	Open in Colab	lab-15-stereo-vision-and-depth.ipynb
16	Structure from Motion	Feature tracks, multi-view triangulation, PnP, gauge ambiguity, factorization, and simplified bundle adjustment.	Open in Colab	lab-16-structure-from-motion.ipynb
17	Bundle Adjustment	Reprojection residuals, Gauss–Newton/LM, sparse Jacobians, Schur complement, robust loss, and uncertainty.	Open in Colab	lab-17-bundle-adjustment.ipynb
18	Probabilistic Models of Images	Image likelihoods, priors, MAP denoising, Wiener filtering, GMM segmentation, entropy, and uncertainty.	Open in Colab	lab-18-probabilistic-models-of-images.ipynb
19	Classification from a Mathematical View	Bayes classification, logistic/softmax regression, LDA/QDA, metrics, calibration, imbalance, and robustness.	Open in Colab	lab-19-classification-mathematical-view.ipynb
20	Representation Learning	PCA embeddings, linear probing, retrieval, clustering, invariance, autoencoders, metric learning, and masked modeling.	Open in Colab	lab-20-representation-learning.ipynb
21	Convolutional Neural Networks	Multichannel convolution, equivariance, pooling, receptive fields, sparse operators, residual blocks, and CNN features.	Open in Colab	lab-21-convolutional-neural-networks.ipynb
22	Training Vision Models	Empirical risk, train/validation/test splits, cross-entropy, backpropagation, optimization, regularization, augmentation, and calibration.	Open in Colab	lab-22-training-vision-models.ipynb
23	Object Detection	Bounding boxes, IoU, anchors, target assignment, NMS, regression losses, AP, and DETR-style matching.	Open in Colab	lab-23-object-detection.ipynb
24	Semantic and Instance Segmentation	Pixel labels, Dice/IoU, pixelwise classification, connected components, mask matching, panoptic quality, and promptable masks.	Open in Colab	lab-24-semantic-instance-segmentation.ipynb
25	Vision Transformers	Patch tokens, positional encodings, attention, multi-head attention, transformer blocks, complexity, and dense tokens.	Open in Colab	lab-25-vision-transformers.ipynb
26	Self-Supervised Vision	Contrastive pairs, InfoNCE, alignment/uniformity, collapse diagnostics, Barlow Twins, VICReg, BYOL intuition, and masked modeling.	Open in Colab	lab-26-self-supervised-vision.ipynb
27	Vision-Language Models	Image-text pairs, two-tower encoders, CLIP-style loss, zero-shot classification, retrieval metrics, and cross-attention.	Open in Colab	lab-27-vision-language-models.ipynb
28	Promptable Segmentation and Foundation Models	Point/box/mask prompts, prompt-conditioned masks, mask ranking, attention, prompt-efficiency, and calibration.	Open in Colab	lab-28-promptable-segmentation-foundation-models.ipynb
29	Diffusion Models for Images	Gaussian noising, DDPM/DDIM sampling, parameterizations, classifier-free guidance, latent diffusion, and inpainting.	Open in Colab	lab-29-diffusion-models-for-images.ipynb
30	Vision in the Age of Multimodal AI	Multimodal spaces, fusion, retrieval, VQA, captioning, video/3D tokens, calibration, hallucination-risk auditing, and deployment.	Open in Colab	lab-30-vision-age-multimodal-ai.ipynb

Suggested learning paths

Full course path

Complete Labs 1–30 in order. This path follows the full mathematical development from pixels and geometry to modern multimodal vision AI.

Classical computer vision path

Complete Labs 1–18. This path emphasizes image formation, filtering, local features, segmentation, geometry, stereo, structure from motion, optimization, and probabilistic image models.

Modern AI vision path

Complete Labs 19–30. This path emphasizes classification, representation learning, CNNs, training methods, object detection, segmentation, transformers, self-supervised learning, vision-language models, promptable foundation models, diffusion, and multimodal AI.

Geometry-focused path

Complete Labs 11–17. This path is useful for students interested in camera geometry, 3D reconstruction, robotics, autonomous systems, and geometric deep learning.

Foundation-model path

Complete Labs 25–30. This path is useful for students interested in vision transformers, self-supervised learning, CLIP-style models, promptable segmentation, diffusion models, and multimodal systems.

Expected prerequisites

Students should be comfortable with basic Python, NumPy arrays, matrix multiplication, elementary probability, derivatives, and optimization. The notebooks review the essential ideas before using them.

The labs intentionally avoid heavy external dependencies when possible. Most labs use only common Colab packages such as NumPy, SciPy, Matplotlib, and scikit-learn. A few optional deep-learning sections may use TensorFlow or PyTorch, depending on the topic.

Repository organization

The recommended repository layout is:

Book-MathCV/
├── index.qmd
├── introduction.qmd
├── labs-summary.qmd
├── chapters/
│   ├── chapter-01-....qmd
│   ├── chapter-02-....qmd
│   └── ...
└── labs/
    ├── README.md
    ├── lab-01-what-is-computer-vision.ipynb
    ├── lab-02-pixels-coordinates-color.ipynb
    └── ...

Instructor notes

These labs can be assigned in several formats:

• Reading companion: students complete one lab after each chapter.
• Weekly computer lab: students complete selected labs in class and finish exercises independently.
• Project preparation: students use the mini-project sections to build final project proposals.
• Independent study: students follow the notebook sequence and use the AI-assisted prompts for self-explanation.

For public GitHub hosting, make sure the notebook files are committed under labs/ on the main branch. The Colab links above assume that exact location.