Final Projects

A final project for The Mathematics of Modern Computer Vision should combine mathematical explanation, computational experimentation, and clear visual evidence. The goal is not only to run a model, but to explain why the method works, what assumptions it makes, where it fails, and how the mathematics appears in the implementation.

The final project is the capstone of the book. It asks students to take one theme from computer vision and study it as a complete mathematical-computational investigation. A good project should connect at least three layers:

\[ \text{mathematical model} \longrightarrow \text{algorithm or architecture} \longrightarrow \text{experiment and interpretation}. \]

Students may work individually or in small teams, depending on the course format. Every project should produce a reproducible notebook, a written report, and a short presentation.

Project Philosophy

Computer vision projects often become purely empirical: download a dataset, train a model, and report an accuracy number. In this book, the project should go deeper. Students should ask:

What is the mathematical object being studied?
What assumptions are built into the model or algorithm?
What loss function, metric, or optimization problem is being solved?
What visual evidence supports the result?
What failure cases reveal the limitations of the method?
How does the project connect to the broader development of modern vision?

For example, an object detection project should not only report mean average precision. It should explain bounding-box parameterizations, IoU, non-maximum suppression or set prediction, confidence calibration, localization errors, and qualitative failure modes. A stereo project should not only show depth maps. It should explain disparity, triangulation, uncertainty, cost volumes, smoothness assumptions, and where the correspondence problem breaks down.

Required Deliverables

Each final project should include the following components.

Deliverable	Description
Proposal	A short description of the question, dataset, mathematical methods, planned experiments, and expected challenges.
Reproducible notebook	A Colab-ready or local Jupyter notebook containing all code needed to reproduce the main experiments.
Written report	A polished report explaining the mathematical background, method, experiments, results, limitations, and conclusions.
Presentation	A short presentation with motivation, method, visual results, failure cases, and lessons learned.
Repository folder	A clean project folder with code, report, figures, and instructions.

A project does not need to be large to be excellent. A small project with careful mathematics, clean experiments, and honest analysis is better than a large project with unclear assumptions and unreproducible code.

Suggested Repository Structure

Students should organize their work in a reproducible structure such as:

final-project/
├── README.md
├── report.qmd
├── presentation.pdf
├── notebooks/
│   └── final-project.ipynb
├── src/
│   ├── data.py
│   ├── models.py
│   └── metrics.py
├── figures/
│   ├── example-inputs.png
│   ├── main-results.png
│   └── failure-cases.png
└── data/
    └── README.md

Large datasets should usually not be committed to GitHub. Instead, include download instructions, links to public datasets, or code that loads the data in Colab. The project should clearly explain how another student can reproduce the main results.

Proposal Template

A project proposal should be one to two pages. It should answer the following questions.

1. Project Title

Choose a title that is specific and mathematical. For example:

Stereo Matching and Depth Uncertainty from Synthetic Rectified Image Pairs
Patch Embeddings and Attention Maps in a Small Vision Transformer
Dice Loss, Boundary Error, and Failure Modes in Binary Segmentation

2. Main Question

State the central question in one paragraph. Examples:

How does disparity error affect depth error in stereo vision?
How do augmentation choices affect contrastive image representations?
How do IoU, Dice, and boundary F1 reward different kinds of segmentation predictions?
How does attention change when patch size or positional encoding changes?

3. Mathematical Background

List the main mathematical concepts from the book. Include equations where appropriate, such as:

\[ \operatorname{IoU}(A,B)=\frac{|A\cap B|}{|A\cup B|}, \qquad \operatorname{Dice}(A,B)=\frac{2|A\cap B|}{|A|+|B|}. \]

\[ Z = \frac{fb}{d}, \qquad \Delta Z \approx \left|\frac{\partial Z}{\partial d}\right|\Delta d = \frac{fb}{d^2}\Delta d. \]

4. Dataset or Synthetic Data

Explain whether the project uses:

a public dataset;
a small curated image collection;
synthetic images generated by code;
outputs from earlier labs;
a combination of real and synthetic data.

Synthetic data is encouraged when it makes the mathematics clearer. For example, synthetic camera geometry, synthetic segmentation masks, or synthetic motion fields allow students to know the ground truth exactly.

5. Planned Experiments

Describe three to five experiments. Each experiment should have a clear purpose. For example:

Compare Gaussian smoothing levels before edge detection.
Measure how calibration error changes when checkerboard poses are less diverse.
Compare PCA features, CNN features, and ViT-style patch features for classification.
Evaluate segmentation masks with IoU, Dice, boundary F1, and qualitative overlays.

6. Expected Results and Risks

Explain what you expect to learn and what could go wrong. Examples:

The dataset may be too small for deep training.
The synthetic task may be too easy unless noise or occlusion is added.
The metric may not reflect perceptual quality.
The model may overfit, requiring a smaller architecture or stronger regularization.

Report Template

The final report should be written as a short research-style article. A recommended structure is:

Abstract. A concise summary of the question, method, result, and conclusion.
Introduction. Motivation and connection to computer vision.
Mathematical Background. Definitions, equations, assumptions, and relevant theory.
Method. Algorithm, model, loss function, metrics, and implementation choices.
Experiments. Dataset, preprocessing, experimental design, and baselines.
Results. Tables, figures, visualizations, and quantitative metrics.
Failure Cases. Examples where the method performs poorly and why.
Discussion. Interpretation, limitations, possible improvements, and relation to the book.
Conclusion. Main lessons learned.
References. Books, papers, documentation, and datasets used.

The report should include mathematical notation, but the notation should support the explanation. Avoid long derivations unless they are central to the project.

Presentation Format

A short final presentation can follow this structure:

Slide	Content
1	Title, team members, and project question.
2	Motivation and computer vision task.
3	Mathematical model or key equation.
4	Algorithm, architecture, or pipeline.
5	Dataset or synthetic data generation.
6	Main quantitative result.
7	Visual results.
8	Failure cases and limitations.
9	What you learned mathematically.
10	Future directions.

The presentation should emphasize visual evidence. Show images, masks, attention maps, depth maps, flow fields, correspondences, or generated samples whenever possible.

Evaluation Rubric

A suggested grading rubric is below.

Category	Weight	Excellent work demonstrates
Mathematical understanding	25%	Clear definitions, correct equations, explicit assumptions, and strong connection to book concepts.
Computational implementation	20%	Clean, reproducible code; appropriate algorithms; correct metrics; readable notebooks.
Experimental design	20%	Meaningful baselines, controlled comparisons, useful ablations, and fair interpretation.
Visual and quantitative analysis	15%	Strong figures, overlays, plots, metrics, and failure-case analysis.
Communication	15%	Clear report and presentation, well-organized explanations, and appropriate citations.
Originality and reflection	5%	Thoughtful extensions, creative choices, or honest discussion of limitations.

The rubric can be adjusted for different course levels. In a mathematically oriented course, the mathematical understanding component may be increased. In a project-based AI course, the implementation and experimental design components may be increased.

Milestones

For a semester course, the project can be developed through four milestones.

Week	Milestone	Expected output
4	Topic selection	One paragraph with project question and possible dataset.
7	Proposal	One to two pages with mathematical background and experiment plan.
10	Prototype	Working notebook with data loading, initial method, and first figures.
13	Draft report	Nearly complete report with results and failure cases.
14	Final submission	Report, notebook, presentation, and repository folder.

For independent study, students can use the same milestones at their own pace.

Project Themes

The following project themes are organized by the five parts of the book.

Theme A. Images as Mathematical Objects

These projects connect to Chapters 1–5 and Labs 1–5.

Project A1. Sampling, Aliasing, and Anti-Aliasing

Study how image quality changes under downsampling and interpolation. Generate images with known frequency content, downsample them with and without smoothing, and compare nearest-neighbor, bilinear, bicubic, and Fourier-based reconstructions.

Mathematical focus: sampling, interpolation, Fourier spectra, aliasing, reconstruction error.

Possible outputs: frequency plots, error curves, visual comparisons, anti-aliasing recommendations.

Project A2. SVD and PCA Compression for Image Collections

Compare single-image SVD compression with PCA compression across a collection of related images. Study rank, reconstruction error, energy explained, and visual artifacts.

Mathematical focus: singular values, projections, low-rank approximation, principal components.

Possible outputs: scree plots, reconstruction grids, feature-space plots, compression-quality tradeoff.

Project A3. Color Spaces and Illumination Robustness

Compare RGB, grayscale, HSV, Lab-style coordinates, and normalized color features for a simple visual task such as object/background separation.

Mathematical focus: coordinate transformations, normalization, histograms, invariance.

Possible outputs: color histograms, segmentation masks, metric comparisons under lighting changes.

Theme B. Classical Operators, Shape, and Motion

These projects connect to Chapters 6–11 and Labs 6–11.

Project B1. Edge Detection Under Noise and Blur

Study Sobel, Prewitt, Laplacian, Difference-of-Gaussians, and Canny-style edge detection under controlled noise and blur.

Mathematical focus: convolution, gradients, scale-space, thresholds, non-maximum suppression.

Possible outputs: precision-recall curves for edge maps, qualitative edge overlays, sensitivity plots.

Project B2. Segmentation Metrics and Shape Errors

Create synthetic masks and perturb them by translation, boundary noise, holes, missing components, and merged objects. Compare IoU, Dice, boundary F1, connected-component counts, and Hausdorff-style distances.

Mathematical focus: sets, masks, morphology, topology of components, metric behavior.

Possible outputs: metric tables, mask overlays, examples where metrics disagree.

Project B3. Optical Flow from Brightness Constancy

Implement Lucas–Kanade or Horn–Schunck optical flow on synthetic image sequences with known motion. Study the aperture problem, conditioning, smoothing, and endpoint error.

Mathematical focus: brightness constancy, local least squares, regularization, warping.

Possible outputs: flow visualizations, endpoint-error maps, conditioning diagnostics.

Project B4. Active Contours and Level Sets

Implement a simple contour evolution method for synthetic image segmentation. Compare edge-based and region-based energies.

Mathematical focus: curves, signed distance functions, curvature, variational energy.

Possible outputs: contour evolution plots, convergence curves, failure cases near weak boundaries.

Theme C. Geometry and 3D Vision

These projects connect to Chapters 12–17 and Labs 12–17.

Project C1. Camera Calibration from Synthetic Checkerboards

Generate synthetic checkerboard views with known camera parameters. Estimate intrinsics and extrinsics using homographies or reprojection minimization. Study how accuracy changes with noise and pose diversity.

Mathematical focus: homographies, intrinsic matrix, extrinsic parameters, reprojection error, conditioning.

Possible outputs: calibration residual plots, estimated vs. true parameters, uncertainty discussion.

Project C2. Epipolar Geometry and RANSAC

Generate two-view correspondences with outliers. Estimate the fundamental matrix using the normalized eight-point algorithm and RANSAC. Compare algebraic, geometric, and Sampson residuals.

Mathematical focus: homogeneous coordinates, epipolar constraint, rank constraint, robust estimation.

Possible outputs: epipolar-line visualizations, inlier/outlier plots, residual histograms.

Project C3. Stereo Depth and Uncertainty

Construct rectified stereo pairs from synthetic scenes and recover disparity and depth. Analyze how disparity noise affects depth uncertainty.

Mathematical focus: disparity, triangulation, inverse depth, uncertainty propagation, matching costs.

Possible outputs: disparity maps, depth maps, uncertainty maps, cost-volume slices.

Project C4. Tiny Structure from Motion Pipeline

Build a small synthetic multi-view reconstruction pipeline: generate cameras and 3D points, project to images, add noise, triangulate, and refine using bundle adjustment.

Mathematical focus: projection matrices, feature tracks, gauge freedom, nonlinear least squares, sparse Jacobians.

Possible outputs: reconstructed point clouds, reprojection error before/after refinement, camera-pose plots.

Theme D. Probability, Learning, Detection, and Segmentation

These projects connect to Chapters 18–24 and Labs 18–24.

Project D1. Probabilistic Image Denoising

Compare Gaussian, Laplace, and Poisson noise models for image denoising. Study maximum likelihood, MAP estimation, and uncertainty maps.

Mathematical focus: likelihoods, priors, MAP estimation, residual distributions, calibration.

Possible outputs: denoising comparisons, PSNR/SSIM-style metrics, residual histograms, uncertainty overlays.

Project D2. Classical Features vs. Learned Features for Classification

Compare raw pixels, histograms, PCA features, HOG-style features, CNN-style fixed filters, and learned representations on a small image classification task.

Mathematical focus: feature maps, embeddings, linear classifiers, decision boundaries, generalization.

Possible outputs: accuracy tables, confusion matrices, feature-space visualizations, robustness checks.

Project D3. Training Diagnostics for Small Vision Models

Train a small classifier or segmenter and study learning curves, validation error, regularization, augmentation, label smoothing, and calibration.

Mathematical focus: empirical risk, cross-entropy, optimization, overfitting, calibration.

Possible outputs: loss curves, calibration plots, ablation table, failure examples.

Project D4. Object Detection Metrics and Failure Cases

Implement a small synthetic object-detection benchmark. Study anchors, IoU thresholds, non-maximum suppression, precision-recall curves, average precision, and localization uncertainty.

Mathematical focus: bounding boxes, IoU, assignment, NMS, AP, focal loss.

Possible outputs: detection overlays, PR curves, AP under different IoU thresholds, NMS failure cases.

Project D5. Semantic and Instance Segmentation Evaluation

Train or simulate segmentation predictions and compare cross-entropy, Dice, IoU, boundary F1, connected components, and panoptic-style matching.

Mathematical focus: pixelwise classification, set similarity, mask matching, boundary metrics.

Possible outputs: mask overlays, metric disagreement examples, component-level analysis.

Theme E. Modern Vision AI

These projects connect to Chapters 25–30 and Labs 25–30.

Project E1. Patch Size and Positional Encoding in Vision Transformers

Study how patch size and positional encoding affect attention patterns and classification or reconstruction performance in a small ViT-style model or toy transformer.

Mathematical focus: patch embeddings, attention, positional encoding, token complexity.

Possible outputs: attention maps, accuracy curves, complexity estimates, qualitative token visualizations.

Project E2. Contrastive Learning and Representation Collapse

Implement a small contrastive-learning experiment with augmentations. Compare InfoNCE, alignment/uniformity metrics, covariance eigenvalues, and linear-probe accuracy.

Mathematical focus: normalized embeddings, temperature, contrastive loss, representation geometry.

Possible outputs: embedding plots, eigenvalue spectra, retrieval examples, augmentation ablations.

Project E3. Vision-Language Retrieval with Toy Embeddings

Build a small image-text retrieval system using hand-designed or learned embeddings. Study cosine similarity, prompt templates, retrieval metrics, false positives, and calibration.

Mathematical focus: shared embedding spaces, similarity matrices, contrastive alignment, retrieval metrics.

Possible outputs: similarity heatmaps, top-k retrieval examples, prompt-sensitivity analysis.

Project E4. Promptable Segmentation as Conditional Prediction

Study point, box, and mask prompts for segmentation on synthetic or small real images. Analyze prompt efficiency, click correction, best-of-$K$ masks, and mask-quality calibration.

Mathematical focus: conditional prediction, prompt encoding, IoU/Dice, uncertainty, interaction curves.

Possible outputs: prompt-response visualizations, prompt-efficiency plots, failure cases.

Project E5. Diffusion as Iterative Denoising

Implement a small diffusion-style denoising process for low-dimensional or small image data. Study the forward noising process, SNR, denoising parameterizations, DDPM/DDIM-style sampling, and inpainting.

Mathematical focus: Gaussian noising, score/denoising interpretation, sampling, guidance, inverse problems.

Possible outputs: noising trajectories, denoising grids, inpainting examples, SNR plots.

Project E6. Auditing a Multimodal Vision System

Design a small audit for a vision-language or multimodal model. Study caption consistency, retrieval mistakes, spurious correlations, hallucination risk, calibration, and failure modes.

Mathematical focus: embedding similarity, uncertainty, distribution shift, evaluation metrics, responsible deployment.

Possible outputs: audit checklist, examples of correct/incorrect outputs, risk matrix, recommendations.

Dataset Suggestions

Students may use small public datasets, self-created datasets, or synthetic datasets. Good project datasets should be small enough to run in Colab and clear enough to support mathematical analysis.

Possible sources include:

small built-in datasets from sklearn.datasets, such as digits;
synthetic images generated with NumPy, SciPy, or scikit-image style routines;
small image collections curated by the student;
open computer vision datasets, using a subset only;
medical, scientific, or engineering images when the licensing and privacy conditions are appropriate.

When using any dataset, students should clearly state:

where the data came from;
what license or usage restrictions apply;
whether personal, sensitive, or private information is present;
how train/validation/test splits were created;
what preprocessing was performed.

Reproducibility Expectations

Every project should be reproducible by another student. At minimum, include:

a README.md with setup and run instructions;
fixed random seeds where appropriate;
package imports at the top of the notebook;
a small default experiment that runs quickly;
clear instructions for larger optional experiments;
saved figures or code that regenerates figures;
comments explaining nontrivial implementation steps.

For Colab notebooks, include a badge or link using the pattern:

[Open in Colab](https://colab.research.google.com/github/wanghemath/Book-MathCV/blob/main/path/to/notebook.ipynb)

Responsible Use and Ethics

Computer vision systems can affect real people. Projects should avoid privacy violations, unsafe surveillance applications, misleading claims, or unlicensed data use. Students should include a short responsible-use paragraph when appropriate.

Questions to consider:

Could this system behave differently across groups, lighting conditions, devices, or environments?
Could the dataset contain sensitive information?
Could errors cause harm if the system were deployed?
Are uncertainty and failure cases communicated clearly?
Are generated or edited images clearly identified as synthetic when necessary?

The purpose of the final project is educational. Students should avoid overstating model performance or implying deployment readiness from small experiments.

AI-Assisted Work Policy

AI tools may be useful for brainstorming, debugging, explanation, and code review. However, students are responsible for the correctness of all mathematics, code, and claims in the final submission.

A recommended policy is:

AI tools may be used to help understand concepts, debug code, improve writing clarity, and generate small examples.
Students must verify all mathematical statements and code outputs.
The report should include a brief note describing how AI tools were used, if they were used.
AI-generated text should be revised by the student and checked for accuracy.
AI tools should not replace the student’s own interpretation of results and failure cases.

Mini-Rubric for Self-Assessment

Before submitting, students should check whether they can answer the following questions.

Question	Yes/No
Can I state the main mathematical problem clearly?
Did I define the key variables, losses, metrics, or maps?
Can another student run my notebook?
Did I include both visual and quantitative results?
Did I compare against at least one baseline or ablation?
Did I include failure cases?
Did I explain what the project taught me mathematically?
Did I cite datasets, papers, documentation, and tools?

Example Project Abstracts

Example 1. Stereo Depth and Disparity Uncertainty

This project studies the sensitivity of stereo depth estimation to disparity error. We generate synthetic rectified stereo pairs with known focal length, baseline, and ground-truth depth. We estimate disparity using block matching with SSD and normalized cross-correlation costs, then recover depth using $Z=fb/d$. We compare measured depth error with the first-order uncertainty estimate $\Delta Z \approx fb\Delta d/d^2$. The experiments show that small disparity errors produce large depth errors for distant points, illustrating why stereo reconstruction becomes unstable at large depth.

Example 2. Segmentation Metrics Under Boundary Perturbations

This project compares IoU, Dice, boundary F1, and connected-component metrics for binary segmentation. Starting from synthetic ground-truth masks, we create predictions with translations, holes, boundary noise, missing parts, and merged components. We show that IoU and Dice may remain high even when boundaries are visibly poor, while boundary F1 is sensitive to edge accuracy. The project explains why segmentation evaluation requires multiple metrics and qualitative overlays.

Example 3. Patch Embeddings and Attention in a Small Vision Transformer

This project investigates how patch size affects attention and classification in a toy vision transformer. We build patch embeddings for small grayscale images, add positional encodings, and compute attention maps across image tokens. We compare small and large patch sizes using accuracy, computational cost, and attention visualizations. The results show the tradeoff between local detail and sequence length, connecting ViT design choices to the mathematics of tokenization and attention.

Closing Advice

Choose a project that is small enough to finish but rich enough to explain. The best projects usually start with a simple mathematical question and then build careful experiments around it.

A strong final project should leave the reader with a clear answer to this question:

What did this project reveal about the mathematics of modern computer vision?

# Final Projects {.unnumbered} ::: {.hero-box} A final project for *The Mathematics of Modern Computer Vision* should combine mathematical explanation, computational experimentation, and clear visual evidence. The goal is not only to run a model, but to explain why the method works, what assumptions it makes, where it fails, and how the mathematics appears in the implementation. ::: The final project is the capstone of the book. It asks students to take one theme from computer vision and study it as a complete mathematical-computational investigation. A good project should connect at least three layers: $$ \text{mathematical model} \longrightarrow \text{algorithm or architecture} \longrightarrow \text{experiment and interpretation}. $$ Students may work individually or in small teams, depending on the course format. Every project should produce a reproducible notebook, a written report, and a short presentation. ## Project Philosophy Computer vision projects often become purely empirical: download a dataset, train a model, and report an accuracy number. In this book, the project should go deeper. Students should ask: 1. What is the mathematical object being studied? 2. What assumptions are built into the model or algorithm? 3. What loss function, metric, or optimization problem is being solved? 4. What visual evidence supports the result? 5. What failure cases reveal the limitations of the method? 6. How does the project connect to the broader development of modern vision? For example, an object detection project should not only report mean average precision. It should explain bounding-box parameterizations, IoU, non-maximum suppression or set prediction, confidence calibration, localization errors, and qualitative failure modes. A stereo project should not only show depth maps. It should explain disparity, triangulation, uncertainty, cost volumes, smoothness assumptions, and where the correspondence problem breaks down. ## Required Deliverables Each final project should include the following components. | Deliverable | Description | |---|---| | **Proposal** | A short description of the question, dataset, mathematical methods, planned experiments, and expected challenges. | | **Reproducible notebook** | A Colab-ready or local Jupyter notebook containing all code needed to reproduce the main experiments. | | **Written report** | A polished report explaining the mathematical background, method, experiments, results, limitations, and conclusions. | | **Presentation** | A short presentation with motivation, method, visual results, failure cases, and lessons learned. | | **Repository folder** | A clean project folder with code, report, figures, and instructions. | A project does not need to be large to be excellent. A small project with careful mathematics, clean experiments, and honest analysis is better than a large project with unclear assumptions and unreproducible code. ## Suggested Repository Structure Students should organize their work in a reproducible structure such as: ```text final-project/ ├── README.md ├── report.qmd ├── presentation.pdf ├── notebooks/ │ └── final-project.ipynb ├── src/ │ ├── data.py │ ├── models.py │ └── metrics.py ├── figures/ │ ├── example-inputs.png │ ├── main-results.png │ └── failure-cases.png └── data/ └── README.md ``` Large datasets should usually **not** be committed to GitHub. Instead, include download instructions, links to public datasets, or code that loads the data in Colab. The project should clearly explain how another student can reproduce the main results. ## Proposal Template A project proposal should be one to two pages. It should answer the following questions. ### 1. Project Title Choose a title that is specific and mathematical. For example: - *Stereo Matching and Depth Uncertainty from Synthetic Rectified Image Pairs* - *Patch Embeddings and Attention Maps in a Small Vision Transformer* - *Dice Loss, Boundary Error, and Failure Modes in Binary Segmentation* ### 2. Main Question State the central question in one paragraph. Examples: - How does disparity error affect depth error in stereo vision? - How do augmentation choices affect contrastive image representations? - How do IoU, Dice, and boundary F1 reward different kinds of segmentation predictions? - How does attention change when patch size or positional encoding changes? ### 3. Mathematical Background List the main mathematical concepts from the book. Include equations where appropriate, such as: $$ \operatorname{IoU}(A,B)=\frac{|A\cap B|}{|A\cup B|}, \qquad \operatorname{Dice}(A,B)=\frac{2|A\cap B|}{|A|+|B|}. $$ or $$ Z = \frac{fb}{d}, \qquad \Delta Z \approx \left|\frac{\partial Z}{\partial d}\right|\Delta d = \frac{fb}{d^2}\Delta d. $$ ### 4. Dataset or Synthetic Data Explain whether the project uses: - a public dataset; - a small curated image collection; - synthetic images generated by code; - outputs from earlier labs; - a combination of real and synthetic data. Synthetic data is encouraged when it makes the mathematics clearer. For example, synthetic camera geometry, synthetic segmentation masks, or synthetic motion fields allow students to know the ground truth exactly. ### 5. Planned Experiments Describe three to five experiments. Each experiment should have a clear purpose. For example: - Compare Gaussian smoothing levels before edge detection. - Measure how calibration error changes when checkerboard poses are less diverse. - Compare PCA features, CNN features, and ViT-style patch features for classification. - Evaluate segmentation masks with IoU, Dice, boundary F1, and qualitative overlays. ### 6. Expected Results and Risks Explain what you expect to learn and what could go wrong. Examples: - The dataset may be too small for deep training. - The synthetic task may be too easy unless noise or occlusion is added. - The metric may not reflect perceptual quality. - The model may overfit, requiring a smaller architecture or stronger regularization. ## Report Template The final report should be written as a short research-style article. A recommended structure is: 1. **Abstract.** A concise summary of the question, method, result, and conclusion. 2. **Introduction.** Motivation and connection to computer vision. 3. **Mathematical Background.** Definitions, equations, assumptions, and relevant theory. 4. **Method.** Algorithm, model, loss function, metrics, and implementation choices. 5. **Experiments.** Dataset, preprocessing, experimental design, and baselines. 6. **Results.** Tables, figures, visualizations, and quantitative metrics. 7. **Failure Cases.** Examples where the method performs poorly and why. 8. **Discussion.** Interpretation, limitations, possible improvements, and relation to the book. 9. **Conclusion.** Main lessons learned. 10. **References.** Books, papers, documentation, and datasets used. The report should include mathematical notation, but the notation should support the explanation. Avoid long derivations unless they are central to the project. ## Presentation Format A short final presentation can follow this structure: | Slide | Content | |---:|---| | 1 | Title, team members, and project question. | | 2 | Motivation and computer vision task. | | 3 | Mathematical model or key equation. | | 4 | Algorithm, architecture, or pipeline. | | 5 | Dataset or synthetic data generation. | | 6 | Main quantitative result. | | 7 | Visual results. | | 8 | Failure cases and limitations. | | 9 | What you learned mathematically. | | 10 | Future directions. | The presentation should emphasize visual evidence. Show images, masks, attention maps, depth maps, flow fields, correspondences, or generated samples whenever possible. ## Evaluation Rubric A suggested grading rubric is below. | Category | Weight | Excellent work demonstrates | |---|---:|---| | **Mathematical understanding** | 25% | Clear definitions, correct equations, explicit assumptions, and strong connection to book concepts. | | **Computational implementation** | 20% | Clean, reproducible code; appropriate algorithms; correct metrics; readable notebooks. | | **Experimental design** | 20% | Meaningful baselines, controlled comparisons, useful ablations, and fair interpretation. | | **Visual and quantitative analysis** | 15% | Strong figures, overlays, plots, metrics, and failure-case analysis. | | **Communication** | 15% | Clear report and presentation, well-organized explanations, and appropriate citations. | | **Originality and reflection** | 5% | Thoughtful extensions, creative choices, or honest discussion of limitations. | The rubric can be adjusted for different course levels. In a mathematically oriented course, the mathematical understanding component may be increased. In a project-based AI course, the implementation and experimental design components may be increased. ## Milestones For a semester course, the project can be developed through four milestones. | Week | Milestone | Expected output | |---:|---|---| | 4 | Topic selection | One paragraph with project question and possible dataset. | | 7 | Proposal | One to two pages with mathematical background and experiment plan. | | 10 | Prototype | Working notebook with data loading, initial method, and first figures. | | 13 | Draft report | Nearly complete report with results and failure cases. | | 14 | Final submission | Report, notebook, presentation, and repository folder. | For independent study, students can use the same milestones at their own pace. ## Project Themes The following project themes are organized by the five parts of the book. ### Theme A. Images as Mathematical Objects These projects connect to Chapters 1--5 and Labs 1--5. #### Project A1. Sampling, Aliasing, and Anti-Aliasing Study how image quality changes under downsampling and interpolation. Generate images with known frequency content, downsample them with and without smoothing, and compare nearest-neighbor, bilinear, bicubic, and Fourier-based reconstructions. **Mathematical focus:** sampling, interpolation, Fourier spectra, aliasing, reconstruction error. **Possible outputs:** frequency plots, error curves, visual comparisons, anti-aliasing recommendations. #### Project A2. SVD and PCA Compression for Image Collections Compare single-image SVD compression with PCA compression across a collection of related images. Study rank, reconstruction error, energy explained, and visual artifacts. **Mathematical focus:** singular values, projections, low-rank approximation, principal components. **Possible outputs:** scree plots, reconstruction grids, feature-space plots, compression-quality tradeoff. #### Project A3. Color Spaces and Illumination Robustness Compare RGB, grayscale, HSV, Lab-style coordinates, and normalized color features for a simple visual task such as object/background separation. **Mathematical focus:** coordinate transformations, normalization, histograms, invariance. **Possible outputs:** color histograms, segmentation masks, metric comparisons under lighting changes. ### Theme B. Classical Operators, Shape, and Motion These projects connect to Chapters 6--11 and Labs 6--11. #### Project B1. Edge Detection Under Noise and Blur Study Sobel, Prewitt, Laplacian, Difference-of-Gaussians, and Canny-style edge detection under controlled noise and blur. **Mathematical focus:** convolution, gradients, scale-space, thresholds, non-maximum suppression. **Possible outputs:** precision-recall curves for edge maps, qualitative edge overlays, sensitivity plots. #### Project B2. Segmentation Metrics and Shape Errors Create synthetic masks and perturb them by translation, boundary noise, holes, missing components, and merged objects. Compare IoU, Dice, boundary F1, connected-component counts, and Hausdorff-style distances. **Mathematical focus:** sets, masks, morphology, topology of components, metric behavior. **Possible outputs:** metric tables, mask overlays, examples where metrics disagree. #### Project B3. Optical Flow from Brightness Constancy Implement Lucas--Kanade or Horn--Schunck optical flow on synthetic image sequences with known motion. Study the aperture problem, conditioning, smoothing, and endpoint error. **Mathematical focus:** brightness constancy, local least squares, regularization, warping. **Possible outputs:** flow visualizations, endpoint-error maps, conditioning diagnostics. #### Project B4. Active Contours and Level Sets Implement a simple contour evolution method for synthetic image segmentation. Compare edge-based and region-based energies. **Mathematical focus:** curves, signed distance functions, curvature, variational energy. **Possible outputs:** contour evolution plots, convergence curves, failure cases near weak boundaries. ### Theme C. Geometry and 3D Vision These projects connect to Chapters 12--17 and Labs 12--17. #### Project C1. Camera Calibration from Synthetic Checkerboards Generate synthetic checkerboard views with known camera parameters. Estimate intrinsics and extrinsics using homographies or reprojection minimization. Study how accuracy changes with noise and pose diversity. **Mathematical focus:** homographies, intrinsic matrix, extrinsic parameters, reprojection error, conditioning. **Possible outputs:** calibration residual plots, estimated vs. true parameters, uncertainty discussion. #### Project C2. Epipolar Geometry and RANSAC Generate two-view correspondences with outliers. Estimate the fundamental matrix using the normalized eight-point algorithm and RANSAC. Compare algebraic, geometric, and Sampson residuals. **Mathematical focus:** homogeneous coordinates, epipolar constraint, rank constraint, robust estimation. **Possible outputs:** epipolar-line visualizations, inlier/outlier plots, residual histograms. #### Project C3. Stereo Depth and Uncertainty Construct rectified stereo pairs from synthetic scenes and recover disparity and depth. Analyze how disparity noise affects depth uncertainty. **Mathematical focus:** disparity, triangulation, inverse depth, uncertainty propagation, matching costs. **Possible outputs:** disparity maps, depth maps, uncertainty maps, cost-volume slices. #### Project C4. Tiny Structure from Motion Pipeline Build a small synthetic multi-view reconstruction pipeline: generate cameras and 3D points, project to images, add noise, triangulate, and refine using bundle adjustment. **Mathematical focus:** projection matrices, feature tracks, gauge freedom, nonlinear least squares, sparse Jacobians. **Possible outputs:** reconstructed point clouds, reprojection error before/after refinement, camera-pose plots. ### Theme D. Probability, Learning, Detection, and Segmentation These projects connect to Chapters 18--24 and Labs 18--24. #### Project D1. Probabilistic Image Denoising Compare Gaussian, Laplace, and Poisson noise models for image denoising. Study maximum likelihood, MAP estimation, and uncertainty maps. **Mathematical focus:** likelihoods, priors, MAP estimation, residual distributions, calibration. **Possible outputs:** denoising comparisons, PSNR/SSIM-style metrics, residual histograms, uncertainty overlays. #### Project D2. Classical Features vs. Learned Features for Classification Compare raw pixels, histograms, PCA features, HOG-style features, CNN-style fixed filters, and learned representations on a small image classification task. **Mathematical focus:** feature maps, embeddings, linear classifiers, decision boundaries, generalization. **Possible outputs:** accuracy tables, confusion matrices, feature-space visualizations, robustness checks. #### Project D3. Training Diagnostics for Small Vision Models Train a small classifier or segmenter and study learning curves, validation error, regularization, augmentation, label smoothing, and calibration. **Mathematical focus:** empirical risk, cross-entropy, optimization, overfitting, calibration. **Possible outputs:** loss curves, calibration plots, ablation table, failure examples. #### Project D4. Object Detection Metrics and Failure Cases Implement a small synthetic object-detection benchmark. Study anchors, IoU thresholds, non-maximum suppression, precision-recall curves, average precision, and localization uncertainty. **Mathematical focus:** bounding boxes, IoU, assignment, NMS, AP, focal loss. **Possible outputs:** detection overlays, PR curves, AP under different IoU thresholds, NMS failure cases. #### Project D5. Semantic and Instance Segmentation Evaluation Train or simulate segmentation predictions and compare cross-entropy, Dice, IoU, boundary F1, connected components, and panoptic-style matching. **Mathematical focus:** pixelwise classification, set similarity, mask matching, boundary metrics. **Possible outputs:** mask overlays, metric disagreement examples, component-level analysis. ### Theme E. Modern Vision AI These projects connect to Chapters 25--30 and Labs 25--30. #### Project E1. Patch Size and Positional Encoding in Vision Transformers Study how patch size and positional encoding affect attention patterns and classification or reconstruction performance in a small ViT-style model or toy transformer. **Mathematical focus:** patch embeddings, attention, positional encoding, token complexity. **Possible outputs:** attention maps, accuracy curves, complexity estimates, qualitative token visualizations. #### Project E2. Contrastive Learning and Representation Collapse Implement a small contrastive-learning experiment with augmentations. Compare InfoNCE, alignment/uniformity metrics, covariance eigenvalues, and linear-probe accuracy. **Mathematical focus:** normalized embeddings, temperature, contrastive loss, representation geometry. **Possible outputs:** embedding plots, eigenvalue spectra, retrieval examples, augmentation ablations. #### Project E3. Vision-Language Retrieval with Toy Embeddings Build a small image-text retrieval system using hand-designed or learned embeddings. Study cosine similarity, prompt templates, retrieval metrics, false positives, and calibration. **Mathematical focus:** shared embedding spaces, similarity matrices, contrastive alignment, retrieval metrics. **Possible outputs:** similarity heatmaps, top-k retrieval examples, prompt-sensitivity analysis. #### Project E4. Promptable Segmentation as Conditional Prediction Study point, box, and mask prompts for segmentation on synthetic or small real images. Analyze prompt efficiency, click correction, best-of-$K$ masks, and mask-quality calibration. **Mathematical focus:** conditional prediction, prompt encoding, IoU/Dice, uncertainty, interaction curves. **Possible outputs:** prompt-response visualizations, prompt-efficiency plots, failure cases. #### Project E5. Diffusion as Iterative Denoising Implement a small diffusion-style denoising process for low-dimensional or small image data. Study the forward noising process, SNR, denoising parameterizations, DDPM/DDIM-style sampling, and inpainting. **Mathematical focus:** Gaussian noising, score/denoising interpretation, sampling, guidance, inverse problems. **Possible outputs:** noising trajectories, denoising grids, inpainting examples, SNR plots. #### Project E6. Auditing a Multimodal Vision System Design a small audit for a vision-language or multimodal model. Study caption consistency, retrieval mistakes, spurious correlations, hallucination risk, calibration, and failure modes. **Mathematical focus:** embedding similarity, uncertainty, distribution shift, evaluation metrics, responsible deployment. **Possible outputs:** audit checklist, examples of correct/incorrect outputs, risk matrix, recommendations. ## Dataset Suggestions Students may use small public datasets, self-created datasets, or synthetic datasets. Good project datasets should be small enough to run in Colab and clear enough to support mathematical analysis. Possible sources include: - small built-in datasets from `sklearn.datasets`, such as digits; - synthetic images generated with NumPy, SciPy, or scikit-image style routines; - small image collections curated by the student; - open computer vision datasets, using a subset only; - medical, scientific, or engineering images when the licensing and privacy conditions are appropriate. When using any dataset, students should clearly state: 1. where the data came from; 2. what license or usage restrictions apply; 3. whether personal, sensitive, or private information is present; 4. how train/validation/test splits were created; 5. what preprocessing was performed. ## Reproducibility Expectations Every project should be reproducible by another student. At minimum, include: - a `README.md` with setup and run instructions; - fixed random seeds where appropriate; - package imports at the top of the notebook; - a small default experiment that runs quickly; - clear instructions for larger optional experiments; - saved figures or code that regenerates figures; - comments explaining nontrivial implementation steps. For Colab notebooks, include a badge or link using the pattern: ```markdown [Open in Colab](https://colab.research.google.com/github/wanghemath/Book-MathCV/blob/main/path/to/notebook.ipynb) ``` ## Responsible Use and Ethics Computer vision systems can affect real people. Projects should avoid privacy violations, unsafe surveillance applications, misleading claims, or unlicensed data use. Students should include a short responsible-use paragraph when appropriate. Questions to consider: - Could this system behave differently across groups, lighting conditions, devices, or environments? - Could the dataset contain sensitive information? - Could errors cause harm if the system were deployed? - Are uncertainty and failure cases communicated clearly? - Are generated or edited images clearly identified as synthetic when necessary? The purpose of the final project is educational. Students should avoid overstating model performance or implying deployment readiness from small experiments. ## AI-Assisted Work Policy AI tools may be useful for brainstorming, debugging, explanation, and code review. However, students are responsible for the correctness of all mathematics, code, and claims in the final submission. A recommended policy is: 1. AI tools may be used to help understand concepts, debug code, improve writing clarity, and generate small examples. 2. Students must verify all mathematical statements and code outputs. 3. The report should include a brief note describing how AI tools were used, if they were used. 4. AI-generated text should be revised by the student and checked for accuracy. 5. AI tools should not replace the student's own interpretation of results and failure cases. ## Mini-Rubric for Self-Assessment Before submitting, students should check whether they can answer the following questions. | Question | Yes/No | |---|---| | Can I state the main mathematical problem clearly? | | | Did I define the key variables, losses, metrics, or maps? | | | Can another student run my notebook? | | | Did I include both visual and quantitative results? | | | Did I compare against at least one baseline or ablation? | | | Did I include failure cases? | | | Did I explain what the project taught me mathematically? | | | Did I cite datasets, papers, documentation, and tools? | | ## Example Project Abstracts ### Example 1. Stereo Depth and Disparity Uncertainty This project studies the sensitivity of stereo depth estimation to disparity error. We generate synthetic rectified stereo pairs with known focal length, baseline, and ground-truth depth. We estimate disparity using block matching with SSD and normalized cross-correlation costs, then recover depth using $Z=fb/d$. We compare measured depth error with the first-order uncertainty estimate $\Delta Z \approx fb\Delta d/d^2$. The experiments show that small disparity errors produce large depth errors for distant points, illustrating why stereo reconstruction becomes unstable at large depth. ### Example 2. Segmentation Metrics Under Boundary Perturbations This project compares IoU, Dice, boundary F1, and connected-component metrics for binary segmentation. Starting from synthetic ground-truth masks, we create predictions with translations, holes, boundary noise, missing parts, and merged components. We show that IoU and Dice may remain high even when boundaries are visibly poor, while boundary F1 is sensitive to edge accuracy. The project explains why segmentation evaluation requires multiple metrics and qualitative overlays. ### Example 3. Patch Embeddings and Attention in a Small Vision Transformer This project investigates how patch size affects attention and classification in a toy vision transformer. We build patch embeddings for small grayscale images, add positional encodings, and compute attention maps across image tokens. We compare small and large patch sizes using accuracy, computational cost, and attention visualizations. The results show the tradeoff between local detail and sequence length, connecting ViT design choices to the mathematics of tokenization and attention. ## Final Submission Checklist A complete final submission should include: - [ ] project title and team members; - [ ] project question and motivation; - [ ] mathematical background with equations; - [ ] dataset description and preprocessing; - [ ] reproducible notebook; - [ ] method or algorithm description; - [ ] metrics and baselines; - [ ] visual results; - [ ] quantitative results; - [ ] failure cases; - [ ] limitations and future work; - [ ] responsible-use discussion when relevant; - [ ] references; - [ ] presentation slides or PDF. ## Closing Advice Choose a project that is small enough to finish but rich enough to explain. The best projects usually start with a simple mathematical question and then build careful experiments around it. A strong final project should leave the reader with a clear answer to this question: > What did this project reveal about the mathematics of modern computer vision?