3  Pixels, Coordinates, and Color Spaces

Before a vision algorithm can reason about objects, scenes, depth, masks, or captions, it must first represent an image correctly. This chapter develops the mathematical language of pixels, coordinate systems, channels, quantization, color spaces, histograms, and normalization. The goal is to make the most basic data structure in computer vision precise enough for later geometry, statistics, optimization, and deep learning.

3.1 Learning Goals

After reading this chapter, students should be able to:

1. distinguish array indices, image-plane coordinates, normalized coordinates, and physical camera coordinates;
2. represent grayscale and color images as matrices, tensors, vector-valued functions, and random arrays;
3. explain pixel centers, pixel cells, quantization, dynamic range, and interpolation;
4. convert between common color representations such as RGB, grayscale, HSV, and luminance-chrominance coordinates;
5. interpret image histograms as empirical probability distributions;
6. explain why modern AI vision systems require careful preprocessing, normalization, and coordinate conventions.

3.2 2.1 Why This Chapter Matters

Computer vision is full of sophisticated mathematical ideas: projective geometry, Fourier analysis, stochastic models, variational optimization, convolutional neural networks, attention, and diffusion. Yet many practical errors arise from a much simpler source: confusing what an image coordinate means.

For example, the same pixel can be described by:

• a row-column index \((i,j)\) in an array;
• an image-plane coordinate \((x,y)\) in pixels;
• a normalized coordinate \((\xi,\eta)\) in \([-1,1]^2\) or \([0,1]^2\);
• a camera coordinate obtained after applying an intrinsic calibration matrix;
• a physical location on a sensor.

A clean mathematical treatment separates these layers.

The first principle of image representation is this: an image is not only a matrix. It is a matrix together with conventions for coordinates, sampling, channel order, intensity scale, and color interpretation.

A grayscale image of height \(H\) and width \(W\) may be stored as a matrix

\[ I \in \mathbb{R}^{H \times W}, \]

but this notation alone does not say whether \(I[0,0]\) is the upper-left pixel, whether intensities are in \([0,255]\) or \([0,1]\), whether they are linear intensities or gamma-corrected values, or whether coordinates refer to pixel corners or pixel centers.

These details matter in later chapters. Camera geometry depends on coordinate conventions. Convolution depends on the relationship between discrete samples and spatial neighborhoods. Neural networks depend on channel order and normalization. Statistical inference depends on the noise model and data scale.

3.3 2.2 Images as Discrete Arrays

A digital grayscale image is usually stored as a rectangular array

\[ I = \bigl(I[i,j]\bigr)_{0 \le i < H,\ 0 \le j < W}. \]

Here:

• \(i\) is the row index;
• \(j\) is the column index;
• \(H\) is the image height;
• \(W\) is the image width.

Most image-processing software uses the convention that \(i=0\) is the top row and \(j=0\) is the left column. Thus the array coordinate system is often downward-positive in the vertical direction.

A color image with \(C\) channels is represented as a third-order tensor

\[ I \in \mathbb{R}^{H \times W \times C}, \]

with entries

\[ I[i,j,c], \qquad 0 \le c < C. \]

For RGB images, \(C=3\), and a pixel value is a vector

\[ I[i,j,:] = \begin{bmatrix} R[i,j] \\ G[i,j] \\ B[i,j] \end{bmatrix} \in \mathbb{R}^3. \]

Many libraries disagree about channel order. Some use RGB, while others use BGR. Some store images as \(H \times W \times C\), while deep-learning frameworks often use either \(N \times C \times H \times W\) or \(N \times H \times W \times C\) for batches.

For a batch of \(n\) images, one common mathematical representation is

\[ \mathcal{I} \in \mathbb{R}^{n \times H \times W \times C}. \]

A convolutional neural network may instead use

\[ \mathcal{I} \in \mathbb{R}^{n \times C \times H \times W}. \]

Both contain the same data, but the indexing conventions differ. A preprocessing error can silently turn a correct model into a poor one.

3.4 2.3 Pixel Cells and Pixel Centers

It is tempting to identify pixel \((i,j)\) with the point \((j,i)\), but a pixel is better viewed as a small cell in the image plane.

In a common convention, pixel \((i,j)\) occupies the rectangle

\[ [j, j+1] \times [i, i+1], \]

and its center is

\[ (x_j,y_i) = \left(j+\frac{1}{2},\ i+\frac{1}{2}\right). \]

The difference between pixel corners and pixel centers matters in resampling, camera calibration, and geometric transformations.

Pixel-center convention. If an image has width \(W\) and height \(H\), then the center of the upper-left pixel is often \((1/2,1/2)\), and the center of the lower-right pixel is \((W-1/2,H-1/2)\).

A coordinate transformation may act on continuous image-plane coordinates rather than discrete indices. For example, a translation by \((a,b)\) sends

\[ (x,y) \mapsto (x+a,y+b). \]

The resulting coordinate usually does not land exactly on a pixel center. Therefore one must interpolate.

The interactive grid above emphasizes a basic distinction:

\[ \text{array index } (i,j) \quad \neq \quad \text{continuous coordinate } (x,y). \]

The row-column index lives in a finite set of integer pairs. The image coordinate lives in a continuous rectangle.

3.5 2.4 Coordinate Systems

There are several coordinate systems in computer vision.

3.5.1 Array coordinates

Array coordinates are usually written as \((i,j)\), where \(i\) is row and \(j\) is column. The index set is

\[ \Lambda_{H,W} = \{0,1,\ldots,H-1\} \times \{0,1,\ldots,W-1\}. \]

The image is a function on this discrete grid:

\[ I: \Lambda_{H,W} \to \mathbb{R}^C. \]

3.5.2 Image-plane coordinates

Image-plane coordinates are usually written as \((x,y)\), where \(x\) increases to the right and \(y\) often increases downward in digital images. A common domain is

\[ \Omega = [0,W] \times [0,H]. \]

With the pixel-center convention, the discrete sample corresponding to row \(i\) and column \(j\) is located at

\[ (x,y)=\left(j+\frac{1}{2}, i+\frac{1}{2}\right). \]

3.5.3 Normalized image coordinates

Many algorithms use normalized coordinates. One common choice maps pixel coordinates to \([-1,1]^2\):

\[ \xi = 2\frac{x}{W}-1, \qquad \eta = 2\frac{y}{H}-1. \]

Another common choice maps to \([0,1]^2\):

\[ u = \frac{x}{W}, \qquad v = \frac{y}{H}. \]

The first convention is common in neural-network grid sampling. The second is common in annotation formats and normalized bounding boxes.

3.5.4 Camera-normalized coordinates

In 3D vision, pixel coordinates are related to camera-normalized coordinates through the intrinsic calibration matrix

\[ K = \begin{bmatrix} f_x & s & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}. \]

A homogeneous pixel coordinate \(\tilde x = (u,v,1)^T\) is mapped to a normalized camera ray direction by

\[ \tilde x_{\mathrm{cam}} = K^{-1}\tilde x. \]

This idea will become central in Chapter 12. For now, the key point is that pixel coordinates are not the same as physical camera rays.

3.6 2.5 Images as Sampled Functions

The discrete image array is often treated as samples of a continuous function

\[ f: \Omega \subset \mathbb{R}^2 \to \mathbb{R}^C. \]

A sampled image can be written as

\[ I[i,j,c] = Q_c\left(f_c\left(j+\frac{1}{2}, i+\frac{1}{2}\right)\right), \]

where \(Q_c\) is a quantization operator for channel \(c\).

This formula separates three ideas:

1. the underlying continuous scene-derived signal \(f\);
2. sampling at a grid of pixel centers;
3. quantization into stored numerical values.

For MS Statistics students, this is a measurement model. The stored pixel value is not the true scene radiance. It is a noisy, sampled, quantized observation of a latent signal.

A simple additive noise model is

\[ I[i,j,c] = f_c(x_j,y_i) + \varepsilon[i,j,c]. \]

A more realistic model may include shot noise, sensor nonlinearities, demosaicing, compression, and gamma correction. For mathematical vision, the simple model is often the first useful approximation.

3.7 2.6 Quantization and Dynamic Range

Many images are stored using 8-bit unsigned integers. Each channel value belongs to

\[ \{0,1,2,\ldots,255\}. \]

A common normalization maps an integer value \(p\) to

\[ z = \frac{p}{255} \in [0,1]. \]

If intensities are stored with \(b\) bits, then there are \(2^b\) possible levels. The quantization step for values scaled to \([0,1]\) is approximately

\[ \Delta = \frac{1}{2^b-1}. \]

Quantization can be modeled by

\[ Q(z) = \Delta \operatorname{round}\left(\frac{z}{\Delta}\right). \]

The quantization error is

\[ e(z) = Q(z)-z. \]

If the error is approximated as uniform on \([-\Delta/2,\Delta/2]\), then

\[ \mathbb{E}[e] = 0, \qquad \operatorname{Var}(e) = \frac{\Delta^2}{12}. \]

This approximation is not always valid, but it is useful for understanding why higher bit depth can preserve subtle intensity differences.

3.8 2.7 Linear Intensity, Gamma, and sRGB

A major source of confusion is the difference between linear intensity and display-encoded intensity.

If \(L \in [0,1]\) is a linear light intensity, a simplified gamma encoding is

\[ s = L^{1/\gamma}, \]

where \(\gamma\) is often near \(2.2\). The inverse map is

\[ L = s^\gamma. \]

Thus averaging two gamma-encoded pixel values is generally not the same as averaging two physical light intensities.

For many learning tasks, a model may still perform well using standard encoded RGB images, because the training and test distributions share the same convention. For physically meaningful image formation, however, linear intensity matters.

Mathematical operations such as averaging, convolution, and interpolation have different interpretations depending on whether the image values are linear intensities or nonlinear display-encoded values.

3.9 2.8 Color Images as Vector-Valued Signals

A color image can be viewed as a vector-valued function

\[ I: \Lambda_{H,W} \to \mathbb{R}^3, \qquad I[i,j] = (R[i,j],G[i,j],B[i,j])^T. \]

At each pixel, the color vector is a point in a color space. In normalized RGB, the color cube is

\[ [0,1]^3. \]

The vertices of this cube correspond to black, red, green, blue, cyan, magenta, yellow, and white.

Linear operations on color vectors include channel mixing. For example, a grayscale intensity can be computed as

\[ Y = w_R R + w_G G + w_B B, \]

where \(w_R,w_G,w_B \ge 0\) and

\[ w_R+w_G+w_B=1. \]

A common luminance-inspired choice is

\[ Y = 0.299R + 0.587G + 0.114B. \]

This is not the only possible grayscale conversion, but it reflects the fact that human visual perception is more sensitive to green than to blue.

3.10 2.9 Linear Color Transforms

Many color spaces are obtained by applying a transformation to the RGB vector. A simple linear transform has the form

\[ z = A r, \]

where

\[ r = \begin{bmatrix} R \\ G \\ B \end{bmatrix}, \qquad z = \begin{bmatrix} z_1 \\ z_2 \\ z_3 \end{bmatrix}, \qquad A \in \mathbb{R}^{3 \times 3}. \]

For example, a luminance-chrominance representation separates brightness from approximate color differences:

\[ \begin{bmatrix} Y \\ C_1 \\ C_2 \end{bmatrix} = \begin{bmatrix} 0.299 & 0.587 & 0.114 \\ -0.1687 & -0.3313 & 0.5 \\ 0.5 & -0.4187 & -0.0813 \end{bmatrix} \begin{bmatrix} R \\ G \\ B \end{bmatrix}. \]

The first coordinate \(Y\) carries brightness information. The remaining coordinates carry chrominance information. This is useful in compression and in some segmentation tasks.

A color transform is a change of coordinates in color space. Linear transforms are matrix multiplications. Nonlinear transforms, such as RGB to HSV, are piecewise nonlinear coordinate changes.

3.11 2.10 HSV as a Nonlinear Coordinate System

RGB coordinates describe a color by amounts of red, green, and blue. HSV coordinates describe a color by hue, saturation, and value.

Let

\[ M = \max(R,G,B), \qquad m = \min(R,G,B), \qquad \Delta = M-m. \]

The value coordinate is

\[ V=M. \]

The saturation coordinate is often defined as

\[ S = \begin{cases} 0, & M=0, \\ \Delta/M, & M>0. \end{cases} \]

The hue coordinate is defined piecewise. For normalized hue measured in degrees,

\[ H = \begin{cases} 60^\circ \left((G-B)/\Delta \bmod 6\right), & M=R,\ \Delta>0, \\ 60^\circ \left((B-R)/\Delta + 2\right), & M=G,\ \Delta>0, \\ 60^\circ \left((R-G)/\Delta + 4\right), & M=B,\ \Delta>0, \\ \text{undefined}, & \Delta=0. \end{cases} \]

Hue is undefined for grayscale colors because when \(R=G=B\), there is no dominant color direction.

HSV is useful for certain intuitive tasks, but it is not a linear vector space coordinate system. Averages and distances in HSV require care, especially because hue is circular.

3.12 2.11 Histograms as Empirical Distributions

A grayscale histogram counts how many pixels fall into each intensity bin. Suppose an image has \(N=HW\) pixels and values in \(\{0,1,\ldots,L-1\}\). The histogram is

\[ h(k) = \#\{(i,j): I[i,j]=k\}, \qquad k=0,\ldots,L-1. \]

The normalized histogram is

\[ \widehat p(k) = \frac{h(k)}{N}. \]

This is an empirical probability mass function:

\[ \widehat p(k) \ge 0, \qquad \sum_{k=0}^{L-1}\widehat p(k)=1. \]

From this viewpoint, the sample mean intensity is

\[ \widehat \mu = \sum_{k=0}^{L-1} k\widehat p(k), \]

and the sample variance is

\[ \widehat \sigma^2 = \sum_{k=0}^{L-1}(k-\widehat \mu)^2\widehat p(k). \]

Histograms ignore spatial arrangement. Two very different images may have the same histogram. Nevertheless, histograms are useful for contrast analysis, thresholding, normalization, and simple image retrieval.

The normalized histogram is the empirical distribution of pixel intensities. Histogram equalization, thresholding, and contrast normalization can all be interpreted as transformations of this empirical distribution.

3.13 2.12 Histogram Equalization: A First Distribution Transform

Let \(X\) be a discrete intensity random variable with probability mass function \(p(k)\). Its cumulative distribution function is

\[ F(k)=\mathbb{P}(X \le k)=\sum_{\ell=0}^k p(\ell). \]

Histogram equalization transforms an observed intensity \(k\) by approximately

\[ T(k) = (L-1)F(k). \]

For an image, we replace \(F\) by the empirical CDF

\[ \widehat F(k)=\sum_{\ell=0}^k \widehat p(\ell). \]

Then the equalized intensity is

\[ I_{\mathrm{eq}}[i,j] = \operatorname{round}\left((L-1)\widehat F(I[i,j])\right). \]

The purpose is to spread intensities more evenly across the available range. This can increase contrast, but it can also amplify noise or produce unnatural appearances.

This is one of the earliest examples in the book where a vision operation is a probability transformation.

3.14 2.13 Normalization for Machine Learning

Modern vision models rarely use raw integer pixel values directly. A common preprocessing pipeline is

\[ I_{\mathrm{uint8}} \longrightarrow I_{[0,1]} \longrightarrow I_{\mathrm{standardized}}. \]

The first step maps

\[ I_{[0,1]} = \frac{I_{\mathrm{uint8}}}{255}. \]

The second step standardizes each channel:

\[ \widetilde I[i,j,c] = \frac{I[i,j,c]-\mu_c}{\sigma_c}, \]

where \(\mu_c\) and \(\sigma_c\) are channel-specific means and standard deviations, usually estimated from the training data.

For a training set of \(n\) images, the channel mean is

\[ \widehat \mu_c = \frac{1}{nHW}\sum_{m=1}^n\sum_{i=0}^{H-1}\sum_{j=0}^{W-1} I_m[i,j,c], \]

and the channel variance is

\[ \widehat \sigma_c^2 = \frac{1}{nHW}\sum_{m=1}^n\sum_{i=0}^{H-1}\sum_{j=0}^{W-1} \left(I_m[i,j,c]-\widehat \mu_c\right)^2. \]

This is not merely a software detail. Standardization changes the scale of the optimization problem used to train a network.

For a neural network, preprocessing is part of the mathematical model. The classifier is not only \(f_\theta(I)\); it is often \(f_\theta(T(I))\), where \(T\) is a deterministic preprocessing transformation.

3.15 2.14 Coordinate Maps and Data Augmentation

Data augmentation applies transformations to images during training. Common examples include translations, rotations, flips, crops, color jitter, and rescaling.

Mathematically, a geometric augmentation is a map

\[ \tau: \Omega \to \Omega. \]

It induces an image transformation

\[ (T_\tau I)(x) = I(\tau^{-1}(x)). \]

The inverse appears because to compute the output value at location \(x\), we ask which input location maps to \(x\).

A horizontal flip of an image of width \(W\) is approximately

\[ \tau(x,y)=(W-x,y), \]

up to the precise pixel-center convention.

A brightness augmentation may instead act on color vectors:

\[ I[i,j,:] \mapsto a I[i,j,:] + b\mathbf{1}, \]

where \(a\) controls contrast and \(b\) controls brightness.

For classification, these transformations are often label-preserving:

\[ y(T_\tau I)=y(I). \]

For segmentation, object detection, pose estimation, or depth estimation, the label must transform together with the image.

A data augmentation is only valid if the transformed image-label pair still represents the intended task. A horizontal flip may be harmless for cats and dogs, but not for distinguishing left-hand from right-hand, reading text, or detecting asymmetric medical findings.

3.16 2.15 Small Python Examples

The following examples are designed for independent study. They use only standard scientific Python ideas. In a Quarto book, these chunks are marked with eval: false so that the chapter renders even if optional image libraries are not installed.

3.16.1 Example 1: Pixel indexing and vectorization

import numpy as np

H, W, C = 4, 5, 3
I = np.zeros((H, W, C), dtype=float)

# Put a red pixel at row i=1, column j=3.
i, j = 1, 3
I[i, j, :] = [1.0, 0.0, 0.0]

print("image shape:", I.shape)
print("pixel value at row 1, column 3:", I[1, 3, :])

# Vectorize the image as one long vector.
x = I.reshape(-1)
print("vector dimension:", x.shape[0])

For an RGB image of shape \((H,W,3)\), vectorization produces a vector in

\[ \mathbb{R}^{3HW}. \]

The vectorized representation is useful for linear algebra. The tensor representation is useful for spatial and channel structure.

3.16.2 Example 2: Grayscale conversion as a matrix operation

import numpy as np

# Three RGB pixels.
RGB = np.array([
    [1.0, 0.0, 0.0],   # red
    [0.0, 1.0, 0.0],   # green
    [0.0, 0.0, 1.0],   # blue
])

w = np.array([0.299, 0.587, 0.114])
Y = RGB @ w
print(Y)

The formula is

\[ Y = Rw_R + Gw_G + Bw_B. \]

In matrix form, for \(N\) pixels stored as rows of a matrix \(X \in \mathbb{R}^{N \times 3}\),

\[ y = Xw. \]

3.16.3 Example 3: Channel standardization

import numpy as np

rng = np.random.default_rng(7243)
images = rng.uniform(0, 1, size=(10, 32, 32, 3))

# Estimate channel means and standard deviations over all images and pixels.
mu = images.mean(axis=(0, 1, 2))
sigma = images.std(axis=(0, 1, 2), ddof=0)

images_std = (images - mu) / sigma

print("channel means before:", mu)
print("channel stds before:", sigma)
print("channel means after:", images_std.mean(axis=(0, 1, 2)))
print("channel stds after:", images_std.std(axis=(0, 1, 2)))

The standardized data has approximately zero mean and unit variance per channel. This usually improves the conditioning of optimization in neural-network training.

3.16.4 Example 4: Histogram as an empirical distribution

import numpy as np

rng = np.random.default_rng(5110)
I = rng.normal(loc=120, scale=25, size=(128, 128))
I = np.clip(np.round(I), 0, 255).astype(np.uint8)

hist, bin_edges = np.histogram(I, bins=256, range=(0, 255))
p_hat = hist / hist.sum()

mean_from_hist = np.sum(np.arange(256) * p_hat)
var_from_hist = np.sum((np.arange(256) - mean_from_hist)**2 * p_hat)

print("mean:", mean_from_hist)
print("variance:", var_from_hist)
print("probabilities sum to:", p_hat.sum())

This example treats pixel intensities as samples from an empirical distribution.

3.16.5 Example 5: A minimal preprocessing class

import numpy as np

class ImageStandardizer:
    def fit(self, X):
        # X has shape (n, H, W, C)
        self.mean_ = X.mean(axis=(0, 1, 2))
        self.std_ = X.std(axis=(0, 1, 2))
        self.std_ = np.maximum(self.std_, 1e-8)
        return self

    def transform(self, X):
        return (X - self.mean_) / self.std_

    def fit_transform(self, X):
        return self.fit(X).transform(X)

This makes explicit that preprocessing has parameters estimated from training data. The same parameters must be used on validation and test images.

3.17 2.16 AI Component: Preprocessing as Part of the Model

Modern AI models are sensitive to input representation. If a pretrained model expects RGB images normalized by specific channel means and standard deviations, then feeding unnormalized images changes the model mathematically.

Suppose a model is trained on standardized images

\[ Z = T(I), \qquad Z[i,j,c]=\frac{I[i,j,c]-\mu_c}{\sigma_c}. \]

The learned classifier is really

\[ g_\theta(I)=f_\theta(T(I)). \]

If test-time preprocessing uses different parameters \((\mu_c',\sigma_c')\), then the deployed classifier is

\[ g_\theta'(I)=f_\theta(T'(I)), \]

which may not match the training distribution.

AI principle. In computer vision, an AI model includes its input convention. Image size, color order, intensity scaling, normalization, and coordinate system are part of the model specification.

3.17.1 AI-assisted study prompt

Ask an AI assistant to check the following preprocessing pipeline:

I train a CNN on images loaded as RGB arrays in \([0,1]\), standardized with training-set channel means. At test time, I load images as BGR arrays in \([0,255]\) and apply the same mean and standard deviation. What mathematical mismatch occurs?

A strong answer should mention both the channel-order mismatch and the scale mismatch.

3.17.2 AI-assisted debugging prompt

Use this prompt when debugging image code:

I have an image tensor with shape (32, 224, 224, 3). Explain what each axis likely means. Then give two common ways this tensor must be rearranged for different deep-learning frameworks. Finally, list three bugs that could occur if I confuse channel order or batch order.

The goal is not to outsource understanding. The goal is to force the conventions to become explicit.

3.17.3 AI-assisted modeling prompt

Use this prompt to connect this chapter to statistical modeling:

Treat a grayscale image as noisy samples of a latent smooth function. Write a simple statistical model for the observed pixels, define the parameters or latent variables, and propose a loss function for estimating the clean image.

This prompt prepares for denoising, smoothing, variational methods, and diffusion models later in the book.

3.18 2.17 Mathematical Checklist for Image Data

When receiving a new vision dataset, ask the following questions.

Shape

What is the tensor shape? Is it \((H,W,C)\), \((C,H,W)\), \((N,H,W,C)\), or \((N,C,H,W)\)?

Coordinates

Are labels expressed in pixel coordinates, normalized coordinates, or physical coordinates?

Channels

Are images grayscale, RGB, BGR, multispectral, hyperspectral, or something else?

Scale

Are values in \(\{0,\ldots,255\}\), \([0,1]\), \([-1,1]\), or standardized units?

Color

Are values linear intensities, sRGB-like display values, or transformed color coordinates?

Labels

Do labels transform when the image is cropped, resized, rotated, or flipped?

This checklist is especially important for applied projects and final projects, where students often combine images, labels, annotations, and pretrained models from different sources.

3.19 2.18 Summary

This chapter introduced the basic mathematical representation of digital images.

A grayscale image can be viewed as a function

\[ I: \Lambda_{H,W} \to \mathbb{R}, \]

and a color image as a function

\[ I: \Lambda_{H,W} \to \mathbb{R}^3. \]

The same image can also be interpreted as samples of a continuous vector-valued function

\[ f: \Omega \to \mathbb{R}^3, \]

or as a random array generated by a measurement process.

The main lessons are:

1. row-column indices and geometric coordinates are different;
2. pixel centers and pixel cells must be distinguished;
3. color images are vector-valued signals;
4. color transforms are coordinate changes in color space;
5. histograms are empirical distributions;
6. preprocessing and normalization are part of the mathematical definition of an AI vision model.

The next chapter moves from discrete pixel arrays to images as functions on continuous domains, preparing for interpolation, gradients, sampling, and differential operators.

3.20 Exercises

3.20.1 Conceptual exercises

1. Explain the difference between an array coordinate \((i,j)\) and an image-plane coordinate \((x,y)\).
2. Why is pixel \((i,j)\) better interpreted as a cell rather than a mathematical point?
3. Give one example where using pixel corners instead of pixel centers could create an error.
4. Explain why a grayscale conversion is a linear map from \(\mathbb{R}^3\) to \(\mathbb{R}\).
5. Why is HSV not a linear color coordinate system?
6. Give an example of a task where horizontal flipping is a valid augmentation and one where it is not.

3.20.2 Mathematical exercises

7. Let \(I \in \mathbb{R}^{H \times W \times 3}\). If \(I\) is vectorized, what is the dimension of the resulting vector space?
8. Derive the inverse map from normalized coordinates \((\xi,\eta) \in [-1,1]^2\) to image coordinates \((x,y) \in [0,W]\times[0,H]\).
9. Suppose \(b=8\). Compute the quantization step \(\Delta\) for intensities scaled to \([0,1]\).
10. Assuming uniform quantization error on \([-\Delta/2,\Delta/2]\), derive \(\operatorname{Var}(e)=\Delta^2/12\).
11. Let \(X \in \mathbb{R}^{N \times 3}\) store RGB pixels as rows. Let \(w=(0.299,0.587,0.114)^T\). Show that \(Xw\) gives a vector of grayscale values.
12. Prove that the normalized histogram \(\widehat p(k)=h(k)/N\) is a probability mass function.
13. For a discrete intensity random variable with CDF \(F\), explain why the transformation \(T(k)=(L-1)F(k)\) tends to spread intensities over the available range.

3.20.3 Python exercises

14. Write a Python function that takes an image of shape (H, W, 3) and returns a grayscale image using luminance weights.
15. Write a Python function that converts an image in [0,255] to [0,1], then standardizes each channel using supplied means and standard deviations.
16. Generate a synthetic grayscale image and plot its histogram. Compute the empirical mean and variance from the histogram.
17. Create a small RGB image and intentionally swap the red and blue channels. Describe the visual and mathematical effect.
18. Implement a horizontal flip for an image and for a bounding box label given in normalized coordinates.

3.20.4 AI and modeling exercises

19. Ask an AI assistant to explain the difference between RGB, BGR, and HSV. Then critique its answer mathematically. Did it distinguish linear and nonlinear transformations?
20. Ask an AI assistant to write preprocessing code for a pretrained CNN. Check whether it specifies image size, channel order, intensity scale, and normalization constants.
21. Design a short experiment showing how a classifier can fail when trained on normalized images but tested on unnormalized images.
22. Describe how a segmentation mask should transform under cropping, resizing, and horizontal flipping.

3.21 Notes and References

The material in this chapter connects to the early image formation and digital image processing discussions in Szeliski (2022) and Gonzalez and Woods (2018). The probabilistic interpretation of images as observations of latent quantities aligns with the model-based perspective in Prince (2012). The discussion of standardization and learned preprocessing connects to general deep-learning practice as described in Goodfellow et al. (2016).

Gonzalez, Rafael C., and Richard E. Woods. 2018. Digital Image Processing. 4th ed. Pearson.

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. https://www.deeplearningbook.org/.

Prince, Simon J. D. 2012. Computer Vision: Models, Learning, and Inference. Cambridge University Press. https://www.computervisionmodels.com/.

Szeliski, Richard. 2022. Computer Vision: Algorithms and Applications. 2nd ed. Springer. https://szeliski.org/Book/.