24 Chapter 23. Object Detection

Bounding Boxes, Localization Losses, Anchors, Region Proposals, One-Stage Detectors, and Set Prediction

24.1 23.1 Why object detection is mathematically different from classification

Image classification assigns one or more labels to an entire image. Object detection asks a more structured question: which objects are present, where are they, and how confident are we? The output is not a single vector of class probabilities. It is a finite set of localized objects:

\[ \widehat{\mathcal Y}(X) = \{(\widehat b_j, \widehat c_j, \widehat s_j):j=1,\dots,\widehat m\}, \]

where $X\in\mathbb R^{H\times W\times C}$ is an image, $\widehat b_j$ is a bounding box, $\widehat c_j$ is a class label, $\widehat s_j\in[0,1]$ is a confidence score, and $\widehat m$ is the predicted number of objects. The true annotation has the same set-valued form

\[ \mathcal Y = \{(b_i,c_i):i=1,\dots,m\}. \]

This set-valued output creates several mathematical complications:

Variable cardinality: the number of objects changes from image to image.
Permutation invariance: the order of objects in a set should not matter.
Joint classification and localization: each object requires both a label and geometric coordinates.
Matching ambiguity: a predicted box must be assigned to a ground-truth box before a loss or metric can be computed.
Severe class imbalance: most image locations contain background, not objects.

Central idea

Object detection is a structured prediction problem in which a model must estimate a finite set of labeled geometric regions. The mathematics combines coordinate geometry, statistical decision theory, nonconvex optimization, matching, and evaluation by thresholded overlap.

Historically, object detection has passed through several regimes: hand-designed features and sliding windows, region proposal methods, two-stage convolutional detectors, one-stage dense detectors, anchor-free detectors, and transformer-based set prediction. This chapter emphasizes the common mathematical objects behind these approaches rather than a catalog of software systems.

24.1.1 Learning goals

After this chapter, you should be able to:

represent bounding boxes using several coordinate systems;
compute intersection-over-union and explain why it is not a smooth loss everywhere;
formulate detection as empirical risk minimization with classification and localization losses;
explain anchors, objectness scores, positive/negative assignment, and anchor-free alternatives;
compare two-stage and one-stage detectors mathematically;
derive the role of non-maximum suppression;
explain focal loss as a response to dense class imbalance;
describe average precision and mean average precision;
connect DETR-style detectors to bipartite matching and permutation-invariant set prediction;
design Python experiments that test localization, confidence calibration, and detection metrics.

24.2 23.2 Bounding boxes as geometric objects

A bounding box is an axis-aligned rectangle in image coordinates. The most common parameterizations are:

\[ b=(x_{\min},y_{\min},x_{\max},y_{\max}), \]

\[ b=(x_c,y_c,w,h), \]

where $(x_c,y_c)$ is the center, and $w,h>0$ are width and height. The conversion is

\[ x_c=\frac{x_{\min}+x_{\max}}{2},\qquad y_c=\frac{y_{\min}+y_{\max}}{2}, \]

\[ w=x_{\max}-x_{\min},\qquad h=y_{\max}-y_{\min}. \]

Conversely,

\[ x_{\min}=x_c-\frac{w}{2},\qquad x_{\max}=x_c+\frac{w}{2}, \]

\[ y_{\min}=y_c-\frac{h}{2},\qquad y_{\max}=y_c+\frac{h}{2}. \]

A valid axis-aligned box satisfies

\[ 0\le x_{\min}<x_{\max}\le W, \qquad 0\le y_{\min}<y_{\max}\le H. \]

In normalized coordinates, one divides horizontal coordinates by $W$ and vertical coordinates by $H$ so that all coordinates lie in $[0,1]$. Normalized coordinates are useful because they make the representation less dependent on the pixel resolution.

Coordinate convention

Always state whether a box is represented by corner coordinates or center-width-height coordinates, and whether coordinates are absolute pixels or normalized coordinates. Many detection bugs come from mixing these conventions.

24.2.1 Boxes as sets

A box $b$ can also be viewed as a subset of the image plane:

\[ B(b)=\{(x,y):x_{\min}\le x\le x_{\max},\ y_{\min}\le y\le y_{\max}\}. \]

Its area is

\[ |B(b)|=(x_{\max}-x_{\min})(y_{\max}-y_{\min}). \]

This set viewpoint is essential for defining overlap.

24.3 23.3 Intersection-over-union

Given two boxes $b$ and $b'$, the intersection-over-union is

\[ \operatorname{IoU}(b,b') = \frac{|B(b)\cap B(b')|}{|B(b)\cup B(b')|}. \]

For corner-coordinate boxes, define

\[ x_I^{\min}=\max(x_{\min},x'_{\min}), \qquad x_I^{\max}=\min(x_{\max},x'_{\max}), \]

\[ y_I^{\min}=\max(y_{\min},y'_{\min}), \qquad y_I^{\max}=\min(y_{\max},y'_{\max}). \]

The intersection width and height are

\[ w_I=\max\{0,x_I^{\max}-x_I^{\min}\}, \qquad h_I=\max\{0,y_I^{\max}-y_I^{\min}\}. \]

Therefore

\[ |B(b)\cap B(b')|=w_Ih_I, \]

and

\[ |B(b)\cup B(b')| =|B(b)|+|B(b')|-|B(b)\cap B(b')|. \]

IoU satisfies

\[ 0\le \operatorname{IoU}(b,b')\le 1, \]

with value $1$ if the boxes coincide and value $0$ if the boxes do not overlap.

24.3.1 IoU as a decision rule

A predicted box is often called a true positive if

\[ \operatorname{IoU}(\widehat b,b)\ge \tau \]

and the predicted class is correct. Common thresholds include $\tau=0.5$ and the COCO-style range $0.50,0.55,\dots,0.95$. Higher thresholds demand more precise localization.

24.3.2 Why IoU is not enough as a training loss

The loss

\[ 1-\operatorname{IoU}(\widehat b,b) \]

is intuitively attractive. However, when two boxes do not overlap, IoU is zero and provides no useful gradient for moving one box toward the other. This motivates generalized IoU, distance IoU, complete IoU, and other localization losses that include information beyond raw overlap.

24.4 23.4 Interactive demo: IoU geometry

Move the predicted box and observe how IoU changes. Notice that small coordinate changes near the boundary of overlap can produce abrupt changes in the derivative.

24.5 23.5 Detection as statistical decision theory

Let an image $X$ be drawn from a population distribution. The target is a random finite set

\[ Y=\{(B_i,C_i):i=1,\dots,M\}, \]

where $M$ is random, $B_i$ is a random box, and $C_i$ is a random class. A detector is a measurable map

\[ f_\theta:X\mapsto \widehat Y_\theta(X). \]

A fully general population risk is

\[ R(\theta)=\mathbb E\,L\bigl(f_\theta(X),Y\bigr), \]

where $L$ is a set loss. Because exact set losses are difficult to optimize, practical detectors use surrogate losses based on classification, localization, and assignment.

A common empirical objective is

\[ \widehat R_n(\theta) = \frac{1}{n}\sum_{i=1}^n \left[ L_{\operatorname{cls}}(X_i,Y_i;\theta) +\lambda L_{\operatorname{box}}(X_i,Y_i;\theta) +\gamma L_{\operatorname{obj}}(X_i,Y_i;\theta) \right]. \]

Here:

$L_{\operatorname{cls}}$ penalizes incorrect class predictions;
$L_{\operatorname{box}}$ penalizes poor localization;
$L_{\operatorname{obj}}$ penalizes wrong objectness or foreground/background predictions;
$\lambda,\gamma>0$ balance the terms.

Detection is not only classification plus regression

The model must decide how many objects exist and which prediction corresponds to which object. Matching and duplicate suppression are not minor implementation details; they are part of the mathematical structure of detection.

24.6 23.6 Sliding windows and dense classification

The earliest detection viewpoint is sliding-window classification. Let $\mathcal W(X)$ be a collection of candidate windows extracted from image $X$. A classifier assigns a score to each window:

\[ s(w,c)=g_c(\phi(w)), \]

where $w\in\mathcal W(X)$, $\phi(w)$ is a feature vector, and $g_c$ is a class-specific scoring function.

The detector returns windows with high scores:

\[ \widehat{\mathcal Y}(X)=\{(w,c,s(w,c)):s(w,c)\ge t_c\}. \]

This formulation is conceptually simple but computationally expensive because the number of possible windows is large. A box is described by four continuous coordinates; after discretization over locations, scales, and aspect ratios, the candidate set can still contain thousands or millions of windows.

24.6.1 Mathematical strengths and weaknesses

Sliding windows reduce detection to binary or multiclass classification over candidate regions. This makes the statistical learning problem familiar, but it has two weaknesses:

Computational burden: the candidate space is large.
Localization coarseness: the detector can only return windows from a fixed search grid unless a refinement step is added.

These weaknesses motivate region proposals, anchors, feature pyramids, and learned dense predictors.

24.7 23.7 Region proposals and two-stage detection

A two-stage detector separates detection into two conceptual steps:

propose candidate object regions;
classify and refine each candidate.

The general form is

\[ X \xrightarrow{\text{proposal network}} \{r_k\}_{k=1}^K \xrightarrow{\text{head network}} \{(\widehat b_k,\widehat p_k)\}_{k=1}^K. \]

A region proposal $r_k$ is a candidate box likely to contain an object. For each proposal, the model predicts:

\[ \widehat p_k\in\Delta^{C}, \qquad \widehat t_k\in\mathbb R^4, \]

where $\widehat p_k$ is a class-probability vector and $\widehat t_k$ is a box-regression vector.

24.7.1 Box regression from proposal coordinates

Given a proposal or anchor box $a=(x_a,y_a,w_a,h_a)$ and a target box $b=(x,y,w,h)$, a common target encoding is

\[ t_x=\frac{x-x_a}{w_a}, \qquad t_y=\frac{y-y_a}{h_a}, \]

\[ t_w=\log\frac{w}{w_a}, \qquad t_h=\log\frac{h}{h_a}. \]

A network predicts $\widehat t=(\widehat t_x,\widehat t_y,\widehat t_w,\widehat t_h)$, and the decoded box is

\[ \widehat x=x_a+w_a\widehat t_x, \qquad \widehat y=y_a+h_a\widehat t_y, \]

\[ \widehat w=w_a\exp(\widehat t_w), \qquad \widehat h=h_a\exp(\widehat t_h). \]

This parameterization normalizes translations by anchor size and treats scale multiplicatively through logarithms.

24.8 23.8 Anchors and dense detection

An anchor is a reference box placed at a feature-map location with a specified scale and aspect ratio. Suppose a feature map has spatial indices $(i,j)$. At each location, we may place anchors

\[ a_{ijr},\qquad r=1,\dots,R, \]

where $r$ indexes aspect ratio and scale. A dense detector predicts, for every anchor,

\[ \widehat p_{ijr}\in\Delta^{C+1}, \qquad \widehat t_{ijr}\in\mathbb R^4. \]

The extra class is usually background. During training, each anchor is assigned a label using IoU thresholds. For example,

\[ y_{ijr}=1 \quad\text{if}\quad \max_k \operatorname{IoU}(a_{ijr},b_k)\ge \tau_+, \]

\[ y_{ijr}=0 \quad\text{if}\quad \max_k \operatorname{IoU}(a_{ijr},b_k)\le \tau_-. \]

Anchors with intermediate IoU may be ignored.

24.8.1 Class imbalance

Most anchors are background. If there are $10^5$ anchors and only a small number overlap true objects, a naive cross-entropy loss will be dominated by easy negatives. This is a statistical imbalance problem. It is also an optimization problem because gradients from the majority class can overwhelm useful localization gradients.

24.9 23.9 Interactive demo: anchor assignment

The following demo shows a small grid of anchor centers and two anchor sizes. Anchors with high IoU to the ground-truth box are positive; anchors with low IoU are negative.

24.10 23.10 Localization losses

A detector must learn coordinates. Let $t\in\mathbb R^4$ be the target box-regression vector and $\widehat t\in\mathbb R^4$ the prediction.

24.10.1 Smooth $L_1$ loss

A classical box-regression loss is the smooth $L_1$ loss:

\[ \rho_\beta(u)= \begin{cases} \frac{1}{2\beta}u^2, & |u|<\beta,\\ |u|-\frac{\beta}{2}, & |u|\ge \beta. \end{cases} \]

Then

\[ L_{\operatorname{box}}(\widehat t,t) = \sum_{k=1}^4 \rho_\beta(\widehat t_k-t_k). \]

This is quadratic near zero and linear for large errors. It behaves like least squares for small residuals and like least absolute deviation for large residuals.

24.10.2 IoU-based losses

A direct geometric loss is

\[ L_{\operatorname{IoU}}(\widehat b,b)=1-\operatorname{IoU}(\widehat b,b). \]

Generalized IoU introduces the smallest enclosing box $C$ containing both boxes:

\[ \operatorname{GIoU}(\widehat b,b) = \operatorname{IoU}(\widehat b,b) - \frac{|C\setminus (\widehat B\cup B)|}{|C|}. \]

Then

\[ L_{\operatorname{GIoU}}=1-\operatorname{GIoU}. \]

The second term provides a signal even when boxes do not overlap.

24.11 23.11 Classification losses and focal loss

For an anchor or candidate region, let $p\in(0,1)$ be the predicted probability of the true binary class. Binary cross-entropy is

\[ L_{\operatorname{CE}}(p)=-\log p. \]

For dense detection, many examples are easy negatives. Focal loss modifies cross-entropy as

\[ L_{\operatorname{focal}}(p) = -\alpha(1-p)^\gamma\log p, \]

where $\alpha\in(0,1)$ balances class frequencies and $\gamma\ge 0$ downweights easy examples. When $\gamma=0$, focal loss reduces to weighted cross-entropy.

For a negative example with predicted probability $q$ for the foreground class, one usually writes

\[ p_t= \begin{cases} q, & y=1,\\ 1-q, & y=0, \end{cases} \]

and

\[ L_{\operatorname{focal}}(p_t)=-\alpha_t(1-p_t)^\gamma\log p_t. \]

24.11.1 Interactive demo: focal loss

24.12 23.12 Non-maximum suppression

Dense detectors often predict many overlapping boxes for the same object. Non-maximum suppression is a post-processing algorithm that removes duplicates.

Given predicted boxes with scores

\[ \{(\widehat b_j,\widehat s_j)\}_{j=1}^N, \]

NMS proceeds as follows:

sort boxes by decreasing score;
select the highest-scoring remaining box;
remove boxes whose IoU with the selected box exceeds a threshold $\tau_{\operatorname{nms}}$;
repeat until no boxes remain.

In mathematical terms, NMS greedily constructs a subset $S\subset\{1,\dots,N\}$ such that high-scoring boxes are retained and nearby duplicates are suppressed.

NMS is simple and effective, but it has important limitations:

it is not differentiable;
it depends on a threshold;
it may suppress nearby objects in crowded scenes;
it is a greedy approximation to a more global selection problem.

Soft-NMS reduces scores instead of deleting boxes, replacing hard suppression by a smoother rule such as

\[ \widehat s_j \leftarrow \widehat s_j\exp\left(-\frac{\operatorname{IoU}(\widehat b_i,\widehat b_j)^2}{\sigma}\right). \]

24.13 23.13 Interactive demo: non-maximum suppression

24.14 23.14 Evaluation: precision, recall, AP, and mAP

Detection evaluation is more subtle than classification evaluation because predictions must be matched to ground-truth objects. For a fixed class $c$ and IoU threshold $\tau$, sort predicted boxes by confidence score. A prediction is counted as a true positive if:

its predicted class is $c$;
it has IoU at least $\tau$ with an unmatched ground-truth object of class $c$;
among duplicate predictions for the same object, only the highest-scoring one is counted as true positive.

For a score threshold $t$, define

\[ \operatorname{Precision}(t)= \frac{\operatorname{TP}(t)}{\operatorname{TP}(t)+\operatorname{FP}(t)}, \]

\[ \operatorname{Recall}(t)= \frac{\operatorname{TP}(t)}{\operatorname{TP}(t)+\operatorname{FN}(t)}. \]

As $t$ varies, the detector traces a precision-recall curve. Average precision is the area under this curve:

\[ \operatorname{AP} =\int_0^1 p(r)\,dr, \]

where $p(r)$ is precision as a function of recall after appropriate interpolation. Mean average precision averages AP across classes and sometimes across IoU thresholds:

\[ \operatorname{mAP} =\frac{1}{C}\sum_{c=1}^C \operatorname{AP}_c. \]

For COCO-style evaluation, one averages over multiple IoU thresholds:

\[ \operatorname{mAP}_{.50:.95} =\frac{1}{10C}\sum_{k=0}^9\sum_{c=1}^C \operatorname{AP}_{c,\,0.50+0.05k}. \]

24.14.1 Statistical interpretation

Precision is a conditional probability estimate:

\[ \operatorname{Precision}\approx P(\text{true object}\mid\text{predicted detection}), \]

while recall estimates

\[ \operatorname{Recall}\approx P(\text{detected}\mid\text{true object}). \]

A high-precision detector makes few false alarms; a high-recall detector misses few objects. The appropriate tradeoff depends on the application.

24.15 23.15 Interactive demo: precision-recall and AP

24.16 23.16 One-stage detectors

One-stage detectors predict boxes and classes directly over a dense grid of feature-map locations. Schematically,

\[ X\xrightarrow{\text{backbone}}F \xrightarrow{\text{detection head}} \{(\widehat b_j,\widehat p_j,\widehat s_j)\}_{j=1}^{N}. \]

The output index $j$ may correspond to a grid location, anchor, scale level, or query. One-stage detectors are usually faster than two-stage detectors because they avoid a separate proposal-and-crop step.

24.16.1 YOLO-style grid prediction

A YOLO-style detector partitions the image into grid cells. Each cell predicts a small number of boxes and class scores. The early mathematical idea is that each grid cell is responsible for objects whose centers fall inside that cell. This creates a map

\[ \mathbb R^{H\times W\times C}\to \mathbb R^{S\times S\times B\times(4+1+C)}, \]

where $S\times S$ is the grid, $B$ is the number of boxes per cell, $4$ represents box coordinates, $1$ represents objectness, and $C$ represents class scores.

24.16.2 SSD and multi-scale prediction

Objects appear at different scales. Single Shot MultiBox Detector-style models predict boxes from multiple feature maps:

\[ F_1,F_2,\dots,F_L, \]

where early maps have higher spatial resolution and later maps contain more semantic information. This is an architectural expression of a mathematical fact: detection must be sensitive to both small local details and large semantic context.

24.16.3 Feature pyramid networks

Feature pyramid networks construct multi-scale feature maps

\[ P_2,P_3,\dots,P_L \]

using top-down and lateral connections. Each pyramid level is assigned objects within a range of sizes. FPNs connect detection to scale-space thinking from earlier chapters.

24.17 23.17 Anchor-free detection

Anchor-free detectors avoid predefined anchor boxes. Instead, they often represent objects by centers, keypoints, or dense distances to box sides. For example, at a feature-map location $(i,j)$ inside a ground-truth box, a detector may predict

\[ (l,t,r,b), \]

where

\[ l=x-x_{\min},\qquad t=y-y_{\min}, \qquad r=x_{\max}-x, \qquad b=y_{\max}-y. \]

The decoded box is

\[ (x-l,y-t,x+r,y+b). \]

Anchor-free methods reduce the need to hand-design scales and aspect ratios. They replace anchor assignment by location assignment, center sampling, keypoint matching, or optimal transport style assignment.

24.18 23.18 DETR-style detection as set prediction

Transformer-based detectors such as DETR formulate detection as direct set prediction. The model outputs a fixed number $N$ of object queries:

\[ \{(\widehat b_j,\widehat p_j):j=1,\dots,N\}. \]

Ground truth has $m\le N$ objects. Add $N-m$ no-object labels so both sides have size $N$. Training solves a bipartite matching problem:

\[ \widehat\sigma =\arg\min_{\sigma\in S_N} \sum_{i=1}^N \mathcal C\bigl(y_i,\widehat y_{\sigma(i)}\bigr), \]

where $S_N$ is the permutation group and $\mathcal C$ is a matching cost. A typical matching cost combines classification and localization:

\[ \mathcal C(y_i,\widehat y_j) = -\log \widehat p_j(c_i) +\lambda_1\|b_i-\widehat b_j\|_1 +\lambda_2 L_{\operatorname{GIoU}}(b_i,\widehat b_j). \]

After matching, the training loss is computed using the matched pairs. This removes the need for NMS in the original DETR formulation because each object should be assigned to one query.

Set prediction viewpoint

DETR changes the mathematical structure of detection from dense local prediction plus NMS to global set prediction plus bipartite matching. This is a clean example of how architecture can encode the symmetry of the output space: object sets are permutation invariant.

24.19 23.19 Detection losses as multi-task learning

A detector usually shares a visual backbone across tasks:

\[ F_\theta(X)=\text{shared feature map}. \]

Different heads predict classification, localization, objectness, centerness, masks, or keypoints. The total loss has the form

\[ L(\theta)= \lambda_{\operatorname{cls}}L_{\operatorname{cls}} +\lambda_{\operatorname{box}}L_{\operatorname{box}} +\lambda_{\operatorname{obj}}L_{\operatorname{obj}} +\lambda_{\operatorname{aux}}L_{\operatorname{aux}}. \]

Choosing the weights is not only an engineering issue. It affects the geometry of the optimization problem. If one loss term has much larger gradient norm than the others, it may dominate training.

A useful diagnostic is to monitor gradient norms:

\[ \|\nabla_\theta L_{\operatorname{cls}}\|, \qquad \|\nabla_\theta L_{\operatorname{box}}\|, \qquad \|\nabla_\theta L_{\operatorname{obj}}\|. \]

24.20 23.20 Detection and uncertainty

A detector produces scores, but a score is not automatically a calibrated probability. Detection uncertainty has several sources:

classification uncertainty: which class is present?
localization uncertainty: where exactly is the object boundary?
objectness uncertainty: is this region foreground or background?
assignment uncertainty: which prediction corresponds to which object?
dataset uncertainty: are annotations complete and consistent?

For localization, one can model a box as a random vector

\[ B\in\mathbb R^4, \]

with predicted mean $\mu_\theta(X)$ and covariance $\Sigma_\theta(X)$. A Gaussian localization likelihood would be

\[ p(b\mid X;\theta) =\mathcal N\bigl(b;\mu_\theta(X),\Sigma_\theta(X)\bigr). \]

The negative log-likelihood is

\[ \frac{1}{2}(b-\mu)^T\Sigma^{-1}(b-\mu) +\frac{1}{2}\log\det\Sigma +\text{constant}. \]

This formulation rewards accurate predictions but penalizes both overconfidence and underconfidence.

24.21 23.21 Python example: computing IoU and NMS

import numpy as np


def box_iou(boxes1, boxes2):
    """Compute pairwise IoU for boxes in xyxy format."""
    boxes1 = np.asarray(boxes1, dtype=float)
    boxes2 = np.asarray(boxes2, dtype=float)

    x11, y11, x12, y12 = boxes1[:, 0], boxes1[:, 1], boxes1[:, 2], boxes1[:, 3]
    x21, y21, x22, y22 = boxes2[:, 0], boxes2[:, 1], boxes2[:, 2], boxes2[:, 3]

    xa = np.maximum(x11[:, None], x21[None, :])
    ya = np.maximum(y11[:, None], y21[None, :])
    xb = np.minimum(x12[:, None], x22[None, :])
    yb = np.minimum(y12[:, None], y22[None, :])

    inter = np.maximum(0, xb - xa) * np.maximum(0, yb - ya)
    area1 = np.maximum(0, x12 - x11) * np.maximum(0, y12 - y11)
    area2 = np.maximum(0, x22 - x21) * np.maximum(0, y22 - y21)
    union = area1[:, None] + area2[None, :] - inter
    return inter / np.maximum(union, 1e-12)


def nms(boxes, scores, iou_threshold=0.5):
    boxes = np.asarray(boxes, dtype=float)
    scores = np.asarray(scores, dtype=float)
    order = scores.argsort()[::-1]
    keep = []

    while order.size > 0:
        i = order[0]
        keep.append(i)
        if order.size == 1:
            break
        rest = order[1:]
        overlaps = box_iou(boxes[[i]], boxes[rest]).ravel()
        order = rest[overlaps < iou_threshold]
    return keep

boxes = np.array([
    [20, 20, 80, 80],
    [25, 25, 82, 82],
    [150, 40, 220, 120],
    [155, 45, 218, 125],
])
scores = np.array([0.95, 0.80, 0.90, 0.60])

print(box_iou(boxes, boxes))
print("kept indices:", nms(boxes, scores, iou_threshold=0.5))

24.22 23.22 Python example: average precision from detections

import numpy as np


def average_precision_from_sorted_labels(tp_flags, num_ground_truth):
    """Compute AP from detections sorted by descending confidence.

    tp_flags[j] is 1 if detection j is a true positive and 0 otherwise.
    """
    tp_flags = np.asarray(tp_flags, dtype=float)
    fp_flags = 1.0 - tp_flags

    tp_cum = np.cumsum(tp_flags)
    fp_cum = np.cumsum(fp_flags)

    recall = tp_cum / max(num_ground_truth, 1)
    precision = tp_cum / np.maximum(tp_cum + fp_cum, 1e-12)

    # Monotone precision envelope.
    mrec = np.r_[0.0, recall, 1.0]
    mpre = np.r_[0.0, precision, 0.0]
    for i in range(len(mpre) - 2, -1, -1):
        mpre[i] = max(mpre[i], mpre[i + 1])

    change = np.where(mrec[1:] != mrec[:-1])[0]
    ap = np.sum((mrec[change + 1] - mrec[change]) * mpre[change + 1])
    return ap, recall, precision

# Example: detections sorted from most confident to least confident.
tp_flags = [1, 0, 1, 1, 0, 0, 1]
num_ground_truth = 5
ap, recall, precision = average_precision_from_sorted_labels(tp_flags, num_ground_truth)
print("AP:", ap)
print("recall:", recall)
print("precision:", precision)

24.23 23.23 Python example: anchor target assignment

import numpy as np


def make_grid_anchors(H=4, W=4, stride=32, scales=(32, 64), ratios=(0.5, 1.0, 2.0)):
    anchors = []
    for i in range(H):
        for j in range(W):
            cx = (j + 0.5) * stride
            cy = (i + 0.5) * stride
            for s in scales:
                for r in ratios:
                    w = s * np.sqrt(r)
                    h = s / np.sqrt(r)
                    anchors.append([cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2])
    return np.array(anchors)

anchors = make_grid_anchors()
gt_boxes = np.array([[35, 40, 95, 110], [100, 20, 150, 70]])
ious = box_iou(anchors, gt_boxes)
best_iou = ious.max(axis=1)

positive = best_iou >= 0.5
negative = best_iou < 0.3
ignore = ~(positive | negative)

print("number of anchors:", len(anchors))
print("positive:", positive.sum())
print("negative:", negative.sum())
print("ignore:", ignore.sum())

24.24 23.24 TensorFlow example: tiny dense detection head

This example is not meant to be a production detector. It shows the shape logic of a dense detection head.

import tensorflow as tf

num_classes = 5
num_anchors = 3

inputs = tf.keras.Input(shape=(224, 224, 3))
x = tf.keras.layers.Rescaling(1.0 / 255)(inputs)

# Simple backbone.
x = tf.keras.layers.Conv2D(32, 3, strides=2, padding="same", activation="relu")(x)
x = tf.keras.layers.Conv2D(64, 3, strides=2, padding="same", activation="relu")(x)
x = tf.keras.layers.Conv2D(128, 3, strides=2, padding="same", activation="relu")(x)
x = tf.keras.layers.Conv2D(128, 3, padding="same", activation="relu")(x)

# Shared detection features.
f = tf.keras.layers.Conv2D(128, 3, padding="same", activation="relu")(x)

# Classification: background plus object classes.
cls_logits = tf.keras.layers.Conv2D(
    num_anchors * (num_classes + 1),
    kernel_size=3,
    padding="same",
    name="class_logits"
)(f)

# Box regression: four offsets per anchor.
box_reg = tf.keras.layers.Conv2D(
    num_anchors * 4,
    kernel_size=3,
    padding="same",
    name="box_regression"
)(f)

model = tf.keras.Model(inputs=inputs, outputs=[cls_logits, box_reg])
model.summary()

# For a 224 x 224 input and stride 8, the feature map is 28 x 28.
# The classification output shape is (batch, 28, 28, A(C+1)).
# The box output shape is (batch, 28, 28, 4A).

24.25 23.25 AI component: using large models to study object detection

AI tools can help students learn object detection, but they should be used in a mathematically disciplined way. Good prompts ask the model to produce equations, assumptions, failure cases, and verification tests.

24.25.1 Prompt 1: verify coordinate conventions

I have boxes stored as $(x_c,y_c,w,h)$ in normalized coordinates. Derive the conversion to $(x_{\min},y_{\min},x_{\max},y_{\max})$. Then list three bugs that can occur when mixing normalized coordinates and pixel coordinates.

24.25.2 Prompt 2: compare localization losses

Explain the difference among smooth $L_1$, IoU loss, generalized IoU loss, and Gaussian negative log-likelihood for bounding-box regression. For each loss, state what mathematical quantity it penalizes and one failure mode.

24.25.3 Prompt 3: design an AP computation test

Create a tiny detection example with three ground-truth boxes and six predicted boxes. Sort predictions by score and compute precision, recall, and AP by hand. Include duplicate detections and one localization false positive.

24.25.4 Prompt 4: critique a detector

A detector has high AP at IoU $0.50$ but much lower AP at IoU $0.75$. Give a mathematical diagnosis of this behavior and propose experiments to decide whether the problem is classification, localization, scale, annotation noise, or NMS.

24.25.5 Prompt 5: connect DETR to matching

Explain why object detection can be formulated as a bipartite matching problem. Write the matching objective and explain why this makes the loss permutation invariant.

24.26 23.26 Common failure modes and mathematical diagnostics

Failure mode	Mathematical symptom	Possible diagnostic
duplicate detections	many high-IoU predictions for one object	inspect NMS threshold and score calibration
missed small objects	low recall for small area bins	evaluate AP by object size
poor localization	AP drops sharply as IoU threshold increases	compare AP50 and AP75
too many false positives	low precision at high scores	reliability and threshold analysis
class confusion	off-diagonal confusion matrix	inspect class-conditional embeddings
crowded-scene suppression	nearby objects removed by NMS	compare NMS and Soft-NMS
anchor mismatch	few positive anchors	plot anchor-size/aspect-ratio coverage
overconfident boxes	high scores for inaccurate boxes	calibrate scores and analyze localization uncertainty

24.27 23.27 Mini-project: building a detection metric from scratch

A useful independent-study project is to implement the evaluation pipeline without using a detection library.

Write a function for pairwise IoU.
For one class, sort predicted boxes by confidence.
Match each prediction to the best unmatched ground-truth box.
Mark true positives and false positives using an IoU threshold.
Compute cumulative precision and recall.
Compute AP.
Repeat for multiple IoU thresholds.
Compare how AP changes when localization noise increases.

The key lesson is that detection metrics are not black boxes. They are explicit functions of scores, boxes, classes, matching rules, and IoU thresholds.

24.28 23.28 Exercises

24.28.1 Conceptual exercises

Explain why object detection output is a set, not a vector with fixed ordering.
Give two reasons why detection is harder than image classification.
Explain the difference between objectness and class probability.
Explain why duplicate detections are a problem even if all boxes overlap a true object.
Describe a case where a detector has high recall but poor precision.
Describe a case where a detector has good classification but poor localization.

24.28.2 Mathematical exercises

Convert a box from $(x_{\min},y_{\min},x_{\max},y_{\max})$ to $(x_c,y_c,w,h)$ and back. State the validity constraints.
Derive the IoU formula for two axis-aligned rectangles.
Show that IoU lies in $[0,1]$.
Suppose two boxes have the same center and aspect ratio, but one has width and height multiplied by $a>0$. Derive IoU as a function of $a$.
Derive the inverse transformation from anchor offsets $(t_x,t_y,t_w,t_h)$ to box coordinates.
For smooth $L_1$, compute the derivative of $\rho_\beta(u)$ and show that it is continuous at $|u|=\beta$.
Show that focal loss reduces to weighted cross-entropy when $\gamma=0$.
Given a sequence of true-positive and false-positive flags sorted by confidence, compute precision, recall, and AP.
Write the bipartite matching objective for a detector with three predictions and two true objects plus one no-object label.
Explain why NMS is not permutation invariant with respect to scores but is permutation invariant with respect to the input ordering after sorting.

24.28.3 Python exercises

Implement pairwise IoU for a batch of boxes.
Implement NMS and test it on overlapping boxes.
Implement Soft-NMS and compare it with hard NMS.
Generate random ground-truth boxes and noisy predicted boxes. Plot AP as a function of localization noise.
Implement anchor generation for three feature-map levels.
Compute the number of positive, negative, and ignored anchors for a synthetic dataset.
Implement smooth $L_1$, IoU loss, and GIoU loss.
Implement focal loss and plot it for different values of $\gamma$.
Build a tiny TensorFlow detection head and verify output shapes.
Write a script that checks whether all predicted boxes satisfy coordinate validity constraints.

24.28.4 AI-assisted exercises

Ask an AI tool to derive IoU for two boxes, then verify every step by writing a small numerical test.
Ask for a minimal object-detection evaluation example and check whether duplicate detections are handled correctly.
Ask for an explanation of focal loss, then rewrite the explanation using only probability and calculus.
Ask the tool to compare Faster R-CNN, RetinaNet, YOLO, and DETR using the categories: proposal mechanism, assignment rule, loss, post-processing, and output symmetry.
Ask the tool to generate likely detector failure cases for medical imaging, autonomous driving, and satellite imagery. For each case, identify the statistical issue.

24.29 23.29 Summary

Object detection extends classification by adding geometry, cardinality, matching, and duplicate suppression. The core mathematical objects are boxes, scores, labels, IoU, assignment rules, and set-valued predictions. Classical detectors use sliding windows or region proposals. Dense detectors use anchors or anchor-free location assignment. Transformer-based detectors formulate detection as direct set prediction with bipartite matching.

For applied mathematics and statistics students, the important lesson is that detection is not simply a neural network architecture. It is a structured inference problem whose behavior depends on geometry, optimization, class imbalance, calibration, and the precise definition of evaluation metrics.

24.30 23.30 Notes and references

Object detection has a long history, including sliding-window methods, boosted cascade detectors, deformable part models, region-based convolutional detectors, one-stage dense detectors, and transformer-based set predictors. Important references include Viola–Jones detection (Viola and Jones 2001), deformable part models (Felzenszwalb et al. 2010), R-CNN (Girshick et al. 2014), Fast R-CNN (Girshick 2015), Faster R-CNN (Ren et al. 2015), SSD (Liu et al. 2016), YOLO (Redmon et al. 2016), feature pyramid networks (Lin, Dollár, et al. 2017), RetinaNet and focal loss (Lin, Goyal, et al. 2017), Mask R-CNN (He et al. 2017), COCO evaluation (Lin et al. 2014), Soft-NMS (Bodla et al. 2017), generalized IoU (Rezatofighi et al. 2019), and DETR (Carion et al. 2020).

Bodla, Navaneeth, Bharat Singh, Rama Chellappa, and Larry S. Davis. 2017. “Soft-NMS – Improving Object Detection with One Line of Code.” IEEE International Conference on Computer Vision, 5561–69. https://arxiv.org/abs/1704.04503.

Carion, Nicolas, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020. “End-to-End Object Detection with Transformers.” European Conference on Computer Vision, 213–29. https://arxiv.org/abs/2005.12872.

Felzenszwalb, Pedro F., Ross B. Girshick, David McAllester, and Deva Ramanan. 2010. “Object Detection with Discriminatively Trained Part-Based Models.” IEEE Transactions on Pattern Analysis and Machine Intelligence 32 (9): 1627–45.

Girshick, Ross. 2015. “Fast r-CNN.” IEEE International Conference on Computer Vision, 1440–48. https://arxiv.org/abs/1504.08083.

Girshick, Ross, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation.” IEEE Conference on Computer Vision and Pattern Recognition, 580–87. https://arxiv.org/abs/1311.2524.

He, Kaiming, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. “Mask r-CNN.” IEEE International Conference on Computer Vision, 2961–69. https://arxiv.org/abs/1703.06870.

Lin, Tsung-Yi, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. “Feature Pyramid Networks for Object Detection.” IEEE Conference on Computer Vision and Pattern Recognition, 2117–25. https://arxiv.org/abs/1612.03144.

Lin, Tsung-Yi, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. “Focal Loss for Dense Object Detection.” IEEE International Conference on Computer Vision, 2980–88. https://arxiv.org/abs/1708.02002.

Lin, Tsung-Yi, Michael Maire, Serge Belongie, et al. 2014. “Microsoft COCO: Common Objects in Context.” European Conference on Computer Vision, 740–55. https://arxiv.org/abs/1405.0312.

Liu, Wei, Dragomir Anguelov, Dumitru Erhan, et al. 2016. “SSD: Single Shot MultiBox Detector.” European Conference on Computer Vision, 21–37. https://arxiv.org/abs/1512.02325.

Redmon, Joseph, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. “You Only Look Once: Unified, Real-Time Object Detection.” IEEE Conference on Computer Vision and Pattern Recognition, 779–88. https://arxiv.org/abs/1506.02640.

Ren, Shaoqing, Kaiming He, Ross Girshick, and Jian Sun. 2015. “Faster r-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” Advances in Neural Information Processing Systems 28. https://arxiv.org/abs/1506.01497.

Rezatofighi, Hamid, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. 2019. “Generalized Intersection over Union: A Metric and a Loss for Bounding Box Regression.” IEEE/CVF Conference on Computer Vision and Pattern Recognition, 658–66. https://arxiv.org/abs/1902.09630.

Viola, Paul, and Michael Jones. 2001. “Rapid Object Detection Using a Boosted Cascade of Simple Features.” IEEE Conference on Computer Vision and Pattern Recognition 1: I-511-I-518.

--- title: "Chapter 23. Object Detection" subtitle: "Bounding Boxes, Localization Losses, Anchors, Region Proposals, One-Stage Detectors, and Set Prediction" format: html: toc: true toc-depth: 3 execute: warning: false message: false --- ## 23.1 Why object detection is mathematically different from classification Image classification assigns one or more labels to an entire image. Object detection asks a more structured question: **which objects are present, where are they, and how confident are we?** The output is not a single vector of class probabilities. It is a finite set of localized objects: $$ \widehat{\mathcal Y}(X) = \{(\widehat b_j, \widehat c_j, \widehat s_j):j=1,\dots,\widehat m\}, $$ where $X\in\mathbb R^{H\times W\times C}$ is an image, $\widehat b_j$ is a bounding box, $\widehat c_j$ is a class label, $\widehat s_j\in[0,1]$ is a confidence score, and $\widehat m$ is the predicted number of objects. The true annotation has the same set-valued form $$ \mathcal Y = \{(b_i,c_i):i=1,\dots,m\}. $$ This set-valued output creates several mathematical complications: 1. **Variable cardinality:** the number of objects changes from image to image. 2. **Permutation invariance:** the order of objects in a set should not matter. 3. **Joint classification and localization:** each object requires both a label and geometric coordinates. 4. **Matching ambiguity:** a predicted box must be assigned to a ground-truth box before a loss or metric can be computed. 5. **Severe class imbalance:** most image locations contain background, not objects. ::: {.callout-note} ## Central idea Object detection is a structured prediction problem in which a model must estimate a finite set of labeled geometric regions. The mathematics combines coordinate geometry, statistical decision theory, nonconvex optimization, matching, and evaluation by thresholded overlap. ::: Historically, object detection has passed through several regimes: hand-designed features and sliding windows, region proposal methods, two-stage convolutional detectors, one-stage dense detectors, anchor-free detectors, and transformer-based set prediction. This chapter emphasizes the common mathematical objects behind these approaches rather than a catalog of software systems. ### Learning goals After this chapter, you should be able to: 1. represent bounding boxes using several coordinate systems; 2. compute intersection-over-union and explain why it is not a smooth loss everywhere; 3. formulate detection as empirical risk minimization with classification and localization losses; 4. explain anchors, objectness scores, positive/negative assignment, and anchor-free alternatives; 5. compare two-stage and one-stage detectors mathematically; 6. derive the role of non-maximum suppression; 7. explain focal loss as a response to dense class imbalance; 8. describe average precision and mean average precision; 9. connect DETR-style detectors to bipartite matching and permutation-invariant set prediction; 10. design Python experiments that test localization, confidence calibration, and detection metrics. ## 23.2 Bounding boxes as geometric objects A bounding box is an axis-aligned rectangle in image coordinates. The most common parameterizations are: $$ b=(x_{\min},y_{\min},x_{\max},y_{\max}), $$ or $$ b=(x_c,y_c,w,h), $$ where $(x_c,y_c)$ is the center, and $w,h>0$ are width and height. The conversion is $$ x_c=\frac{x_{\min}+x_{\max}}{2},\qquad y_c=\frac{y_{\min}+y_{\max}}{2}, $$ $$ w=x_{\max}-x_{\min},\qquad h=y_{\max}-y_{\min}. $$ Conversely, $$ x_{\min}=x_c-\frac{w}{2},\qquad x_{\max}=x_c+\frac{w}{2}, $$ $$ y_{\min}=y_c-\frac{h}{2},\qquad y_{\max}=y_c+\frac{h}{2}. $$ A valid axis-aligned box satisfies $$ 0\le x_{\min}<x_{\max}\le W, \qquad 0\le y_{\min}<y_{\max}\le H. $$ In normalized coordinates, one divides horizontal coordinates by $W$ and vertical coordinates by $H$ so that all coordinates lie in $[0,1]$. Normalized coordinates are useful because they make the representation less dependent on the pixel resolution. ::: {.callout-tip} ## Coordinate convention Always state whether a box is represented by corner coordinates or center-width-height coordinates, and whether coordinates are absolute pixels or normalized coordinates. Many detection bugs come from mixing these conventions. ::: ### Boxes as sets A box $b$ can also be viewed as a subset of the image plane: $$ B(b)=\{(x,y):x_{\min}\le x\le x_{\max},\ y_{\min}\le y\le y_{\max}\}. $$ Its area is $$ |B(b)|=(x_{\max}-x_{\min})(y_{\max}-y_{\min}). $$ This set viewpoint is essential for defining overlap. ## 23.3 Intersection-over-union Given two boxes $b$ and $b'$, the intersection-over-union is $$ \operatorname{IoU}(b,b') = \frac{|B(b)\cap B(b')|}{|B(b)\cup B(b')|}. $$ For corner-coordinate boxes, define $$ x_I^{\min}=\max(x_{\min},x'_{\min}), \qquad x_I^{\max}=\min(x_{\max},x'_{\max}), $$ $$ y_I^{\min}=\max(y_{\min},y'_{\min}), \qquad y_I^{\max}=\min(y_{\max},y'_{\max}). $$ The intersection width and height are $$ w_I=\max\{0,x_I^{\max}-x_I^{\min}\}, \qquad h_I=\max\{0,y_I^{\max}-y_I^{\min}\}. $$ Therefore $$ |B(b)\cap B(b')|=w_Ih_I, $$ and $$ |B(b)\cup B(b')| =|B(b)|+|B(b')|-|B(b)\cap B(b')|. $$ IoU satisfies $$ 0\le \operatorname{IoU}(b,b')\le 1, $$ with value $1$ if the boxes coincide and value $0$ if the boxes do not overlap. ### IoU as a decision rule A predicted box is often called a true positive if $$ \operatorname{IoU}(\widehat b,b)\ge \tau $$ and the predicted class is correct. Common thresholds include $\tau=0.5$ and the COCO-style range $0.50,0.55,\dots,0.95$. Higher thresholds demand more precise localization. ### Why IoU is not enough as a training loss The loss $$ 1-\operatorname{IoU}(\widehat b,b) $$ is intuitively attractive. However, when two boxes do not overlap, IoU is zero and provides no useful gradient for moving one box toward the other. This motivates generalized IoU, distance IoU, complete IoU, and other localization losses that include information beyond raw overlap. ## 23.4 Interactive demo: IoU geometry Move the predicted box and observe how IoU changes. Notice that small coordinate changes near the boundary of overlap can produce abrupt changes in the derivative. ::: {.interactive-card} <div id="ch23-iou-demo" class="plotly-demo"></div> ::: <script> (function(){ function rectTrace(x0,y0,x1,y1,name){ return { x:[x0,x1,x1,x0,x0], y:[y0,y0,y1,y1,y0], mode:'lines', name:name, line:{width:4} }; } function iou(a,b){ const xi0=Math.max(a[0],b[0]), yi0=Math.max(a[1],b[1]); const xi1=Math.min(a[2],b[2]), yi1=Math.min(a[3],b[3]); const iw=Math.max(0,xi1-xi0), ih=Math.max(0,yi1-yi0); const inter=iw*ih; const areaA=(a[2]-a[0])*(a[3]-a[1]); const areaB=(b[2]-b[0])*(b[3]-b[1]); return inter/(areaA+areaB-inter+1e-12); } const gt=[2,2,6,5]; const pred=[4,3,8,6]; const val=iou(gt,pred); const data=[rectTrace(...gt,'ground truth'),rectTrace(...pred,'prediction')]; const layout={ title:'Intersection-over-union: IoU = '+val.toFixed(3), xaxis:{range:[0,10],scaleanchor:'y',title:'x'}, yaxis:{range:[0,8],title:'y'}, margin:{t:55,l:45,r:20,b:45} }; if(window.Plotly){Plotly.newPlot('ch23-iou-demo',data,layout,{responsive:true});} })(); </script> ## 23.5 Detection as statistical decision theory Let an image $X$ be drawn from a population distribution. The target is a random finite set $$ Y=\{(B_i,C_i):i=1,\dots,M\}, $$ where $M$ is random, $B_i$ is a random box, and $C_i$ is a random class. A detector is a measurable map $$ f_\theta:X\mapsto \widehat Y_\theta(X). $$ A fully general population risk is $$ R(\theta)=\mathbb E\,L\bigl(f_\theta(X),Y\bigr), $$ where $L$ is a set loss. Because exact set losses are difficult to optimize, practical detectors use surrogate losses based on classification, localization, and assignment. A common empirical objective is $$ \widehat R_n(\theta) = \frac{1}{n}\sum_{i=1}^n \left[ L_{\operatorname{cls}}(X_i,Y_i;\theta) +\lambda L_{\operatorname{box}}(X_i,Y_i;\theta) +\gamma L_{\operatorname{obj}}(X_i,Y_i;\theta) \right]. $$ Here: - $L_{\operatorname{cls}}$ penalizes incorrect class predictions; - $L_{\operatorname{box}}$ penalizes poor localization; - $L_{\operatorname{obj}}$ penalizes wrong objectness or foreground/background predictions; - $\lambda,\gamma>0$ balance the terms. ::: {.callout-warning} ## Detection is not only classification plus regression The model must decide **how many** objects exist and **which** prediction corresponds to which object. Matching and duplicate suppression are not minor implementation details; they are part of the mathematical structure of detection. ::: ## 23.6 Sliding windows and dense classification The earliest detection viewpoint is sliding-window classification. Let $\mathcal W(X)$ be a collection of candidate windows extracted from image $X$. A classifier assigns a score to each window: $$ s(w,c)=g_c(\phi(w)), $$ where $w\in\mathcal W(X)$, $\phi(w)$ is a feature vector, and $g_c$ is a class-specific scoring function. The detector returns windows with high scores: $$ \widehat{\mathcal Y}(X)=\{(w,c,s(w,c)):s(w,c)\ge t_c\}. $$ This formulation is conceptually simple but computationally expensive because the number of possible windows is large. A box is described by four continuous coordinates; after discretization over locations, scales, and aspect ratios, the candidate set can still contain thousands or millions of windows. ### Mathematical strengths and weaknesses Sliding windows reduce detection to binary or multiclass classification over candidate regions. This makes the statistical learning problem familiar, but it has two weaknesses: 1. **Computational burden:** the candidate space is large. 2. **Localization coarseness:** the detector can only return windows from a fixed search grid unless a refinement step is added. These weaknesses motivate region proposals, anchors, feature pyramids, and learned dense predictors. ## 23.7 Region proposals and two-stage detection A two-stage detector separates detection into two conceptual steps: 1. propose candidate object regions; 2. classify and refine each candidate. The general form is $$ X \xrightarrow{\text{proposal network}} \{r_k\}_{k=1}^K \xrightarrow{\text{head network}} \{(\widehat b_k,\widehat p_k)\}_{k=1}^K. $$ A region proposal $r_k$ is a candidate box likely to contain an object. For each proposal, the model predicts: $$ \widehat p_k\in\Delta^{C}, \qquad \widehat t_k\in\mathbb R^4, $$ where $\widehat p_k$ is a class-probability vector and $\widehat t_k$ is a box-regression vector. ### Box regression from proposal coordinates Given a proposal or anchor box $a=(x_a,y_a,w_a,h_a)$ and a target box $b=(x,y,w,h)$, a common target encoding is $$ t_x=\frac{x-x_a}{w_a}, \qquad t_y=\frac{y-y_a}{h_a}, $$ $$ t_w=\log\frac{w}{w_a}, \qquad t_h=\log\frac{h}{h_a}. $$ A network predicts $\widehat t=(\widehat t_x,\widehat t_y,\widehat t_w,\widehat t_h)$, and the decoded box is $$ \widehat x=x_a+w_a\widehat t_x, \qquad \widehat y=y_a+h_a\widehat t_y, $$ $$ \widehat w=w_a\exp(\widehat t_w), \qquad \widehat h=h_a\exp(\widehat t_h). $$ This parameterization normalizes translations by anchor size and treats scale multiplicatively through logarithms. ## 23.8 Anchors and dense detection An anchor is a reference box placed at a feature-map location with a specified scale and aspect ratio. Suppose a feature map has spatial indices $(i,j)$. At each location, we may place anchors $$ a_{ijr},\qquad r=1,\dots,R, $$ where $r$ indexes aspect ratio and scale. A dense detector predicts, for every anchor, $$ \widehat p_{ijr}\in\Delta^{C+1}, \qquad \widehat t_{ijr}\in\mathbb R^4. $$ The extra class is usually background. During training, each anchor is assigned a label using IoU thresholds. For example, $$ y_{ijr}=1 \quad\text{if}\quad \max_k \operatorname{IoU}(a_{ijr},b_k)\ge \tau_+, $$ $$ y_{ijr}=0 \quad\text{if}\quad \max_k \operatorname{IoU}(a_{ijr},b_k)\le \tau_-. $$ Anchors with intermediate IoU may be ignored. ### Class imbalance Most anchors are background. If there are $10^5$ anchors and only a small number overlap true objects, a naive cross-entropy loss will be dominated by easy negatives. This is a statistical imbalance problem. It is also an optimization problem because gradients from the majority class can overwhelm useful localization gradients. ## 23.9 Interactive demo: anchor assignment The following demo shows a small grid of anchor centers and two anchor sizes. Anchors with high IoU to the ground-truth box are positive; anchors with low IoU are negative. ::: {.interactive-card} <div id="ch23-anchor-demo" class="plotly-demo"></div> ::: <script> (function(){ const gt=[3.2,2.4,6.4,5.4]; function iou(a,b){ const xi0=Math.max(a[0],b[0]), yi0=Math.max(a[1],b[1]); const xi1=Math.min(a[2],b[2]), yi1=Math.min(a[3],b[3]); const iw=Math.max(0,xi1-xi0), ih=Math.max(0,yi1-yi0); const inter=iw*ih; const areaA=(a[2]-a[0])*(a[3]-a[1]); const areaB=(b[2]-b[0])*(b[3]-b[1]); return inter/(areaA+areaB-inter+1e-12); } function rect(a,name,width){return {x:[a[0],a[2],a[2],a[0],a[0]],y:[a[1],a[1],a[3],a[3],a[1]],mode:'lines',name:name,line:{width:width||1},showlegend:false};} const data=[rect(gt,'gt',5)]; const pos=[], mid=[], neg=[]; for(let x=1;x<=9;x+=1.5){ for(let y=1;y<=7;y+=1.5){ [[2.5,2.0],[3.5,3.0]].forEach(sz=>{ const a=[x-sz[0]/2,y-sz[1]/2,x+sz[0]/2,y+sz[1]/2]; const v=iou(a,gt); const tr=rect(a,'anchor',1); if(v>=0.45){pos.push(tr);} else if(v<=0.15){neg.push(tr);} else {mid.push(tr);} }); } } data.push(...neg.slice(0,25),...mid,...pos); data.push({x:[5],y:[7.6],mode:'text',text:['positive anchors have large IoU with the object'],showlegend:false}); const layout={ title:'Anchor assignment by IoU thresholds', xaxis:{range:[0,10],scaleanchor:'y',title:'x'}, yaxis:{range:[0,8],title:'y'}, margin:{t:55,l:45,r:20,b:45} }; if(window.Plotly){Plotly.newPlot('ch23-anchor-demo',data,layout,{responsive:true});} })(); </script> ## 23.10 Localization losses A detector must learn coordinates. Let $t\in\mathbb R^4$ be the target box-regression vector and $\widehat t\in\mathbb R^4$ the prediction. ### Smooth $L_1$ loss A classical box-regression loss is the smooth $L_1$ loss: $$ \rho_\beta(u)= \begin{cases} \frac{1}{2\beta}u^2, & |u|<\beta,\\ |u|-\frac{\beta}{2}, & |u|\ge \beta. \end{cases} $$ Then $$ L_{\operatorname{box}}(\widehat t,t) = \sum_{k=1}^4 \rho_\beta(\widehat t_k-t_k). $$ This is quadratic near zero and linear for large errors. It behaves like least squares for small residuals and like least absolute deviation for large residuals. ### IoU-based losses A direct geometric loss is $$ L_{\operatorname{IoU}}(\widehat b,b)=1-\operatorname{IoU}(\widehat b,b). $$ Generalized IoU introduces the smallest enclosing box $C$ containing both boxes: $$ \operatorname{GIoU}(\widehat b,b) = \operatorname{IoU}(\widehat b,b) - \frac{|C\setminus (\widehat B\cup B)|}{|C|}. $$ Then $$ L_{\operatorname{GIoU}}=1-\operatorname{GIoU}. $$ The second term provides a signal even when boxes do not overlap. ## 23.11 Classification losses and focal loss For an anchor or candidate region, let $p\in(0,1)$ be the predicted probability of the true binary class. Binary cross-entropy is $$ L_{\operatorname{CE}}(p)=-\log p. $$ For dense detection, many examples are easy negatives. Focal loss modifies cross-entropy as $$ L_{\operatorname{focal}}(p) = -\alpha(1-p)^\gamma\log p, $$ where $\alpha\in(0,1)$ balances class frequencies and $\gamma\ge 0$ downweights easy examples. When $\gamma=0$, focal loss reduces to weighted cross-entropy. For a negative example with predicted probability $q$ for the foreground class, one usually writes $$ p_t= \begin{cases} q, & y=1,\\ 1-q, & y=0, \end{cases} $$ and $$ L_{\operatorname{focal}}(p_t)=-\alpha_t(1-p_t)^\gamma\log p_t. $$ ### Interactive demo: focal loss ::: {.interactive-card} <div id="ch23-focal-loss-demo" class="plotly-demo"></div> ::: <script> (function(){ const ps=[]; for(let i=1;i<100;i++){ps.push(i/100);} function fl(p,g){return -Math.pow(1-p,g)*Math.log(p);} const data=[0,1,2,4].map(g=>({x:ps,y:ps.map(p=>fl(p,g)),mode:'lines',name:'gamma = '+g})); const layout={ title:'Focal loss downweights easy examples', xaxis:{title:'p_t: probability assigned to the correct class'}, yaxis:{title:'loss'}, margin:{t:55,l:60,r:20,b:50} }; if(window.Plotly){Plotly.newPlot('ch23-focal-loss-demo',data,layout,{responsive:true});} })(); </script> ## 23.12 Non-maximum suppression Dense detectors often predict many overlapping boxes for the same object. Non-maximum suppression is a post-processing algorithm that removes duplicates. Given predicted boxes with scores $$ \{(\widehat b_j,\widehat s_j)\}_{j=1}^N, $$ NMS proceeds as follows: 1. sort boxes by decreasing score; 2. select the highest-scoring remaining box; 3. remove boxes whose IoU with the selected box exceeds a threshold $\tau_{\operatorname{nms}}$; 4. repeat until no boxes remain. In mathematical terms, NMS greedily constructs a subset $S\subset\{1,\dots,N\}$ such that high-scoring boxes are retained and nearby duplicates are suppressed. NMS is simple and effective, but it has important limitations: - it is not differentiable; - it depends on a threshold; - it may suppress nearby objects in crowded scenes; - it is a greedy approximation to a more global selection problem. Soft-NMS reduces scores instead of deleting boxes, replacing hard suppression by a smoother rule such as $$ \widehat s_j \leftarrow \widehat s_j\exp\left(-\frac{\operatorname{IoU}(\widehat b_i,\widehat b_j)^2}{\sigma}\right). $$ ## 23.13 Interactive demo: non-maximum suppression ::: {.interactive-card} <div id="ch23-nms-demo" class="plotly-demo"></div> ::: <script> (function(){ const boxes=[ [1.8,1.6,5.1,4.6,0.93], [2.1,1.9,5.4,4.8,0.81], [2.4,2.0,5.2,4.9,0.73], [6.0,2.2,8.7,5.0,0.88], [6.2,2.4,8.9,5.1,0.62], [1.2,5.3,3.8,7.4,0.55] ]; function iou(a,b){ const xi0=Math.max(a[0],b[0]), yi0=Math.max(a[1],b[1]); const xi1=Math.min(a[2],b[2]), yi1=Math.min(a[3],b[3]); const iw=Math.max(0,xi1-xi0), ih=Math.max(0,yi1-yi0); const inter=iw*ih; const areaA=(a[2]-a[0])*(a[3]-a[1]); const areaB=(b[2]-b[0])*(b[3]-b[1]); return inter/(areaA+areaB-inter+1e-12); } const order=[...boxes.keys()].sort((i,j)=>boxes[j][4]-boxes[i][4]); const keep=[]; order.forEach(i=>{ if(keep.every(k=>iou(boxes[i],boxes[k])<0.45)){keep.push(i);} }); function rect(b,name,keepFlag){ return {x:[b[0],b[2],b[2],b[0],b[0]],y:[b[1],b[1],b[3],b[3],b[1]],mode:'lines+text',text:['','','score '+b[4],'',''],textposition:'top center',name:name,line:{width:keepFlag?4:1,dash:keepFlag?'solid':'dot'},showlegend:false}; } const data=boxes.map((b,i)=>rect(b,'box '+i,keep.includes(i))); const layout={ title:'Non-maximum suppression keeps high-scoring non-overlapping boxes', xaxis:{range:[0,10],scaleanchor:'y',title:'x'}, yaxis:{range:[0,8],title:'y'}, margin:{t:55,l:45,r:20,b:45} }; if(window.Plotly){Plotly.newPlot('ch23-nms-demo',data,layout,{responsive:true});} })(); </script> ## 23.14 Evaluation: precision, recall, AP, and mAP Detection evaluation is more subtle than classification evaluation because predictions must be matched to ground-truth objects. For a fixed class $c$ and IoU threshold $\tau$, sort predicted boxes by confidence score. A prediction is counted as a true positive if: 1. its predicted class is $c$; 2. it has IoU at least $\tau$ with an unmatched ground-truth object of class $c$; 3. among duplicate predictions for the same object, only the highest-scoring one is counted as true positive. For a score threshold $t$, define $$ \operatorname{Precision}(t)= \frac{\operatorname{TP}(t)}{\operatorname{TP}(t)+\operatorname{FP}(t)}, $$ $$ \operatorname{Recall}(t)= \frac{\operatorname{TP}(t)}{\operatorname{TP}(t)+\operatorname{FN}(t)}. $$ As $t$ varies, the detector traces a precision-recall curve. Average precision is the area under this curve: $$ \operatorname{AP} =\int_0^1 p(r)\,dr, $$ where $p(r)$ is precision as a function of recall after appropriate interpolation. Mean average precision averages AP across classes and sometimes across IoU thresholds: $$ \operatorname{mAP} =\frac{1}{C}\sum_{c=1}^C \operatorname{AP}_c. $$ For COCO-style evaluation, one averages over multiple IoU thresholds: $$ \operatorname{mAP}_{.50:.95} =\frac{1}{10C}\sum_{k=0}^9\sum_{c=1}^C \operatorname{AP}_{c,\,0.50+0.05k}. $$ ### Statistical interpretation Precision is a conditional probability estimate: $$ \operatorname{Precision}\approx P(\text{true object}\mid\text{predicted detection}), $$ while recall estimates $$ \operatorname{Recall}\approx P(\text{detected}\mid\text{true object}). $$ A high-precision detector makes few false alarms; a high-recall detector misses few objects. The appropriate tradeoff depends on the application. ## 23.15 Interactive demo: precision-recall and AP ::: {.interactive-card} <div id="ch23-pr-demo" class="plotly-demo"></div> ::: <script> (function(){ const recall=[0,0.1,0.2,0.35,0.5,0.62,0.74,0.83,0.91,1.0]; const precision=[1.0,0.96,0.91,0.86,0.78,0.71,0.65,0.57,0.48,0.39]; let ap=0; for(let i=1;i<recall.length;i++){ap += (recall[i]-recall[i-1])*(precision[i]+precision[i-1])/2;} const data=[{x:recall,y:precision,mode:'lines+markers',fill:'tozeroy',name:'PR curve'}]; const layout={ title:'Precision-recall curve: approximate AP = '+ap.toFixed(3), xaxis:{title:'recall',range:[0,1]}, yaxis:{title:'precision',range:[0,1.05]}, margin:{t:55,l:60,r:20,b:50} }; if(window.Plotly){Plotly.newPlot('ch23-pr-demo',data,layout,{responsive:true});} })(); </script> ## 23.16 One-stage detectors One-stage detectors predict boxes and classes directly over a dense grid of feature-map locations. Schematically, $$ X\xrightarrow{\text{backbone}}F \xrightarrow{\text{detection head}} \{(\widehat b_j,\widehat p_j,\widehat s_j)\}_{j=1}^{N}. $$ The output index $j$ may correspond to a grid location, anchor, scale level, or query. One-stage detectors are usually faster than two-stage detectors because they avoid a separate proposal-and-crop step. ### YOLO-style grid prediction A YOLO-style detector partitions the image into grid cells. Each cell predicts a small number of boxes and class scores. The early mathematical idea is that each grid cell is responsible for objects whose centers fall inside that cell. This creates a map $$ \mathbb R^{H\times W\times C}\to \mathbb R^{S\times S\times B\times(4+1+C)}, $$ where $S\times S$ is the grid, $B$ is the number of boxes per cell, $4$ represents box coordinates, $1$ represents objectness, and $C$ represents class scores. ### SSD and multi-scale prediction Objects appear at different scales. Single Shot MultiBox Detector-style models predict boxes from multiple feature maps: $$ F_1,F_2,\dots,F_L, $$ where early maps have higher spatial resolution and later maps contain more semantic information. This is an architectural expression of a mathematical fact: detection must be sensitive to both small local details and large semantic context. ### Feature pyramid networks Feature pyramid networks construct multi-scale feature maps $$ P_2,P_3,\dots,P_L $$ using top-down and lateral connections. Each pyramid level is assigned objects within a range of sizes. FPNs connect detection to scale-space thinking from earlier chapters. ## 23.17 Anchor-free detection Anchor-free detectors avoid predefined anchor boxes. Instead, they often represent objects by centers, keypoints, or dense distances to box sides. For example, at a feature-map location $(i,j)$ inside a ground-truth box, a detector may predict $$ (l,t,r,b), $$ where $$ l=x-x_{\min},\qquad t=y-y_{\min}, \qquad r=x_{\max}-x, \qquad b=y_{\max}-y. $$ The decoded box is $$ (x-l,y-t,x+r,y+b). $$ Anchor-free methods reduce the need to hand-design scales and aspect ratios. They replace anchor assignment by location assignment, center sampling, keypoint matching, or optimal transport style assignment. ## 23.18 DETR-style detection as set prediction Transformer-based detectors such as DETR formulate detection as direct set prediction. The model outputs a fixed number $N$ of object queries: $$ \{(\widehat b_j,\widehat p_j):j=1,\dots,N\}. $$ Ground truth has $m\le N$ objects. Add $N-m$ no-object labels so both sides have size $N$. Training solves a bipartite matching problem: $$ \widehat\sigma =\arg\min_{\sigma\in S_N} \sum_{i=1}^N \mathcal C\bigl(y_i,\widehat y_{\sigma(i)}\bigr), $$ where $S_N$ is the permutation group and $\mathcal C$ is a matching cost. A typical matching cost combines classification and localization: $$ \mathcal C(y_i,\widehat y_j) = -\log \widehat p_j(c_i) +\lambda_1\|b_i-\widehat b_j\|_1 +\lambda_2 L_{\operatorname{GIoU}}(b_i,\widehat b_j). $$ After matching, the training loss is computed using the matched pairs. This removes the need for NMS in the original DETR formulation because each object should be assigned to one query. ::: {.callout-note} ## Set prediction viewpoint DETR changes the mathematical structure of detection from dense local prediction plus NMS to global set prediction plus bipartite matching. This is a clean example of how architecture can encode the symmetry of the output space: object sets are permutation invariant. ::: ## 23.19 Detection losses as multi-task learning A detector usually shares a visual backbone across tasks: $$ F_\theta(X)=\text{shared feature map}. $$ Different heads predict classification, localization, objectness, centerness, masks, or keypoints. The total loss has the form $$ L(\theta)= \lambda_{\operatorname{cls}}L_{\operatorname{cls}} +\lambda_{\operatorname{box}}L_{\operatorname{box}} +\lambda_{\operatorname{obj}}L_{\operatorname{obj}} +\lambda_{\operatorname{aux}}L_{\operatorname{aux}}. $$ Choosing the weights is not only an engineering issue. It affects the geometry of the optimization problem. If one loss term has much larger gradient norm than the others, it may dominate training. A useful diagnostic is to monitor gradient norms: $$ \|\nabla_\theta L_{\operatorname{cls}}\|, \qquad \|\nabla_\theta L_{\operatorname{box}}\|, \qquad \|\nabla_\theta L_{\operatorname{obj}}\|. $$ ## 23.20 Detection and uncertainty A detector produces scores, but a score is not automatically a calibrated probability. Detection uncertainty has several sources: 1. **classification uncertainty:** which class is present? 2. **localization uncertainty:** where exactly is the object boundary? 3. **objectness uncertainty:** is this region foreground or background? 4. **assignment uncertainty:** which prediction corresponds to which object? 5. **dataset uncertainty:** are annotations complete and consistent? For localization, one can model a box as a random vector $$ B\in\mathbb R^4, $$ with predicted mean $\mu_\theta(X)$ and covariance $\Sigma_\theta(X)$. A Gaussian localization likelihood would be $$ p(b\mid X;\theta) =\mathcal N\bigl(b;\mu_\theta(X),\Sigma_\theta(X)\bigr). $$ The negative log-likelihood is $$ \frac{1}{2}(b-\mu)^T\Sigma^{-1}(b-\mu) +\frac{1}{2}\log\det\Sigma +\text{constant}. $$ This formulation rewards accurate predictions but penalizes both overconfidence and underconfidence. ## 23.21 Python example: computing IoU and NMS ```{python} #| label: ch23-iou-nms-python #| eval: false import numpy as np def box_iou(boxes1, boxes2): """Compute pairwise IoU for boxes in xyxy format.""" boxes1 = np.asarray(boxes1, dtype=float) boxes2 = np.asarray(boxes2, dtype=float) x11, y11, x12, y12 = boxes1[:, 0], boxes1[:, 1], boxes1[:, 2], boxes1[:, 3] x21, y21, x22, y22 = boxes2[:, 0], boxes2[:, 1], boxes2[:, 2], boxes2[:, 3] xa = np.maximum(x11[:, None], x21[None, :]) ya = np.maximum(y11[:, None], y21[None, :]) xb = np.minimum(x12[:, None], x22[None, :]) yb = np.minimum(y12[:, None], y22[None, :]) inter = np.maximum(0, xb - xa) * np.maximum(0, yb - ya) area1 = np.maximum(0, x12 - x11) * np.maximum(0, y12 - y11) area2 = np.maximum(0, x22 - x21) * np.maximum(0, y22 - y21) union = area1[:, None] + area2[None, :] - inter return inter / np.maximum(union, 1e-12) def nms(boxes, scores, iou_threshold=0.5): boxes = np.asarray(boxes, dtype=float) scores = np.asarray(scores, dtype=float) order = scores.argsort()[::-1] keep = [] while order.size > 0: i = order[0] keep.append(i) if order.size == 1: break rest = order[1:] overlaps = box_iou(boxes[[i]], boxes[rest]).ravel() order = rest[overlaps < iou_threshold] return keep boxes = np.array([ [20, 20, 80, 80], [25, 25, 82, 82], [150, 40, 220, 120], [155, 45, 218, 125], ]) scores = np.array([0.95, 0.80, 0.90, 0.60]) print(box_iou(boxes, boxes)) print("kept indices:", nms(boxes, scores, iou_threshold=0.5)) ``` ## 23.22 Python example: average precision from detections ```{python} #| label: ch23-average-precision-python #| eval: false import numpy as np def average_precision_from_sorted_labels(tp_flags, num_ground_truth): """Compute AP from detections sorted by descending confidence. tp_flags[j] is 1 if detection j is a true positive and 0 otherwise. """ tp_flags = np.asarray(tp_flags, dtype=float) fp_flags = 1.0 - tp_flags tp_cum = np.cumsum(tp_flags) fp_cum = np.cumsum(fp_flags) recall = tp_cum / max(num_ground_truth, 1) precision = tp_cum / np.maximum(tp_cum + fp_cum, 1e-12) # Monotone precision envelope. mrec = np.r_[0.0, recall, 1.0] mpre = np.r_[0.0, precision, 0.0] for i in range(len(mpre) - 2, -1, -1): mpre[i] = max(mpre[i], mpre[i + 1]) change = np.where(mrec[1:] != mrec[:-1])[0] ap = np.sum((mrec[change + 1] - mrec[change]) * mpre[change + 1]) return ap, recall, precision # Example: detections sorted from most confident to least confident. tp_flags = [1, 0, 1, 1, 0, 0, 1] num_ground_truth = 5 ap, recall, precision = average_precision_from_sorted_labels(tp_flags, num_ground_truth) print("AP:", ap) print("recall:", recall) print("precision:", precision) ``` ## 23.23 Python example: anchor target assignment ```{python} #| label: ch23-anchor-assignment-python #| eval: false import numpy as np def make_grid_anchors(H=4, W=4, stride=32, scales=(32, 64), ratios=(0.5, 1.0, 2.0)): anchors = [] for i in range(H): for j in range(W): cx = (j + 0.5) * stride cy = (i + 0.5) * stride for s in scales: for r in ratios: w = s * np.sqrt(r) h = s / np.sqrt(r) anchors.append([cx - w / 2, cy - h / 2, cx + w / 2, cy + h / 2]) return np.array(anchors) anchors = make_grid_anchors() gt_boxes = np.array([[35, 40, 95, 110], [100, 20, 150, 70]]) ious = box_iou(anchors, gt_boxes) best_iou = ious.max(axis=1) positive = best_iou >= 0.5 negative = best_iou < 0.3 ignore = ~(positive | negative) print("number of anchors:", len(anchors)) print("positive:", positive.sum()) print("negative:", negative.sum()) print("ignore:", ignore.sum()) ``` ## 23.24 TensorFlow example: tiny dense detection head This example is not meant to be a production detector. It shows the shape logic of a dense detection head. ```{python} #| label: ch23-tensorflow-detection-head #| eval: false import tensorflow as tf num_classes = 5 num_anchors = 3 inputs = tf.keras.Input(shape=(224, 224, 3)) x = tf.keras.layers.Rescaling(1.0 / 255)(inputs) # Simple backbone. x = tf.keras.layers.Conv2D(32, 3, strides=2, padding="same", activation="relu")(x) x = tf.keras.layers.Conv2D(64, 3, strides=2, padding="same", activation="relu")(x) x = tf.keras.layers.Conv2D(128, 3, strides=2, padding="same", activation="relu")(x) x = tf.keras.layers.Conv2D(128, 3, padding="same", activation="relu")(x) # Shared detection features. f = tf.keras.layers.Conv2D(128, 3, padding="same", activation="relu")(x) # Classification: background plus object classes. cls_logits = tf.keras.layers.Conv2D( num_anchors * (num_classes + 1), kernel_size=3, padding="same", name="class_logits" )(f) # Box regression: four offsets per anchor. box_reg = tf.keras.layers.Conv2D( num_anchors * 4, kernel_size=3, padding="same", name="box_regression" )(f) model = tf.keras.Model(inputs=inputs, outputs=[cls_logits, box_reg]) model.summary() # For a 224 x 224 input and stride 8, the feature map is 28 x 28. # The classification output shape is (batch, 28, 28, A(C+1)). # The box output shape is (batch, 28, 28, 4A). ``` ## 23.25 AI component: using large models to study object detection AI tools can help students learn object detection, but they should be used in a mathematically disciplined way. Good prompts ask the model to produce equations, assumptions, failure cases, and verification tests. ### Prompt 1: verify coordinate conventions > I have boxes stored as $(x_c,y_c,w,h)$ in normalized coordinates. Derive the conversion to $(x_{\min},y_{\min},x_{\max},y_{\max})$. Then list three bugs that can occur when mixing normalized coordinates and pixel coordinates. ### Prompt 2: compare localization losses > Explain the difference among smooth $L_1$, IoU loss, generalized IoU loss, and Gaussian negative log-likelihood for bounding-box regression. For each loss, state what mathematical quantity it penalizes and one failure mode. ### Prompt 3: design an AP computation test > Create a tiny detection example with three ground-truth boxes and six predicted boxes. Sort predictions by score and compute precision, recall, and AP by hand. Include duplicate detections and one localization false positive. ### Prompt 4: critique a detector > A detector has high AP at IoU $0.50$ but much lower AP at IoU $0.75$. Give a mathematical diagnosis of this behavior and propose experiments to decide whether the problem is classification, localization, scale, annotation noise, or NMS. ### Prompt 5: connect DETR to matching > Explain why object detection can be formulated as a bipartite matching problem. Write the matching objective and explain why this makes the loss permutation invariant. ## 23.26 Common failure modes and mathematical diagnostics | Failure mode | Mathematical symptom | Possible diagnostic | |---|---|---| | duplicate detections | many high-IoU predictions for one object | inspect NMS threshold and score calibration | | missed small objects | low recall for small area bins | evaluate AP by object size | | poor localization | AP drops sharply as IoU threshold increases | compare AP50 and AP75 | | too many false positives | low precision at high scores | reliability and threshold analysis | | class confusion | off-diagonal confusion matrix | inspect class-conditional embeddings | | crowded-scene suppression | nearby objects removed by NMS | compare NMS and Soft-NMS | | anchor mismatch | few positive anchors | plot anchor-size/aspect-ratio coverage | | overconfident boxes | high scores for inaccurate boxes | calibrate scores and analyze localization uncertainty | ## 23.27 Mini-project: building a detection metric from scratch A useful independent-study project is to implement the evaluation pipeline without using a detection library. 1. Write a function for pairwise IoU. 2. For one class, sort predicted boxes by confidence. 3. Match each prediction to the best unmatched ground-truth box. 4. Mark true positives and false positives using an IoU threshold. 5. Compute cumulative precision and recall. 6. Compute AP. 7. Repeat for multiple IoU thresholds. 8. Compare how AP changes when localization noise increases. The key lesson is that detection metrics are not black boxes. They are explicit functions of scores, boxes, classes, matching rules, and IoU thresholds. ## 23.28 Exercises ### Conceptual exercises 1. Explain why object detection output is a set, not a vector with fixed ordering. 2. Give two reasons why detection is harder than image classification. 3. Explain the difference between objectness and class probability. 4. Explain why duplicate detections are a problem even if all boxes overlap a true object. 5. Describe a case where a detector has high recall but poor precision. 6. Describe a case where a detector has good classification but poor localization. ### Mathematical exercises 1. Convert a box from $(x_{\min},y_{\min},x_{\max},y_{\max})$ to $(x_c,y_c,w,h)$ and back. State the validity constraints. 2. Derive the IoU formula for two axis-aligned rectangles. 3. Show that IoU lies in $[0,1]$. 4. Suppose two boxes have the same center and aspect ratio, but one has width and height multiplied by $a>0$. Derive IoU as a function of $a$. 5. Derive the inverse transformation from anchor offsets $(t_x,t_y,t_w,t_h)$ to box coordinates. 6. For smooth $L_1$, compute the derivative of $\rho_\beta(u)$ and show that it is continuous at $|u|=\beta$. 7. Show that focal loss reduces to weighted cross-entropy when $\gamma=0$. 8. Given a sequence of true-positive and false-positive flags sorted by confidence, compute precision, recall, and AP. 9. Write the bipartite matching objective for a detector with three predictions and two true objects plus one no-object label. 10. Explain why NMS is not permutation invariant with respect to scores but is permutation invariant with respect to the input ordering after sorting. ### Python exercises 1. Implement pairwise IoU for a batch of boxes. 2. Implement NMS and test it on overlapping boxes. 3. Implement Soft-NMS and compare it with hard NMS. 4. Generate random ground-truth boxes and noisy predicted boxes. Plot AP as a function of localization noise. 5. Implement anchor generation for three feature-map levels. 6. Compute the number of positive, negative, and ignored anchors for a synthetic dataset. 7. Implement smooth $L_1$, IoU loss, and GIoU loss. 8. Implement focal loss and plot it for different values of $\gamma$. 9. Build a tiny TensorFlow detection head and verify output shapes. 10. Write a script that checks whether all predicted boxes satisfy coordinate validity constraints. ### AI-assisted exercises 1. Ask an AI tool to derive IoU for two boxes, then verify every step by writing a small numerical test. 2. Ask for a minimal object-detection evaluation example and check whether duplicate detections are handled correctly. 3. Ask for an explanation of focal loss, then rewrite the explanation using only probability and calculus. 4. Ask the tool to compare Faster R-CNN, RetinaNet, YOLO, and DETR using the categories: proposal mechanism, assignment rule, loss, post-processing, and output symmetry. 5. Ask the tool to generate likely detector failure cases for medical imaging, autonomous driving, and satellite imagery. For each case, identify the statistical issue. ## 23.29 Summary Object detection extends classification by adding geometry, cardinality, matching, and duplicate suppression. The core mathematical objects are boxes, scores, labels, IoU, assignment rules, and set-valued predictions. Classical detectors use sliding windows or region proposals. Dense detectors use anchors or anchor-free location assignment. Transformer-based detectors formulate detection as direct set prediction with bipartite matching. For applied mathematics and statistics students, the important lesson is that detection is not simply a neural network architecture. It is a structured inference problem whose behavior depends on geometry, optimization, class imbalance, calibration, and the precise definition of evaluation metrics. ## 23.30 Notes and references Object detection has a long history, including sliding-window methods, boosted cascade detectors, deformable part models, region-based convolutional detectors, one-stage dense detectors, and transformer-based set predictors. Important references include Viola--Jones detection [@viola2001rapid], deformable part models [@felzenszwalb2010object], R-CNN [@girshick2014rich], Fast R-CNN [@girshick2015fast], Faster R-CNN [@ren2015faster], SSD [@liu2016ssd], YOLO [@redmon2016you], feature pyramid networks [@lin2017feature], RetinaNet and focal loss [@lin2017focal], Mask R-CNN [@he2017mask], COCO evaluation [@lin2014microsoft], Soft-NMS [@bodla2017soft], generalized IoU [@rezatofighi2019generalized], and DETR [@carion2020end].