10 Morphology and Shape

Chapter goal. This chapter develops the mathematical language for binary morphology, grayscale morphology, connected components, skeletons, distance transforms, and shape descriptors. The emphasis is on clean definitions, algebraic properties, statistical interpretation, and the way these classical ideas still appear inside modern segmentation pipelines and AI vision systems.

Computer vision often begins with an image, but many tasks end with a shape. A medical segmentation model outputs a tumor region. A document-analysis system extracts characters. A satellite-image model finds roads and buildings. A robotics system estimates the silhouette of an object. In all these examples, the output is not merely a probability value; it is a subset of the image domain with geometry and topology.

Mathematical morphology is the part of image analysis that studies shapes through set operations. It was developed to describe how objects grow, shrink, merge, break apart, and simplify under local probes called structuring elements (Matheron 1975; Serra 1982; Soille 2003). In modern computer vision, morphology is still important because learned segmentation masks usually need geometric interpretation, topological checking, and post-processing. The tools in this chapter connect naturally to graph theory, metric geometry, topology, probability, and deep learning.

10.1 Learning objectives

After studying this chapter, you should be able to:

represent a binary image as a subset of a pixel grid or as an indicator function;
define dilation, erosion, opening, closing, boundary extraction, and morphological gradients;
prove basic algebraic properties such as monotonicity, duality, and idempotence of opening/closing;
compute connected components using graph adjacency;
explain distance transforms, medial axes, and skeletons;
compute shape descriptors from moments and covariance matrices;
interpret morphology as max-plus/min-plus local filtering;
understand how morphology appears in segmentation post-processing, medical imaging, and differentiable AI layers.

10.2 9.1 Binary images as sets

A grayscale image can be viewed as a function

\[ I: \Omega \subset \mathbb{R}^2 \to \mathbb{R}. \]

A binary image is a special image whose values are either 0 or 1. Equivalently, it is the indicator function of a subset:

\[ \mathbf{1}_A(x)= \begin{cases} 1, & x\in A,\\ 0, & x\notin A. \end{cases} \]

Here $A\subset \Omega$ is the foreground object. On a finite pixel grid, one usually writes

\[ \Omega_h=\{0,1,\ldots,m-1\}\times \{0,1,\ldots,n-1\}, \]

and a binary image is a subset $A\subseteq \Omega_h$.

This set viewpoint is useful because many shape operations can be expressed without mentioning pixel intensities. For example, “make the object thicker” becomes an operation on the set $A$. “Remove small holes” becomes an operation on the complement $A^c$. “Keep only large connected pieces” becomes a graph-theoretic operation on the adjacency graph of foreground pixels.

10.2.1 Indicator-function viewpoint

The same binary image can be viewed in three equivalent ways:

Representation	Mathematical object	Typical use
Set	$A\subseteq \Omega_h$	morphology, geometry, topology
Indicator function	$\mathbf{1}_A:\Omega_h\to\{0,1\}$	image processing, arrays
Random field	$Y_s\in\{0,1\}$ for each pixel $s$	statistics, segmentation uncertainty

For MA Applied Math and MS Statistics students, the third viewpoint is especially useful. A neural segmentation network may output probabilities

\[ P_\theta(Y_s=1\mid I),\qquad s\in \Omega_h, \]

and the final binary mask is often obtained by thresholding:

\[ \widehat{A}_\tau=\{s\in\Omega_h: P_\theta(Y_s=1\mid I)\ge \tau\}. \]

Morphology then acts on $\widehat{A}_\tau$ to enforce shape constraints.

10.3 9.2 Structuring elements

A structuring element is a small set $B\subseteq \mathbb{Z}^2$ used to probe the image. Common examples are:

\[ B_{\text{cross}}=\{(0,0),(1,0),(-1,0),(0,1),(0,-1)\}, \]

and

\[ B_{3\times 3}=\{-1,0,1\}\times\{-1,0,1\}. \]

The first gives 4-neighbor geometry; the second gives 8-neighbor geometry. More generally, one may use disks, lines, ellipses, or anisotropic elements that reflect prior knowledge about the shape.

For $z\in\mathbb{Z}^2$, define the translated structuring element

\[ B_z=\{b+z:b\in B\}. \]

The reflected structuring element is

\[ \check{B}=\{-b:b\in B\}. \]

Many morphology formulas use reflection. If $B$ is symmetric, then $B=\check{B}$.

10.4 9.3 Dilation and erosion

The two fundamental operations are dilation and erosion.

10.4.1 Dilation

The dilation of a set $A$ by a structuring element $B$ is

\[ A\oplus B=\{a+b:a\in A,\ b\in B\}. \]

Equivalently,

\[ A\oplus B=\{z: (\check{B})_z\cap A\neq \emptyset\}. \]

Dilation expands the foreground. A pixel $z$ is included in $A\oplus B$ if the structuring element placed at $z$ touches the original object.

10.4.2 Erosion

The erosion of $A$ by $B$ is

\[ A\ominus B=\{z:B_z\subseteq A\}. \]

Erosion shrinks the foreground. A pixel $z$ remains if the entire structuring element placed at $z$ fits inside the object.

10.4.3 Intuition

Operation	Local rule	Effect
Dilation	include if neighborhood intersects foreground	expands objects, fills small gaps
Erosion	include if neighborhood lies inside foreground	shrinks objects, removes thin parts

The two operations are not inverses. Erosion can delete small objects completely; dilation cannot recover them without additional information.

10.4.4 Dilation as Minkowski addition

Dilation is exactly a Minkowski sum:

\[ A\oplus B=A+B. \]

This connects morphology to convex geometry and mathematical analysis. If $A$ is a continuous planar set and $B_r$ is a disk of radius $r$, then

\[ A\oplus B_r=\{x\in\mathbb{R}^2: d(x,A)\le r\}, \]

where

\[ d(x,A)=\inf_{a\in A}\|x-a\|_2. \]

Thus dilation by a disk is the $r$-neighborhood of a set.

10.4.5 Erosion as distance from the complement

For a disk $B_r$,

\[ A\ominus B_r=\{x\in A: d(x,A^c)>r\}. \]

So erosion keeps points that are at least distance $r$ away from the background.

Statistical interpretation

If a predicted segmentation mask has boundary uncertainty of roughly $r$ pixels, then dilation and erosion by a radius-$r$ structuring element produce outer and inner uncertainty bands. This is useful in medical imaging when the exact boundary is uncertain.

10.5 9.4 Interactive: dilation and erosion on a binary grid

The following interactive shows how dilation and erosion act on a synthetic binary object. Increase the radius to see how the set expands or shrinks.

Dilation and erosion on a binary grid

Choose a morphological operation and a square structuring-element radius.

Operation: radius = 1

10.6 9.5 Duality, monotonicity, and algebraic properties

Morphological operations have clean algebraic structure. Let $A,C\subseteq \Omega$ and let $B$ be a structuring element.

10.6.1 Monotonicity

If $A\subseteq C$, then

\[ A\oplus B\subseteq C\oplus B \]

and

\[ A\ominus B\subseteq C\ominus B. \]

Proof. If $z\in A\oplus B$, then $z=a+b$ for some $a\in A$ and $b\in B$. Since $A\subseteq C$, $a\in C$, so $z\in C\oplus B$. For erosion, if $z\in A\ominus B$, then $B_z\subseteq A\subseteq C$, hence $z\in C\ominus B$. $\square$

10.6.2 Duality

Dilation and erosion are dual under complementation:

\[ (A\oplus B)^c=A^c\ominus \check{B}, \]

and

\[ (A\ominus B)^c=A^c\oplus \check{B}. \]

For symmetric $B$, this simplifies to

\[ (A\oplus B)^c=A^c\ominus B, \qquad (A\ominus B)^c=A^c\oplus B. \]

This explains why dilation of the foreground is equivalent to erosion of the background.

10.6.3 Translation equivariance

For a shift $t\in\mathbb{Z}^2$, define

\[ A+t=\{a+t:a\in A\}. \]

Then

\[ (A+t)\oplus B=(A\oplus B)+t, \]

and

\[ (A+t)\ominus B=(A\ominus B)+t. \]

This resembles the translation equivariance of convolution. In fact, dilation and erosion can be viewed as nonlinear translation-equivariant local operators.

10.7 9.6 Opening and closing

Two important composite operations are opening and closing.

10.7.1 Opening

The opening of $A$ by $B$ is

\[ A\circ B=(A\ominus B)\oplus B. \]

Opening erodes first and then dilates. It removes small foreground components and thin protrusions while preserving larger shapes.

10.7.2 Closing

The closing of $A$ by $B$ is

\[ A\bullet B=(A\oplus B)\ominus B. \]

Closing dilates first and then erodes. It fills small background holes and narrow gaps.

10.7.3 Properties

Opening is:

anti-extensive: $A\circ B\subseteq A$;
monotone: if $A\subseteq C$, then $A\circ B\subseteq C\circ B$;
idempotent: $(A\circ B)\circ B=A\circ B$.

Closing is:

extensive: $A\subseteq A\bullet B$;
monotone: if $A\subseteq C$, then $A\bullet B\subseteq C\bullet B$;
idempotent: $(A\bullet B)\bullet B=A\bullet B$.

Idempotence means that applying the operation twice gives no further change. This is a hallmark of a shape simplification operator.

10.7.4 Opening as a shape filter

Opening can be interpreted as keeping only those parts of $A$ into which a translated copy of $B$ can fit. Formally,

\[ A\circ B=\bigcup_{z:B_z\subseteq A} B_z. \]

Thus opening approximates $A$ by a union of copies of the structuring element.

Practical rule

Use opening to remove small false-positive islands. Use closing to fill small false-negative holes. Use both carefully: too large a structuring element can remove real anatomical or geometric details.

10.8 9.7 Interactive: opening and closing

Opening and closing as shape filters

Opening removes small bright structures; closing fills small dark holes and gaps.

Operation: radius = 1

10.9 9.8 Boundary extraction and morphological gradients

The internal boundary of $A$ with respect to $B$ is

\[ \partial_B A=A\setminus(A\ominus B). \]

It consists of foreground pixels that do not survive erosion. The external boundary can be written

\[ \partial_B^{\text{ext}} A=(A\oplus B)\setminus A. \]

The morphological gradient is

\[ G_B(A)=(A\oplus B)\setminus(A\ominus B). \]

For grayscale images, a morphological gradient is often defined as

\[ G_B f(x)=(f\oplus B)(x)-(f\ominus B)(x), \]

where grayscale dilation and erosion are introduced below. This acts like an edge detector but uses local maxima and minima rather than local derivatives.

10.10 9.9 Grayscale morphology

For a grayscale image $f:\Omega_h\to\mathbb{R}$ and a flat structuring element $B\subseteq \mathbb{Z}^2$, define grayscale dilation by

\[ (f\oplus B)(x)=\max_{b\in B} f(x-b), \]

and grayscale erosion by

\[ (f\ominus B)(x)=\min_{b\in B} f(x+b). \]

These are max and min filters. They are nonlinear analogues of convolution.

10.10.1 Non-flat structuring elements

A non-flat structuring element is a function $g:B\to\mathbb{R}$. Then dilation can be defined as

\[ (f\oplus g)(x)=\max_{b\in B}\{f(x-b)+g(b)\}, \]

and erosion as

\[ (f\ominus g)(x)=\min_{b\in B}\{f(x+b)-g(b)\}. \]

This places morphology in max-plus algebra and min-plus algebra. These algebraic systems replace ordinary addition and multiplication with operations such as maximum plus addition. They appear in optimization, dynamic programming, and shortest-path problems.

10.10.2 Relation to pooling in neural networks

Max pooling in CNNs is a grayscale dilation over a local window followed by subsampling. Min pooling can be implemented as

\[ \min_{b\in B} f(x+b)=-\max_{b\in B}\{-f(x+b)\}. \]

Thus erosion is negative max pooling applied to the negative image.

10.11 9.10 Connected components and adjacency graphs

A binary image $A\subseteq\Omega_h$ can be turned into a graph. Let

\[ G_A=(V_A,E_A), \]

where

\[ V_A=A \]

and where two foreground pixels are connected by an edge if they are neighbors. With 4-connectivity,

\[ (i,j)\sim(i',j') \quad \Longleftrightarrow \quad |i-i'|+|j-j'|=1. \]

With 8-connectivity,

\[ (i,j)\sim(i',j') \quad \Longleftrightarrow \quad \max\{|i-i'|,|j-j'|\}=1. \]

A connected component is a connected component of this graph. Component labeling is a standard operation in post-processing segmentation masks.

10.11.1 Why connectivity matters

The choice of foreground connectivity affects topology. If diagonal pixels count as connected under 8-connectivity, then two diagonal pixels belong to the same component. Under 4-connectivity, they do not.

To avoid paradoxes, one often uses complementary choices:

Foreground connectivity	Background connectivity
4-connected foreground	8-connected background
8-connected foreground	4-connected background

This prevents ambiguous diagonal connections.

10.11.2 Euler characteristic

For a planar binary shape, the Euler characteristic is

\[ \chi(A)=\#\{\text{connected components}\}-\#\{\text{holes}\}. \]

In topological language,

\[ \chi(A)=\beta_0(A)-\beta_1(A), \]

where $\beta_0$ counts connected components and $\beta_1$ counts one-dimensional holes.

This is a simple but powerful topological descriptor. In medical image segmentation, for example, one may expect a certain anatomical structure to have one connected component and no holes. A segmentation mask that violates this expectation may need correction.

Connection to topological data analysis

Morphology modifies the topology of a binary set. Dilation can merge components and fill holes. Erosion can split components and create holes. Persistent homology tracks how connected components and holes change over a scale parameter.

10.12 9.11 Distance transforms

For a foreground set $A\subseteq\Omega_h$, the distance transform assigns to each pixel its distance to the nearest foreground pixel:

\[ D_A(x)=\min_{a\in A} d(x,a). \]

Common choices include:

\[ d_1(x,a)=|x_1-a_1|+|x_2-a_2|, \]

\[ d_2(x,a)=\sqrt{(x_1-a_1)^2+(x_2-a_2)^2}, \]

and

\[ d_\infty(x,a)=\max\{|x_1-a_1|,|x_2-a_2|\}. \]

For points inside the foreground, one often computes distance to the background:

\[ D_{A^c}(x)=\min_{y\in A^c}d(x,y),\qquad x\in A. \]

This value is large near the center of thick regions and small near the boundary.

10.12.1 Signed distance function

A signed distance function is

\[ \phi_A(x)= \begin{cases} -d(x,A^c), & x\in A,\\ d(x,A), & x\notin A. \end{cases} \]

The boundary is represented by the zero level set:

\[ \partial A=\{x:\phi_A(x)=0\}. \]

Signed distance functions connect morphology to level-set methods, shape optimization, and PDE-based vision.

10.13 9.12 Skeletons and medial axes

The medial axis of a shape is the set of centers of maximal balls contained in the shape. Informally, it is the set of points that have at least two closest boundary points.

For a continuous shape $A\subset\mathbb{R}^2$, a disk $B(x,r)$ is maximal in $A$ if

\[ B(x,r)\subseteq A \]

and it is not strictly contained in a larger disk contained in $A$. The medial axis consists of the centers of these maximal disks.

A skeleton should preserve the main topology and geometry of the shape while reducing it to a thin centerline. This is useful for roads, vessels, plant roots, handwriting, and shape matching.

10.13.1 Morphological skeleton

One classical morphological skeleton is defined using erosions and openings:

\[ S_k(A)=(A\ominus kB)\setminus\big((A\ominus kB)\circ B\big), \]

where $kB$ denotes repeated dilation of $B$ with itself. The full skeleton is

\[ S(A)=\bigcup_{k\ge 0} S_k(A). \]

The original shape can be approximately reconstructed from skeleton layers by

\[ A=\bigcup_{k\ge 0} \left(S_k(A)\oplus kB\right), \]

under suitable discrete assumptions.

10.14 9.13 Interactive: distance transform and skeleton intuition

Distance transform and skeleton intuition

The skeleton is approximated by ridges of the distance-to-boundary function.

View:

10.15 9.14 Shape descriptors

A shape descriptor maps a set $A$ to a numerical feature vector:

\[ \Psi(A)\in\mathbb{R}^d. \]

Good descriptors should be stable, interpretable, and relevant to the task. Some descriptors should be invariant to translation, rotation, or scale; others should preserve orientation and size.

10.15.1 Area and perimeter

The area of a binary shape is the number of foreground pixels:

\[ |A|=\sum_{(i,j)\in\Omega_h}\mathbf{1}_A(i,j). \]

Perimeter can be approximated by counting boundary pixels, or more accurately by using contour length estimates. A simple boundary-count estimate is

\[ P_B(A)=|A\setminus(A\ominus B)|. \]

10.15.2 Centroid

The centroid is

\[ \bar{x}=\frac{1}{|A|}\sum_{x\in A}x. \]

Writing $x=(x_1,x_2)^T$, this gives

\[ \bar{x}=\begin{pmatrix}\bar{x}_1\\ \bar{x}_2\end{pmatrix}. \]

The centroid is translation equivariant: shifting the shape shifts the centroid by the same amount.

10.15.3 Covariance matrix of a shape

The shape covariance matrix is

\[ \Sigma_A=\frac{1}{|A|}\sum_{x\in A}(x-\bar{x})(x-\bar{x})^T. \]

The eigenvectors of $\Sigma_A$ give principal axes. The eigenvalues describe spread along those axes. If

\[ \lambda_1\ge \lambda_2\ge 0, \]

then an eccentricity-type descriptor is

\[ e(A)=\sqrt{1-\frac{\lambda_2}{\lambda_1}}. \]

This is close to 0 for round shapes and close to 1 for elongated shapes.

10.15.4 Moment descriptors

Raw moments are

\[ m_{pq}=\sum_{(x,y)\in A} x^p y^q. \]

Central moments are

\[ \mu_{pq}=\sum_{(x,y)\in A}(x-\bar{x})^p(y-\bar{y})^q. \]

Normalized central moments are

\[ \eta_{pq}=\frac{\mu_{pq}}{\mu_{00}^{1+(p+q)/2}}. \]

Moment invariants combine normalized central moments to obtain descriptors invariant under translation, scaling, and sometimes rotation. These are useful for classical shape recognition but can be unstable for noisy boundaries.

10.15.5 Compactness and circularity

A common compactness measure is

\[ C(A)=\frac{4\pi |A|}{P(A)^2}. \]

For an ideal disk, $C(A)=1$. For elongated or irregular shapes, $C(A)$ is smaller.

Discrete caution

Pixel-based perimeter estimates can be biased by grid orientation. For statistical analysis, use consistent definitions across the dataset, and be careful when comparing shapes at different resolutions.

10.16 9.15 Interactive: moments and principal axes

Shape covariance and principal axes

Change the elongation and rotation of a synthetic ellipse. The covariance eigenvectors estimate the principal axes.

angle = 30 degrees elongation = 2.0

10.17 9.16 Fourier descriptors for shape contours

Suppose the boundary of a planar shape is sampled as an ordered sequence of points

\[ z_k=x_k+iy_k,\qquad k=0,1,\ldots,N-1. \]

The discrete Fourier transform of the contour is

\[ \widehat{z}_\ell=\sum_{k=0}^{N-1} z_k\exp\left(-2\pi i k\ell/N\right). \]

Low-frequency coefficients describe the coarse shape, while high-frequency coefficients describe fine boundary details. A smoothed contour can be reconstructed from a truncated Fourier series:

\[ z_k^{(K)}=\frac{1}{N}\sum_{|\ell|\le K}\widehat{z}_\ell\exp\left(2\pi i k\ell/N\right). \]

Fourier descriptors make shape analysis look like signal processing on closed curves. Translation is controlled by the zero-frequency coefficient, scaling by coefficient magnitudes, and rotation by phase.

10.18 9.17 Interactive: Fourier shape reconstruction

Fourier descriptors of a closed contour

Low frequencies preserve the coarse shape. Higher frequencies add boundary details.

number of frequency pairs K = 3

10.19 9.18 Granulometry and morphological scale

In many applications, object size is itself a feature. A granulometry studies how the area of a shape changes under openings of increasing size. Let $B_r$ be a structuring element of scale $r$. Define

\[ G_A(r)=|A\circ B_r|. \]

As $r$ increases, small structures disappear. The function $G_A(r)$ is nonincreasing. A discrete size distribution can be defined by differences:

\[ \Delta G_A(r)=G_A(r)-G_A(r+1). \]

Large drops indicate that many structures of size approximately $r$ were removed. This is useful for particle analysis, materials imaging, cell morphology, and remote sensing.

10.20 9.19 Statistical viewpoint: shapes as estimators

In statistical vision, the true object $A^*$ is unknown. A model produces an estimated mask $\widehat{A}$. Shape descriptors become statistics:

\[ T(\widehat{A})=\Psi(\widehat{A}). \]

Examples include area, perimeter, eccentricity, number of components, and compactness.

10.20.1 Bias from morphology

Morphological operations can introduce bias. If $A$ is dilated by a disk of radius $r$, its area generally increases. For sufficiently regular planar sets, the continuous Steiner formula gives

\[ |A\oplus B_r|=|A|+rP(A)+\pi r^2, \]

for convex $A$. This formula shows that dilation changes area in a predictable way involving perimeter.

For erosion,

\[ |A\ominus B_r|\approx |A|-rP(A)+\pi r^2, \]

under regularity assumptions and for small $r$. These formulas are useful when interpreting morphological preprocessing statistically.

10.20.2 Uncertainty in shape descriptors

Suppose a segmentation model outputs random masks $A^{(1)},\ldots,A^{(M)}$, for example from test-time augmentation, Monte Carlo dropout, or posterior sampling. A shape descriptor $\Psi$ then has an empirical distribution:

\[ \Psi(A^{(1)}),\ldots,\Psi(A^{(M)}). \]

We can estimate uncertainty using

\[ \widehat{\operatorname{Var}}[\Psi] =\frac{1}{M-1}\sum_{m=1}^M \left(\Psi(A^{(m)})-\overline{\Psi}\right)^2. \]

This is important when a downstream scientific conclusion depends on a shape measurement.

10.21 9.20 AI components: morphology in modern vision systems

Classical morphology remains important in AI-based computer vision.

10.21.1 1. Post-processing segmentation masks

A neural network often produces a probability map

\[ p_\theta(x)=P_\theta(Y_x=1\mid I). \]

A common pipeline is:

\[ I \longrightarrow p_\theta \longrightarrow \widehat{A}_\tau \longrightarrow \text{morphological cleanup} \longrightarrow \text{component filtering}. \]

For example:

threshold the probability map;
remove small connected components;
close small holes;
compute area, perimeter, and shape descriptors;
flag masks with unusual topology.

10.21.2 2. Differentiable morphology

Grayscale dilation and erosion can be written using max and min operations. A differentiable approximation to maximum is the log-sum-exp function:

\[ \max_i a_i \approx \frac{1}{\alpha}\log\sum_i \exp(\alpha a_i), \]

where large $\alpha$ gives a closer approximation. This permits differentiable morphology layers inside neural networks.

10.21.3 3. Max pooling as morphology

Max pooling is a local dilation:

\[ (\operatorname{MaxPool}_B f)(x)=\max_{b\in B} f(x+b). \]

This creates a direct bridge between morphology and CNNs.

10.21.4 4. Topological and shape-aware losses

For segmentation, pixelwise loss may not be enough. A mask can have high pixel accuracy but incorrect topology. Shape-aware training may include penalties for:

incorrect number of connected components;
holes that should not exist;
disconnected vessels or roads;
unrealistic area/perimeter ratios;
deviation from expected shape priors.

10.21.5 5. Foundation-model masks still need shape analysis

Promptable segmentation systems can produce high-quality masks, but the outputs are still sets. For scientific and medical use, one must measure, validate, and summarize these sets. Morphology and shape descriptors remain essential.

10.22 9.21 Python example: binary morphology

The following code illustrates basic morphology using scipy.ndimage. The code is marked eval: false so that the book does not require SciPy during rendering.

import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage as ndi

n = 80
y, x = np.mgrid[-1:1:complex(n), -1:1:complex(n)]
mask = (x**2 / 0.45**2 + y**2 / 0.30**2) <= 1
mask[(x > 0.05) & (x < 0.25) & (y > -0.1) & (y < 0.15)] = False
mask[(x + 0.65)**2 + (y - 0.55)**2 < 0.04**2] = True

structure = np.ones((5, 5), dtype=bool)

dilated = ndi.binary_dilation(mask, structure=structure)
eroded = ndi.binary_erosion(mask, structure=structure)
opened = ndi.binary_opening(mask, structure=structure)
closed = ndi.binary_closing(mask, structure=structure)

fig, axes = plt.subplots(1, 5, figsize=(12, 3))
for ax, img, title in zip(
    axes,
    [mask, dilated, eroded, opened, closed],
    ["original", "dilation", "erosion", "opening", "closing"]
):
    ax.imshow(img, cmap="gray")
    ax.set_title(title)
    ax.axis("off")
plt.tight_layout()
plt.show()

10.23 9.22 Python example: connected components and shape features

import numpy as np
from scipy import ndimage as ndi

# Suppose mask is a binary segmentation result.
labeled, num_components = ndi.label(mask)
component_sizes = ndi.sum(mask, labeled, index=np.arange(1, num_components + 1))

print("number of connected components:", num_components)
print("component sizes:", component_sizes)

# Keep only components with at least 50 pixels.
keep_labels = np.where(component_sizes >= 50)[0] + 1
clean = np.isin(labeled, keep_labels)

# Simple centroid and covariance matrix of the largest component.
largest_label = int(np.argmax(component_sizes) + 1)
coords = np.column_stack(np.nonzero(labeled == largest_label))
centroid = coords.mean(axis=0)
centered = coords - centroid
cov = centered.T @ centered / coords.shape[0]
evals, evecs = np.linalg.eigh(cov)

print("centroid:", centroid)
print("covariance matrix:\n", cov)
print("principal variances:", evals[::-1])
print("principal directions:\n", evecs[:, ::-1])

10.24 9.23 Python example: distance transform and skeleton

import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage as ndi
from skimage.morphology import medial_axis

# Distance from foreground pixels to background.
dist = ndi.distance_transform_edt(mask)

# Skeleton from the medial axis.
skeleton, dist_on_skeleton = medial_axis(mask, return_distance=True)

fig, axes = plt.subplots(1, 3, figsize=(10, 3))
axes[0].imshow(mask, cmap="gray")
axes[0].set_title("mask")
axes[1].imshow(dist, cmap="viridis")
axes[1].set_title("distance transform")
axes[2].imshow(skeleton, cmap="gray")
axes[2].set_title("skeleton")
for ax in axes:
    ax.axis("off")
plt.tight_layout()
plt.show()

10.25 9.24 Python example: grayscale morphology

import numpy as np
import matplotlib.pyplot as plt
from scipy import ndimage as ndi

rng = np.random.default_rng(7243)
n = 100
y, x = np.mgrid[-1:1:complex(n), -1:1:complex(n)]
image = np.exp(-((x + 0.25)**2 + (y + 0.1)**2) / 0.08)
image += 0.7 * np.exp(-((x - 0.35)**2 + (y - 0.25)**2) / 0.02)
image += 0.08 * rng.normal(size=(n, n))

size = 7
gray_dilation = ndi.maximum_filter(image, size=size)
gray_erosion = ndi.minimum_filter(image, size=size)
gray_gradient = gray_dilation - gray_erosion

fig, axes = plt.subplots(1, 4, figsize=(12, 3))
for ax, img, title in zip(
    axes,
    [image, gray_dilation, gray_erosion, gray_gradient],
    ["image", "dilation/max filter", "erosion/min filter", "morphological gradient"]
):
    ax.imshow(img, cmap="gray")
    ax.set_title(title)
    ax.axis("off")
plt.tight_layout()
plt.show()

10.26 9.25 TensorFlow example: morphology using pooling

The next example implements grayscale dilation and erosion using max pooling. This is useful because it connects morphology directly to neural-network layers.

import tensorflow as tf


def dilation2d(x, ksize=3):
    """Flat grayscale dilation for NHWC tensors."""
    return tf.nn.max_pool2d(
        x,
        ksize=ksize,
        strides=1,
        padding="SAME"
    )


def erosion2d(x, ksize=3):
    """Flat grayscale erosion using negative max pooling."""
    return -tf.nn.max_pool2d(
        -x,
        ksize=ksize,
        strides=1,
        padding="SAME"
    )


def opening2d(x, ksize=3):
    return dilation2d(erosion2d(x, ksize), ksize)


def closing2d(x, ksize=3):
    return erosion2d(dilation2d(x, ksize), ksize)

# Example tensor: batch size 1, height 64, width 64, one channel.
x = tf.random.uniform((1, 64, 64, 1))
y_open = opening2d(x, ksize=5)
y_close = closing2d(x, ksize=5)

print(y_open.shape, y_close.shape)

10.27 9.26 AI-assisted study prompts

Use an AI assistant as a mathematical critic, not as a replacement for derivation.

10.27.1 Prompt 1: definitions

Explain dilation, erosion, opening, and closing using both set notation and indicator-function notation. Give one example where opening is useful and one example where closing is useful.

10.27.2 Prompt 2: proof checking

I claim that opening is idempotent. Write a proof using the definition $A\circ B=(A\ominus B)\oplus B$. Identify any assumptions about the structuring element.

10.27.3 Prompt 3: segmentation post-processing

A neural network outputs a noisy probability map for a tumor. Design a mask post-processing pipeline using thresholding, opening, closing, connected components, and shape descriptors. Explain the statistical risk of each step.

10.27.4 Prompt 4: coding critique

Review my Python code for connected-component filtering. Check whether it accidentally removes true small objects and suggest a validation strategy.

10.27.5 Prompt 5: shape descriptors

Compare area, perimeter, compactness, eccentricity, and Euler characteristic as shape features. Which are invariant to translation, rotation, and scaling?

10.28 9.27 Common mistakes

Confusing dilation with blurring. Dilation is not averaging. It expands sets or takes local maxima.
Using too large a structuring element. This can remove real small structures.
Ignoring connectivity choice. 4-connectivity and 8-connectivity can give different component counts.
Treating post-processing as harmless. Morphology changes statistical estimators such as area and perimeter.
Measuring topology after resizing without care. Interpolation and thresholding can create or destroy components and holes.
Using raw perimeter counts across resolutions. Perimeter estimates depend strongly on pixel size and contour representation.

10.29 9.28 Exercises

10.29.1 Conceptual exercises

Explain why dilation expands a foreground set but erosion shrinks it.
Give an example where opening removes a false positive but also removes a true object.
Give an example where closing fills a hole but incorrectly merges two nearby objects.
Explain the difference between 4-connectivity and 8-connectivity.
Explain why the skeleton of a shape is sensitive to boundary noise.

10.29.2 Mathematical exercises

Prove that dilation is monotone: if $A\subseteq C$, then $A\oplus B\subseteq C\oplus B$.
Prove the duality formula \[ (A\oplus B)^c=A^c\ominus \check{B}. \]
Show that opening is anti-extensive: $A\circ B\subseteq A$.
Show that closing is extensive: $A\subseteq A\bullet B$.
Suppose $B$ is symmetric and contains the origin. Prove that $A\ominus B\subseteq A\subseteq A\oplus B$.
For a disk-shaped set $A$ and disk structuring element $B_r$, describe geometrically $A\oplus B_r$ and $A\ominus B_r$.
Let $A$ consist of two disks whose centers are distance $d$ apart. For a disk structuring element of radius $r$, determine when dilation causes the two components to merge.
Derive the centroid and covariance matrix of an ideal continuous ellipse centered at the origin.

10.29.3 Python exercises

Generate a binary image with three components of different sizes. Use connected-component labeling to remove components below a size threshold.
Compare opening with square and disk-like structuring elements.
Compute the area, centroid, covariance matrix, and eccentricity of a binary mask.
Add random salt-and-pepper noise to a binary image and compare median filtering, opening, and closing.
Compute a distance transform and overlay the skeleton on the original image.
Implement binary dilation from scratch using only NumPy loops. Then compare with scipy.ndimage.binary_dilation.
Implement grayscale dilation using TensorFlow max pooling.
Create a small experiment showing how estimated area changes after dilation with increasing radius.

10.29.4 AI and modeling exercises

Take a segmentation model output from any image dataset. Apply thresholding, opening, closing, and connected-component filtering. Report how IoU and Dice change.
Use test-time augmentation to obtain multiple predicted masks. Compute uncertainty intervals for area and eccentricity.
Design a shape-aware validation rule for medical segmentation masks.
Explain how morphology could be used to post-process masks produced by a promptable segmentation model.
Compare a CNN max-pooling layer with grayscale dilation. What is the same? What is different?
Propose a differentiable approximation to erosion and explain where it could be used in a neural network.

10.30 9.29 Chapter summary

This chapter treated shapes as mathematical objects. A binary image is a set, an indicator function, and possibly a random field. Morphological operations such as dilation, erosion, opening, and closing manipulate sets through local probes called structuring elements. These operations have clean algebraic properties: monotonicity, duality, translation equivariance, extensivity or anti-extensivity, and idempotence.

We then moved from morphology to shape analysis. Connected components turn masks into graphs. Distance transforms and skeletons describe the geometry of interiors. Moments, covariance matrices, compactness, eccentricity, and Fourier descriptors summarize shape numerically. Finally, we connected these classical tools to modern AI vision pipelines: segmentation post-processing, differentiable morphology, max pooling, shape-aware losses, and validation of foundation-model masks.

The main message is that computer vision does not end when a model produces a mask. The mask is a mathematical object. Its geometry, topology, uncertainty, and statistical meaning must still be analyzed.

10.31 References and further reading

Classical mathematical morphology was developed by Matheron and Serra (Matheron 1975; Serra 1982). A practical image-analysis treatment is given by Soille (Soille 2003). For general image processing and morphology operations, see Gonzalez and Woods (Gonzalez and Woods 2018). For connected components, segmentation, and shape analysis in a broad computer vision context, see Szeliski (Szeliski 2022).

Gonzalez, Rafael C., and Richard E. Woods. 2018. Digital Image Processing. 4th ed. Pearson.

Matheron, Georges. 1975. Random Sets and Integral Geometry. Wiley.

Serra, Jean. 1982. Image Analysis and Mathematical Morphology. Academic Press.

Soille, Pierre. 2003. Morphological Image Analysis: Principles and Applications. 2nd ed. Springer.

Szeliski, Richard. 2022. Computer Vision: Algorithms and Applications. 2nd ed. Springer. https://szeliski.org/Book/.

# Morphology and Shape {#sec-morphology-shape} > **Chapter goal.** This chapter develops the mathematical language for binary morphology, grayscale morphology, connected components, skeletons, distance transforms, and shape descriptors. The emphasis is on clean definitions, algebraic properties, statistical interpretation, and the way these classical ideas still appear inside modern segmentation pipelines and AI vision systems. Computer vision often begins with an image, but many tasks end with a **shape**. A medical segmentation model outputs a tumor region. A document-analysis system extracts characters. A satellite-image model finds roads and buildings. A robotics system estimates the silhouette of an object. In all these examples, the output is not merely a probability value; it is a subset of the image domain with geometry and topology. Mathematical morphology is the part of image analysis that studies shapes through set operations. It was developed to describe how objects grow, shrink, merge, break apart, and simplify under local probes called **structuring elements** [@matheron1975; @serra1982; @soille2003]. In modern computer vision, morphology is still important because learned segmentation masks usually need geometric interpretation, topological checking, and post-processing. The tools in this chapter connect naturally to graph theory, metric geometry, topology, probability, and deep learning. ## Learning objectives After studying this chapter, you should be able to: 1. represent a binary image as a subset of a pixel grid or as an indicator function; 2. define dilation, erosion, opening, closing, boundary extraction, and morphological gradients; 3. prove basic algebraic properties such as monotonicity, duality, and idempotence of opening/closing; 4. compute connected components using graph adjacency; 5. explain distance transforms, medial axes, and skeletons; 6. compute shape descriptors from moments and covariance matrices; 7. interpret morphology as max-plus/min-plus local filtering; 8. understand how morphology appears in segmentation post-processing, medical imaging, and differentiable AI layers. ## 9.1 Binary images as sets A grayscale image can be viewed as a function $$ I: \Omega \subset \mathbb{R}^2 \to \mathbb{R}. $$ A **binary image** is a special image whose values are either 0 or 1. Equivalently, it is the indicator function of a subset: $$ \mathbf{1}_A(x)= \begin{cases} 1, & x\in A,\\ 0, & x\notin A. \end{cases} $$ Here $A\subset \Omega$ is the foreground object. On a finite pixel grid, one usually writes $$ \Omega_h=\{0,1,\ldots,m-1\}\times \{0,1,\ldots,n-1\}, $$ and a binary image is a subset $A\subseteq \Omega_h$. This set viewpoint is useful because many shape operations can be expressed without mentioning pixel intensities. For example, "make the object thicker" becomes an operation on the set $A$. "Remove small holes" becomes an operation on the complement $A^c$. "Keep only large connected pieces" becomes a graph-theoretic operation on the adjacency graph of foreground pixels. ### Indicator-function viewpoint The same binary image can be viewed in three equivalent ways: | Representation | Mathematical object | Typical use | |---|---|---| | Set | $A\subseteq \Omega_h$ | morphology, geometry, topology | | Indicator function | $\mathbf{1}_A:\Omega_h\to\{0,1\}$ | image processing, arrays | | Random field | $Y_s\in\{0,1\}$ for each pixel $s$ | statistics, segmentation uncertainty | For MA Applied Math and MS Statistics students, the third viewpoint is especially useful. A neural segmentation network may output probabilities $$ P_\theta(Y_s=1\mid I),\qquad s\in \Omega_h, $$ and the final binary mask is often obtained by thresholding: $$ \widehat{A}_\tau=\{s\in\Omega_h: P_\theta(Y_s=1\mid I)\ge \tau\}. $$ Morphology then acts on $\widehat{A}_\tau$ to enforce shape constraints. ## 9.2 Structuring elements A **structuring element** is a small set $B\subseteq \mathbb{Z}^2$ used to probe the image. Common examples are: $$ B_{\text{cross}}=\{(0,0),(1,0),(-1,0),(0,1),(0,-1)\}, $$ and $$ B_{3\times 3}=\{-1,0,1\}\times\{-1,0,1\}. $$ The first gives 4-neighbor geometry; the second gives 8-neighbor geometry. More generally, one may use disks, lines, ellipses, or anisotropic elements that reflect prior knowledge about the shape. For $z\in\mathbb{Z}^2$, define the translated structuring element $$ B_z=\{b+z:b\in B\}. $$ The reflected structuring element is $$ \check{B}=\{-b:b\in B\}. $$ Many morphology formulas use reflection. If $B$ is symmetric, then $B=\check{B}$. ## 9.3 Dilation and erosion The two fundamental operations are **dilation** and **erosion**. ### Dilation The dilation of a set $A$ by a structuring element $B$ is $$ A\oplus B=\{a+b:a\in A,\ b\in B\}. $$ Equivalently, $$ A\oplus B=\{z: (\check{B})_z\cap A\neq \emptyset\}. $$ Dilation expands the foreground. A pixel $z$ is included in $A\oplus B$ if the structuring element placed at $z$ touches the original object. ### Erosion The erosion of $A$ by $B$ is $$ A\ominus B=\{z:B_z\subseteq A\}. $$ Erosion shrinks the foreground. A pixel $z$ remains if the entire structuring element placed at $z$ fits inside the object. ### Intuition | Operation | Local rule | Effect | |---|---|---| | Dilation | include if neighborhood intersects foreground | expands objects, fills small gaps | | Erosion | include if neighborhood lies inside foreground | shrinks objects, removes thin parts | The two operations are not inverses. Erosion can delete small objects completely; dilation cannot recover them without additional information. ### Dilation as Minkowski addition Dilation is exactly a Minkowski sum: $$ A\oplus B=A+B. $$ This connects morphology to convex geometry and mathematical analysis. If $A$ is a continuous planar set and $B_r$ is a disk of radius $r$, then $$ A\oplus B_r=\{x\in\mathbb{R}^2: d(x,A)\le r\}, $$ where $$ d(x,A)=\inf_{a\in A}\|x-a\|_2. $$ Thus dilation by a disk is the $r$-neighborhood of a set. ### Erosion as distance from the complement For a disk $B_r$, $$ A\ominus B_r=\{x\in A: d(x,A^c)>r\}. $$ So erosion keeps points that are at least distance $r$ away from the background. ::: {.callout-note} ## Statistical interpretation If a predicted segmentation mask has boundary uncertainty of roughly $r$ pixels, then dilation and erosion by a radius-$r$ structuring element produce outer and inner uncertainty bands. This is useful in medical imaging when the exact boundary is uncertain. ::: ## 9.4 Interactive: dilation and erosion on a binary grid The following interactive shows how dilation and erosion act on a synthetic binary object. Increase the radius to see how the set expands or shrinks. ```{=html} <div class="interactive-card"> <h3>Dilation and erosion on a binary grid</h3> <p>Choose a morphological operation and a square structuring-element radius.</p> <label for="ch9-op">Operation:</label> <select id="ch9-op"> <option value="original">Original</option> <option value="dilation">Dilation</option> <option value="erosion">Erosion</option> <option value="boundary">Boundary = A - erosion(A)</option> </select> <label for="ch9-radius" style="margin-left:1rem;">radius = <span id="ch9-radius-value">1</span></label> <input id="ch9-radius" type="range" min="1" max="4" step="1" value="1" style="width:220px;"> <div id="ch9-morphology-demo" class="plotly-demo"></div> </div> <script> (function(){ function makeObject(n){ var A = []; for(var i=0;i<n;i++){ var row=[]; for(var j=0;j<n;j++){ var x = j-(n-1)/2; var y = i-(n-1)/2; var ellipse = (x*x/42 + y*y/20) <= 1; var notch = (x > 0 && x < 5 && y > -3 && y < 4); var dot = (Math.pow(x+6,2)+Math.pow(y-6,2) <= 2.4); row.push((ellipse && !notch) || dot ? 1 : 0); } A.push(row); } return A; } function neighborhood(r){ var B=[]; for(var a=-r;a<=r;a++){ for(var b=-r;b<=r;b++){ B.push([a,b]); } } return B; } function dilate(A,r){ var n=A.length, m=A[0].length, B=neighborhood(r); var C=Array.from({length:n},()=>Array(m).fill(0)); for(var i=0;i<n;i++) for(var j=0;j<m;j++){ if(A[i][j]===1){ for(var k=0;k<B.length;k++){ var ii=i+B[k][0], jj=j+B[k][1]; if(ii>=0 && ii<n && jj>=0 && jj<m) C[ii][jj]=1; } } } return C; } function erode(A,r){ var n=A.length, m=A[0].length, B=neighborhood(r); var C=Array.from({length:n},()=>Array(m).fill(0)); for(var i=0;i<n;i++) for(var j=0;j<m;j++){ var ok=true; for(var k=0;k<B.length;k++){ var ii=i+B[k][0], jj=j+B[k][1]; if(ii<0 || ii>=n || jj<0 || jj>=m || A[ii][jj]===0){ ok=false; break; } } C[i][j]=ok?1:0; } return C; } function subtract(A,E){ return A.map((row,i)=>row.map((v,j)=>Math.max(0,v-E[i][j]))); } function update(){ var n=31; var A=makeObject(n); var r=parseInt(document.getElementById('ch9-radius').value); var op=document.getElementById('ch9-op').value; var Z=A; if(op==='dilation') Z=dilate(A,r); if(op==='erosion') Z=erode(A,r); if(op==='boundary') Z=subtract(A,erode(A,r)); document.getElementById('ch9-radius-value').textContent=r; Plotly.newPlot('ch9-morphology-demo', [{z:Z, type:'heatmap', showscale:false, zmin:0, zmax:1}], { margin:{l:40,r:20,t:20,b:40}, xaxis:{title:'column j', constrain:'domain'}, yaxis:{title:'row i', scaleanchor:'x', autorange:'reversed'}, height:430 }, {responsive:true}); } document.getElementById('ch9-op').addEventListener('change', update); document.getElementById('ch9-radius').addEventListener('input', update); update(); })(); </script> ``` ## 9.5 Duality, monotonicity, and algebraic properties Morphological operations have clean algebraic structure. Let $A,C\subseteq \Omega$ and let $B$ be a structuring element. ### Monotonicity If $A\subseteq C$, then $$ A\oplus B\subseteq C\oplus B $$ and $$ A\ominus B\subseteq C\ominus B. $$ **Proof.** If $z\in A\oplus B$, then $z=a+b$ for some $a\in A$ and $b\in B$. Since $A\subseteq C$, $a\in C$, so $z\in C\oplus B$. For erosion, if $z\in A\ominus B$, then $B_z\subseteq A\subseteq C$, hence $z\in C\ominus B$. $\square$ ### Duality Dilation and erosion are dual under complementation: $$ (A\oplus B)^c=A^c\ominus \check{B}, $$ and $$ (A\ominus B)^c=A^c\oplus \check{B}. $$ For symmetric $B$, this simplifies to $$ (A\oplus B)^c=A^c\ominus B, \qquad (A\ominus B)^c=A^c\oplus B. $$ This explains why dilation of the foreground is equivalent to erosion of the background. ### Translation equivariance For a shift $t\in\mathbb{Z}^2$, define $$ A+t=\{a+t:a\in A\}. $$ Then $$ (A+t)\oplus B=(A\oplus B)+t, $$ and $$ (A+t)\ominus B=(A\ominus B)+t. $$ This resembles the translation equivariance of convolution. In fact, dilation and erosion can be viewed as nonlinear translation-equivariant local operators. ## 9.6 Opening and closing Two important composite operations are **opening** and **closing**. ### Opening The opening of $A$ by $B$ is $$ A\circ B=(A\ominus B)\oplus B. $$ Opening erodes first and then dilates. It removes small foreground components and thin protrusions while preserving larger shapes. ### Closing The closing of $A$ by $B$ is $$ A\bullet B=(A\oplus B)\ominus B. $$ Closing dilates first and then erodes. It fills small background holes and narrow gaps. ### Properties Opening is: 1. **anti-extensive**: $A\circ B\subseteq A$; 2. **monotone**: if $A\subseteq C$, then $A\circ B\subseteq C\circ B$; 3. **idempotent**: $(A\circ B)\circ B=A\circ B$. Closing is: 1. **extensive**: $A\subseteq A\bullet B$; 2. **monotone**: if $A\subseteq C$, then $A\bullet B\subseteq C\bullet B$; 3. **idempotent**: $(A\bullet B)\bullet B=A\bullet B$. Idempotence means that applying the operation twice gives no further change. This is a hallmark of a shape simplification operator. ### Opening as a shape filter Opening can be interpreted as keeping only those parts of $A$ into which a translated copy of $B$ can fit. Formally, $$ A\circ B=\bigcup_{z:B_z\subseteq A} B_z. $$ Thus opening approximates $A$ by a union of copies of the structuring element. ::: {.callout-tip} ## Practical rule Use opening to remove small false-positive islands. Use closing to fill small false-negative holes. Use both carefully: too large a structuring element can remove real anatomical or geometric details. ::: ## 9.7 Interactive: opening and closing ```{=html} <div class="interactive-card"> <h3>Opening and closing as shape filters</h3> <p>Opening removes small bright structures; closing fills small dark holes and gaps.</p> <label for="ch9-openclose-op">Operation:</label> <select id="ch9-openclose-op"> <option value="original">Original noisy mask</option> <option value="opening">Opening</option> <option value="closing">Closing</option> <option value="openclose">Opening then closing</option> </select> <label for="ch9-openclose-radius" style="margin-left:1rem;">radius = <span id="ch9-openclose-radius-value">1</span></label> <input id="ch9-openclose-radius" type="range" min="1" max="3" step="1" value="1" style="width:220px;"> <div id="ch9-openclose-demo" class="plotly-demo"></div> </div> <script> (function(){ function noisyMask(n){ var A=[]; for(var i=0;i<n;i++){ var row=[]; for(var j=0;j<n;j++){ var x=j-(n-1)/2, y=i-(n-1)/2; var main=(x*x/50+y*y/32)<=1; var hole=(Math.pow(x-2,2)+Math.pow(y+1,2)<=5); var gap=(x>-2 && x<3 && y>0 && y<3); var speck1=(Math.pow(x+10,2)+Math.pow(y-7,2)<=2); var speck2=(Math.pow(x-11,2)+Math.pow(y+9,2)<=2); var speck3=(Math.pow(x+7,2)+Math.pow(y+10,2)<=1.5); row.push(((main && !hole && !gap) || speck1 || speck2 || speck3) ? 1 : 0); } A.push(row); } return A; } function B(r){ var pts=[]; for(var a=-r;a<=r;a++) for(var b=-r;b<=r;b++) pts.push([a,b]); return pts; } function dilate(A,r){ var n=A.length,m=A[0].length,pts=B(r),C=Array.from({length:n},()=>Array(m).fill(0)); for(var i=0;i<n;i++) for(var j=0;j<m;j++) if(A[i][j]){ for(var k=0;k<pts.length;k++){ var ii=i+pts[k][0], jj=j+pts[k][1]; if(ii>=0&&ii<n&&jj>=0&&jj<m) C[ii][jj]=1; } } return C; } function erode(A,r){ var n=A.length,m=A[0].length,pts=B(r),C=Array.from({length:n},()=>Array(m).fill(0)); for(var i=0;i<n;i++) for(var j=0;j<m;j++){ var ok=true; for(var k=0;k<pts.length;k++){ var ii=i+pts[k][0], jj=j+pts[k][1]; if(ii<0||ii>=n||jj<0||jj>=m||!A[ii][jj]){ok=false;break;} } C[i][j]=ok?1:0; } return C; } function opening(A,r){return dilate(erode(A,r),r);} function closing(A,r){return erode(dilate(A,r),r);} function update(){ var A=noisyMask(35); var r=parseInt(document.getElementById('ch9-openclose-radius').value); var op=document.getElementById('ch9-openclose-op').value; var Z=A; if(op==='opening') Z=opening(A,r); if(op==='closing') Z=closing(A,r); if(op==='openclose') Z=closing(opening(A,r),r); document.getElementById('ch9-openclose-radius-value').textContent=r; Plotly.newPlot('ch9-openclose-demo',[{z:Z,type:'heatmap',showscale:false,zmin:0,zmax:1}],{ margin:{l:40,r:20,t:20,b:40}, xaxis:{title:'column j'}, yaxis:{title:'row i', scaleanchor:'x', autorange:'reversed'}, height:430 },{responsive:true}); } document.getElementById('ch9-openclose-op').addEventListener('change',update); document.getElementById('ch9-openclose-radius').addEventListener('input',update); update(); })(); </script> ``` ## 9.8 Boundary extraction and morphological gradients The **internal boundary** of $A$ with respect to $B$ is $$ \partial_B A=A\setminus(A\ominus B). $$ It consists of foreground pixels that do not survive erosion. The **external boundary** can be written $$ \partial_B^{\text{ext}} A=(A\oplus B)\setminus A. $$ The **morphological gradient** is $$ G_B(A)=(A\oplus B)\setminus(A\ominus B). $$ For grayscale images, a morphological gradient is often defined as $$ G_B f(x)=(f\oplus B)(x)-(f\ominus B)(x), $$ where grayscale dilation and erosion are introduced below. This acts like an edge detector but uses local maxima and minima rather than local derivatives. ## 9.9 Grayscale morphology For a grayscale image $f:\Omega_h\to\mathbb{R}$ and a flat structuring element $B\subseteq \mathbb{Z}^2$, define grayscale dilation by $$ (f\oplus B)(x)=\max_{b\in B} f(x-b), $$ and grayscale erosion by $$ (f\ominus B)(x)=\min_{b\in B} f(x+b). $$ These are max and min filters. They are nonlinear analogues of convolution. ### Non-flat structuring elements A non-flat structuring element is a function $g:B\to\mathbb{R}$. Then dilation can be defined as $$ (f\oplus g)(x)=\max_{b\in B}\{f(x-b)+g(b)\}, $$ and erosion as $$ (f\ominus g)(x)=\min_{b\in B}\{f(x+b)-g(b)\}. $$ This places morphology in **max-plus algebra** and **min-plus algebra**. These algebraic systems replace ordinary addition and multiplication with operations such as maximum plus addition. They appear in optimization, dynamic programming, and shortest-path problems. ### Relation to pooling in neural networks Max pooling in CNNs is a grayscale dilation over a local window followed by subsampling. Min pooling can be implemented as $$ \min_{b\in B} f(x+b)=-\max_{b\in B}\{-f(x+b)\}. $$ Thus erosion is negative max pooling applied to the negative image. ## 9.10 Connected components and adjacency graphs A binary image $A\subseteq\Omega_h$ can be turned into a graph. Let $$ G_A=(V_A,E_A), $$ where $$ V_A=A $$ and where two foreground pixels are connected by an edge if they are neighbors. With 4-connectivity, $$ (i,j)\sim(i',j') \quad \Longleftrightarrow \quad |i-i'|+|j-j'|=1. $$ With 8-connectivity, $$ (i,j)\sim(i',j') \quad \Longleftrightarrow \quad \max\{|i-i'|,|j-j'|\}=1. $$ A **connected component** is a connected component of this graph. Component labeling is a standard operation in post-processing segmentation masks. ### Why connectivity matters The choice of foreground connectivity affects topology. If diagonal pixels count as connected under 8-connectivity, then two diagonal pixels belong to the same component. Under 4-connectivity, they do not. To avoid paradoxes, one often uses complementary choices: | Foreground connectivity | Background connectivity | |---|---| | 4-connected foreground | 8-connected background | | 8-connected foreground | 4-connected background | This prevents ambiguous diagonal connections. ### Euler characteristic For a planar binary shape, the Euler characteristic is $$ \chi(A)=\#\{\text{connected components}\}-\#\{\text{holes}\}. $$ In topological language, $$ \chi(A)=\beta_0(A)-\beta_1(A), $$ where $\beta_0$ counts connected components and $\beta_1$ counts one-dimensional holes. This is a simple but powerful topological descriptor. In medical image segmentation, for example, one may expect a certain anatomical structure to have one connected component and no holes. A segmentation mask that violates this expectation may need correction. ::: {.callout-note} ## Connection to topological data analysis Morphology modifies the topology of a binary set. Dilation can merge components and fill holes. Erosion can split components and create holes. Persistent homology tracks how connected components and holes change over a scale parameter. ::: ## 9.11 Distance transforms For a foreground set $A\subseteq\Omega_h$, the distance transform assigns to each pixel its distance to the nearest foreground pixel: $$ D_A(x)=\min_{a\in A} d(x,a). $$ Common choices include: $$ d_1(x,a)=|x_1-a_1|+|x_2-a_2|, $$ $$ d_2(x,a)=\sqrt{(x_1-a_1)^2+(x_2-a_2)^2}, $$ and $$ d_\infty(x,a)=\max\{|x_1-a_1|,|x_2-a_2|\}. $$ For points inside the foreground, one often computes distance to the background: $$ D_{A^c}(x)=\min_{y\in A^c}d(x,y),\qquad x\in A. $$ This value is large near the center of thick regions and small near the boundary. ### Signed distance function A signed distance function is $$ \phi_A(x)= \begin{cases} -d(x,A^c), & x\in A,\\ d(x,A), & x\notin A. \end{cases} $$ The boundary is represented by the zero level set: $$ \partial A=\{x:\phi_A(x)=0\}. $$ Signed distance functions connect morphology to level-set methods, shape optimization, and PDE-based vision. ## 9.12 Skeletons and medial axes The **medial axis** of a shape is the set of centers of maximal balls contained in the shape. Informally, it is the set of points that have at least two closest boundary points. For a continuous shape $A\subset\mathbb{R}^2$, a disk $B(x,r)$ is **maximal in $A$** if $$ B(x,r)\subseteq A $$ and it is not strictly contained in a larger disk contained in $A$. The medial axis consists of the centers of these maximal disks. A skeleton should preserve the main topology and geometry of the shape while reducing it to a thin centerline. This is useful for roads, vessels, plant roots, handwriting, and shape matching. ### Morphological skeleton One classical morphological skeleton is defined using erosions and openings: $$ S_k(A)=(A\ominus kB)\setminus\big((A\ominus kB)\circ B\big), $$ where $kB$ denotes repeated dilation of $B$ with itself. The full skeleton is $$ S(A)=\bigcup_{k\ge 0} S_k(A). $$ The original shape can be approximately reconstructed from skeleton layers by $$ A=\bigcup_{k\ge 0} \left(S_k(A)\oplus kB\right), $$ under suitable discrete assumptions. ## 9.13 Interactive: distance transform and skeleton intuition ```{=html} <div class="interactive-card"> <h3>Distance transform and skeleton intuition</h3> <p>The skeleton is approximated by ridges of the distance-to-boundary function.</p> <label for="ch9-distance-view">View:</label> <select id="ch9-distance-view"> <option value="mask">Binary mask</option> <option value="distance">Distance to boundary</option> <option value="skeleton">Approximate skeleton ridge</option> </select> <div id="ch9-distance-demo" class="plotly-demo"></div> </div> <script> (function(){ function mask(n){ var A=[]; for(var i=0;i<n;i++){ var row=[]; for(var j=0;j<n;j++){ var x=j-(n-1)/2, y=i-(n-1)/2; var body=(x*x/70+y*y/25)<=1; var arm=(Math.abs(y-0.45*x)<2.4 && x>-12 && x<14); var cut=(x>6 && x<10 && y>-3 && y<6); row.push((body || arm) && !cut ? 1 : 0); } A.push(row); } return A; } function distToBoundary(A){ var n=A.length,m=A[0].length,D=Array.from({length:n},()=>Array(m).fill(0)); var boundary=[]; var dirs=[[1,0],[-1,0],[0,1],[0,-1]]; for(var i=0;i<n;i++) for(var j=0;j<m;j++) if(A[i][j]){ var isB=false; for(var d=0;d<dirs.length;d++){ var ii=i+dirs[d][0], jj=j+dirs[d][1]; if(ii<0||ii>=n||jj<0||jj>=m||!A[ii][jj]){isB=true;break;} } if(isB) boundary.push([i,j]); } for(var i=0;i<n;i++) for(var j=0;j<m;j++) if(A[i][j]){ var best=1e9; for(var k=0;k<boundary.length;k++){ var di=i-boundary[k][0], dj=j-boundary[k][1]; var dd=Math.sqrt(di*di+dj*dj); if(dd<best) best=dd; } D[i][j]=best; } return D; } function skeleton(D,A){ var n=A.length,m=A[0].length,S=Array.from({length:n},()=>Array(m).fill(0)); for(var i=1;i<n-1;i++) for(var j=1;j<m-1;j++) if(A[i][j]){ var v=D[i][j]; if(v<1.5) continue; var maxN=true; for(var a=-1;a<=1;a++) for(var b=-1;b<=1;b++){ if(a===0&&b===0) continue; if(D[i+a][j+b]>v+0.05) maxN=false; } S[i][j]=maxN?1:0; } return S; } function update(){ var A=mask(45), D=distToBoundary(A), S=skeleton(D,A); var view=document.getElementById('ch9-distance-view').value; var Z=A, colorscale=false, zmax=1; if(view==='distance'){ Z=D; colorscale='Viridis'; zmax=null; } if(view==='skeleton'){ Z=S; colorscale=false; zmax=1; } Plotly.newPlot('ch9-distance-demo',[{z:Z,type:'heatmap',showscale:view==='distance',zmin:0,zmax:zmax,colorscale:colorscale}],{ margin:{l:40,r:20,t:20,b:40}, xaxis:{title:'column j'}, yaxis:{title:'row i',scaleanchor:'x',autorange:'reversed'}, height:430 },{responsive:true}); } document.getElementById('ch9-distance-view').addEventListener('change',update); update(); })(); </script> ``` ## 9.14 Shape descriptors A shape descriptor maps a set $A$ to a numerical feature vector: $$ \Psi(A)\in\mathbb{R}^d. $$ Good descriptors should be stable, interpretable, and relevant to the task. Some descriptors should be invariant to translation, rotation, or scale; others should preserve orientation and size. ### Area and perimeter The area of a binary shape is the number of foreground pixels: $$ |A|=\sum_{(i,j)\in\Omega_h}\mathbf{1}_A(i,j). $$ Perimeter can be approximated by counting boundary pixels, or more accurately by using contour length estimates. A simple boundary-count estimate is $$ P_B(A)=|A\setminus(A\ominus B)|. $$ ### Centroid The centroid is $$ \bar{x}=\frac{1}{|A|}\sum_{x\in A}x. $$ Writing $x=(x_1,x_2)^T$, this gives $$ \bar{x}=\begin{pmatrix}\bar{x}_1\\ \bar{x}_2\end{pmatrix}. $$ The centroid is translation equivariant: shifting the shape shifts the centroid by the same amount. ### Covariance matrix of a shape The shape covariance matrix is $$ \Sigma_A=\frac{1}{|A|}\sum_{x\in A}(x-\bar{x})(x-\bar{x})^T. $$ The eigenvectors of $\Sigma_A$ give principal axes. The eigenvalues describe spread along those axes. If $$ \lambda_1\ge \lambda_2\ge 0, $$ then an eccentricity-type descriptor is $$ e(A)=\sqrt{1-\frac{\lambda_2}{\lambda_1}}. $$ This is close to 0 for round shapes and close to 1 for elongated shapes. ### Moment descriptors Raw moments are $$ m_{pq}=\sum_{(x,y)\in A} x^p y^q. $$ Central moments are $$ \mu_{pq}=\sum_{(x,y)\in A}(x-\bar{x})^p(y-\bar{y})^q. $$ Normalized central moments are $$ \eta_{pq}=\frac{\mu_{pq}}{\mu_{00}^{1+(p+q)/2}}. $$ Moment invariants combine normalized central moments to obtain descriptors invariant under translation, scaling, and sometimes rotation. These are useful for classical shape recognition but can be unstable for noisy boundaries. ### Compactness and circularity A common compactness measure is $$ C(A)=\frac{4\pi |A|}{P(A)^2}. $$ For an ideal disk, $C(A)=1$. For elongated or irregular shapes, $C(A)$ is smaller. ::: {.callout-warning} ## Discrete caution Pixel-based perimeter estimates can be biased by grid orientation. For statistical analysis, use consistent definitions across the dataset, and be careful when comparing shapes at different resolutions. ::: ## 9.15 Interactive: moments and principal axes ```{=html} <div class="interactive-card"> <h3>Shape covariance and principal axes</h3> <p>Change the elongation and rotation of a synthetic ellipse. The covariance eigenvectors estimate the principal axes.</p> <label for="ch9-angle">angle = <span id="ch9-angle-value">30</span> degrees</label> <input id="ch9-angle" type="range" min="0" max="170" step="10" value="30" style="width:220px;"> <label for="ch9-ecc" style="margin-left:1rem;">elongation = <span id="ch9-ecc-value">2.0</span></label> <input id="ch9-ecc" type="range" min="1" max="4" step="0.25" value="2" style="width:220px;"> <div id="ch9-moments-demo" class="plotly-demo"></div> </div> <script> (function(){ function points(angleDeg, elong){ var theta=angleDeg*Math.PI/180; var c=Math.cos(theta), s=Math.sin(theta); var ptsX=[], ptsY=[]; var a=6*elong, b=6; for(var i=-22;i<=22;i++){ for(var j=-22;j<=22;j++){ var xr=c*j+s*i; var yr=-s*j+c*i; if((xr*xr)/(a*a)+(yr*yr)/(b*b)<=1){ ptsX.push(j); ptsY.push(i); } } } return [ptsX,ptsY]; } function covAndMean(xs,ys){ var n=xs.length, mx=0,my=0; for(var k=0;k<n;k++){mx+=xs[k]; my+=ys[k];} mx/=n; my/=n; var a=0,b=0,d=0; for(var k=0;k<n;k++){ var x=xs[k]-mx,y=ys[k]-my; a+=x*x; b+=x*y; d+=y*y; } a/=n; b/=n; d/=n; var tr=a+d, det=a*d-b*b; var disc=Math.sqrt(Math.max(0,tr*tr/4-det)); var l1=tr/2+disc, l2=tr/2-disc; var v1x=b, v1y=l1-a; if(Math.abs(v1x)+Math.abs(v1y)<1e-8){v1x=1;v1y=0;} var norm=Math.sqrt(v1x*v1x+v1y*v1y); v1x/=norm; v1y/=norm; var v2x=-v1y, v2y=v1x; return {mx:mx,my:my,l1:l1,l2:l2,v1x:v1x,v1y:v1y,v2x:v2x,v2y:v2y}; } function update(){ var angle=parseFloat(document.getElementById('ch9-angle').value); var elong=parseFloat(document.getElementById('ch9-ecc').value); document.getElementById('ch9-angle-value').textContent=angle; document.getElementById('ch9-ecc-value').textContent=elong.toFixed(2); var p=points(angle,elong), xs=p[0], ys=p[1], st=covAndMean(xs,ys); var scale1=Math.sqrt(st.l1)*2.2, scale2=Math.sqrt(st.l2)*2.2; var axis1={x:[st.mx-scale1*st.v1x, st.mx+scale1*st.v1x], y:[st.my-scale1*st.v1y, st.my+scale1*st.v1y], mode:'lines', name:'first principal axis', line:{width:4}}; var axis2={x:[st.mx-scale2*st.v2x, st.mx+scale2*st.v2x], y:[st.my-scale2*st.v2y, st.my+scale2*st.v2y], mode:'lines', name:'second principal axis', line:{width:4, dash:'dot'}}; var centroid={x:[st.mx],y:[st.my],mode:'markers',name:'centroid',marker:{size:10}}; var cloud={x:xs,y:ys,mode:'markers',type:'scatter',name:'foreground pixels',marker:{size:5}}; Plotly.newPlot('ch9-moments-demo',[cloud,centroid,axis1,axis2],{ margin:{l:40,r:20,t:20,b:40}, xaxis:{title:'x',range:[-24,24]}, yaxis:{title:'y',range:[-24,24],scaleanchor:'x'}, height:430, showlegend:true },{responsive:true}); } document.getElementById('ch9-angle').addEventListener('input',update); document.getElementById('ch9-ecc').addEventListener('input',update); update(); })(); </script> ``` ## 9.16 Fourier descriptors for shape contours Suppose the boundary of a planar shape is sampled as an ordered sequence of points $$ z_k=x_k+iy_k,\qquad k=0,1,\ldots,N-1. $$ The discrete Fourier transform of the contour is $$ \widehat{z}_\ell=\sum_{k=0}^{N-1} z_k\exp\left(-2\pi i k\ell/N\right). $$ Low-frequency coefficients describe the coarse shape, while high-frequency coefficients describe fine boundary details. A smoothed contour can be reconstructed from a truncated Fourier series: $$ z_k^{(K)}=\frac{1}{N}\sum_{|\ell|\le K}\widehat{z}_\ell\exp\left(2\pi i k\ell/N\right). $$ Fourier descriptors make shape analysis look like signal processing on closed curves. Translation is controlled by the zero-frequency coefficient, scaling by coefficient magnitudes, and rotation by phase. ## 9.17 Interactive: Fourier shape reconstruction ```{=html} <div class="interactive-card"> <h3>Fourier descriptors of a closed contour</h3> <p>Low frequencies preserve the coarse shape. Higher frequencies add boundary details.</p> <label for="ch9-fourier-k">number of frequency pairs K = <span id="ch9-fourier-k-value">3</span></label> <input id="ch9-fourier-k" type="range" min="1" max="20" step="1" value="3" style="width:260px;"> <div id="ch9-fourier-demo" class="plotly-demo"></div> </div> <script> (function(){ function contour(N){ var xs=[],ys=[]; for(var k=0;k<N;k++){ var t=2*Math.PI*k/N; var r=1+0.22*Math.cos(3*t)+0.12*Math.sin(5*t)+0.08*Math.cos(9*t); xs.push(r*Math.cos(t)); ys.push(0.78*r*Math.sin(t)); } return [xs,ys]; } function dftComplex(xs,ys){ var N=xs.length, Ar=[],Ai=[]; for(var l=0;l<N;l++){ var sr=0,si=0; for(var k=0;k<N;k++){ var ang=-2*Math.PI*k*l/N; var cr=Math.cos(ang), ci=Math.sin(ang); var zr=xs[k], zi=ys[k]; sr += zr*cr - zi*ci; si += zr*ci + zi*cr; } Ar.push(sr); Ai.push(si); } return [Ar,Ai]; } function reconstruct(Ar,Ai,K,N){ var xs=[],ys=[]; for(var k=0;k<N;k++){ var zr=0,zi=0; var inds=[0]; for(var h=1;h<=K;h++){inds.push(h); inds.push(N-h);} for(var q=0;q<inds.length;q++){ var l=inds[q]; var ang=2*Math.PI*k*l/N; var cr=Math.cos(ang), ci=Math.sin(ang); zr += Ar[l]*cr - Ai[l]*ci; zi += Ar[l]*ci + Ai[l]*cr; } xs.push(zr/N); ys.push(zi/N); } xs.push(xs[0]); ys.push(ys[0]); return [xs,ys]; } function update(){ var N=128; var p=contour(N), xs=p[0], ys=p[1]; var F=dftComplex(xs,ys), K=parseInt(document.getElementById('ch9-fourier-k').value); document.getElementById('ch9-fourier-k-value').textContent=K; var rec=reconstruct(F[0],F[1],K,N); var xclosed=xs.slice(); xclosed.push(xs[0]); var yclosed=ys.slice(); yclosed.push(ys[0]); Plotly.newPlot('ch9-fourier-demo',[ {x:xclosed,y:yclosed,mode:'lines',name:'original contour'}, {x:rec[0],y:rec[1],mode:'lines',name:'Fourier reconstruction K='+K,line:{width:4}} ],{ margin:{l:40,r:20,t:20,b:40}, xaxis:{title:'x',range:[-1.6,1.6]}, yaxis:{title:'y',range:[-1.3,1.3],scaleanchor:'x'}, height:430 },{responsive:true}); } document.getElementById('ch9-fourier-k').addEventListener('input',update); update(); })(); </script> ``` ## 9.18 Granulometry and morphological scale In many applications, object size is itself a feature. A **granulometry** studies how the area of a shape changes under openings of increasing size. Let $B_r$ be a structuring element of scale $r$. Define $$ G_A(r)=|A\circ B_r|. $$ As $r$ increases, small structures disappear. The function $G_A(r)$ is nonincreasing. A discrete size distribution can be defined by differences: $$ \Delta G_A(r)=G_A(r)-G_A(r+1). $$ Large drops indicate that many structures of size approximately $r$ were removed. This is useful for particle analysis, materials imaging, cell morphology, and remote sensing. ## 9.19 Statistical viewpoint: shapes as estimators In statistical vision, the true object $A^*$ is unknown. A model produces an estimated mask $\widehat{A}$. Shape descriptors become statistics: $$ T(\widehat{A})=\Psi(\widehat{A}). $$ Examples include area, perimeter, eccentricity, number of components, and compactness. ### Bias from morphology Morphological operations can introduce bias. If $A$ is dilated by a disk of radius $r$, its area generally increases. For sufficiently regular planar sets, the continuous Steiner formula gives $$ |A\oplus B_r|=|A|+rP(A)+\pi r^2, $$ for convex $A$. This formula shows that dilation changes area in a predictable way involving perimeter. For erosion, $$ |A\ominus B_r|\approx |A|-rP(A)+\pi r^2, $$ under regularity assumptions and for small $r$. These formulas are useful when interpreting morphological preprocessing statistically. ### Uncertainty in shape descriptors Suppose a segmentation model outputs random masks $A^{(1)},\ldots,A^{(M)}$, for example from test-time augmentation, Monte Carlo dropout, or posterior sampling. A shape descriptor $\Psi$ then has an empirical distribution: $$ \Psi(A^{(1)}),\ldots,\Psi(A^{(M)}). $$ We can estimate uncertainty using $$ \widehat{\operatorname{Var}}[\Psi] =\frac{1}{M-1}\sum_{m=1}^M \left(\Psi(A^{(m)})-\overline{\Psi}\right)^2. $$ This is important when a downstream scientific conclusion depends on a shape measurement. ## 9.20 AI components: morphology in modern vision systems Classical morphology remains important in AI-based computer vision. ### 1. Post-processing segmentation masks A neural network often produces a probability map $$ p_\theta(x)=P_\theta(Y_x=1\mid I). $$ A common pipeline is: $$ I \longrightarrow p_\theta \longrightarrow \widehat{A}_\tau \longrightarrow \text{morphological cleanup} \longrightarrow \text{component filtering}. $$ For example: 1. threshold the probability map; 2. remove small connected components; 3. close small holes; 4. compute area, perimeter, and shape descriptors; 5. flag masks with unusual topology. ### 2. Differentiable morphology Grayscale dilation and erosion can be written using max and min operations. A differentiable approximation to maximum is the log-sum-exp function: $$ \max_i a_i \approx \frac{1}{\alpha}\log\sum_i \exp(\alpha a_i), $$ where large $\alpha$ gives a closer approximation. This permits differentiable morphology layers inside neural networks. ### 3. Max pooling as morphology Max pooling is a local dilation: $$ (\operatorname{MaxPool}_B f)(x)=\max_{b\in B} f(x+b). $$ This creates a direct bridge between morphology and CNNs. ### 4. Topological and shape-aware losses For segmentation, pixelwise loss may not be enough. A mask can have high pixel accuracy but incorrect topology. Shape-aware training may include penalties for: - incorrect number of connected components; - holes that should not exist; - disconnected vessels or roads; - unrealistic area/perimeter ratios; - deviation from expected shape priors. ### 5. Foundation-model masks still need shape analysis Promptable segmentation systems can produce high-quality masks, but the outputs are still sets. For scientific and medical use, one must measure, validate, and summarize these sets. Morphology and shape descriptors remain essential. ## 9.21 Python example: binary morphology The following code illustrates basic morphology using `scipy.ndimage`. The code is marked `eval: false` so that the book does not require SciPy during rendering. ```{python} #| label: py-ch9-binary-morphology #| eval: false import numpy as np import matplotlib.pyplot as plt from scipy import ndimage as ndi n = 80 y, x = np.mgrid[-1:1:complex(n), -1:1:complex(n)] mask = (x**2 / 0.45**2 + y**2 / 0.30**2) <= 1 mask[(x > 0.05) & (x < 0.25) & (y > -0.1) & (y < 0.15)] = False mask[(x + 0.65)**2 + (y - 0.55)**2 < 0.04**2] = True structure = np.ones((5, 5), dtype=bool) dilated = ndi.binary_dilation(mask, structure=structure) eroded = ndi.binary_erosion(mask, structure=structure) opened = ndi.binary_opening(mask, structure=structure) closed = ndi.binary_closing(mask, structure=structure) fig, axes = plt.subplots(1, 5, figsize=(12, 3)) for ax, img, title in zip( axes, [mask, dilated, eroded, opened, closed], ["original", "dilation", "erosion", "opening", "closing"] ): ax.imshow(img, cmap="gray") ax.set_title(title) ax.axis("off") plt.tight_layout() plt.show() ``` ## 9.22 Python example: connected components and shape features ```{python} #| label: py-ch9-components-shape-features #| eval: false import numpy as np from scipy import ndimage as ndi # Suppose mask is a binary segmentation result. labeled, num_components = ndi.label(mask) component_sizes = ndi.sum(mask, labeled, index=np.arange(1, num_components + 1)) print("number of connected components:", num_components) print("component sizes:", component_sizes) # Keep only components with at least 50 pixels. keep_labels = np.where(component_sizes >= 50)[0] + 1 clean = np.isin(labeled, keep_labels) # Simple centroid and covariance matrix of the largest component. largest_label = int(np.argmax(component_sizes) + 1) coords = np.column_stack(np.nonzero(labeled == largest_label)) centroid = coords.mean(axis=0) centered = coords - centroid cov = centered.T @ centered / coords.shape[0] evals, evecs = np.linalg.eigh(cov) print("centroid:", centroid) print("covariance matrix:\n", cov) print("principal variances:", evals[::-1]) print("principal directions:\n", evecs[:, ::-1]) ``` ## 9.23 Python example: distance transform and skeleton ```{python} #| label: py-ch9-distance-transform-skeleton #| eval: false import numpy as np import matplotlib.pyplot as plt from scipy import ndimage as ndi from skimage.morphology import medial_axis # Distance from foreground pixels to background. dist = ndi.distance_transform_edt(mask) # Skeleton from the medial axis. skeleton, dist_on_skeleton = medial_axis(mask, return_distance=True) fig, axes = plt.subplots(1, 3, figsize=(10, 3)) axes[0].imshow(mask, cmap="gray") axes[0].set_title("mask") axes[1].imshow(dist, cmap="viridis") axes[1].set_title("distance transform") axes[2].imshow(skeleton, cmap="gray") axes[2].set_title("skeleton") for ax in axes: ax.axis("off") plt.tight_layout() plt.show() ``` ## 9.24 Python example: grayscale morphology ```{python} #| label: py-ch9-grayscale-morphology #| eval: false import numpy as np import matplotlib.pyplot as plt from scipy import ndimage as ndi rng = np.random.default_rng(7243) n = 100 y, x = np.mgrid[-1:1:complex(n), -1:1:complex(n)] image = np.exp(-((x + 0.25)**2 + (y + 0.1)**2) / 0.08) image += 0.7 * np.exp(-((x - 0.35)**2 + (y - 0.25)**2) / 0.02) image += 0.08 * rng.normal(size=(n, n)) size = 7 gray_dilation = ndi.maximum_filter(image, size=size) gray_erosion = ndi.minimum_filter(image, size=size) gray_gradient = gray_dilation - gray_erosion fig, axes = plt.subplots(1, 4, figsize=(12, 3)) for ax, img, title in zip( axes, [image, gray_dilation, gray_erosion, gray_gradient], ["image", "dilation/max filter", "erosion/min filter", "morphological gradient"] ): ax.imshow(img, cmap="gray") ax.set_title(title) ax.axis("off") plt.tight_layout() plt.show() ``` ## 9.25 TensorFlow example: morphology using pooling The next example implements grayscale dilation and erosion using max pooling. This is useful because it connects morphology directly to neural-network layers. ```{python} #| label: py-ch9-tensorflow-morphology #| eval: false import tensorflow as tf def dilation2d(x, ksize=3): """Flat grayscale dilation for NHWC tensors.""" return tf.nn.max_pool2d( x, ksize=ksize, strides=1, padding="SAME" ) def erosion2d(x, ksize=3): """Flat grayscale erosion using negative max pooling.""" return -tf.nn.max_pool2d( -x, ksize=ksize, strides=1, padding="SAME" ) def opening2d(x, ksize=3): return dilation2d(erosion2d(x, ksize), ksize) def closing2d(x, ksize=3): return erosion2d(dilation2d(x, ksize), ksize) # Example tensor: batch size 1, height 64, width 64, one channel. x = tf.random.uniform((1, 64, 64, 1)) y_open = opening2d(x, ksize=5) y_close = closing2d(x, ksize=5) print(y_open.shape, y_close.shape) ``` ## 9.26 AI-assisted study prompts Use an AI assistant as a mathematical critic, not as a replacement for derivation. ### Prompt 1: definitions > Explain dilation, erosion, opening, and closing using both set notation and indicator-function notation. Give one example where opening is useful and one example where closing is useful. ### Prompt 2: proof checking > I claim that opening is idempotent. Write a proof using the definition $A\circ B=(A\ominus B)\oplus B$. Identify any assumptions about the structuring element. ### Prompt 3: segmentation post-processing > A neural network outputs a noisy probability map for a tumor. Design a mask post-processing pipeline using thresholding, opening, closing, connected components, and shape descriptors. Explain the statistical risk of each step. ### Prompt 4: coding critique > Review my Python code for connected-component filtering. Check whether it accidentally removes true small objects and suggest a validation strategy. ### Prompt 5: shape descriptors > Compare area, perimeter, compactness, eccentricity, and Euler characteristic as shape features. Which are invariant to translation, rotation, and scaling? ## 9.27 Common mistakes 1. **Confusing dilation with blurring.** Dilation is not averaging. It expands sets or takes local maxima. 2. **Using too large a structuring element.** This can remove real small structures. 3. **Ignoring connectivity choice.** 4-connectivity and 8-connectivity can give different component counts. 4. **Treating post-processing as harmless.** Morphology changes statistical estimators such as area and perimeter. 5. **Measuring topology after resizing without care.** Interpolation and thresholding can create or destroy components and holes. 6. **Using raw perimeter counts across resolutions.** Perimeter estimates depend strongly on pixel size and contour representation. ## 9.28 Exercises ### Conceptual exercises 1. Explain why dilation expands a foreground set but erosion shrinks it. 2. Give an example where opening removes a false positive but also removes a true object. 3. Give an example where closing fills a hole but incorrectly merges two nearby objects. 4. Explain the difference between 4-connectivity and 8-connectivity. 5. Explain why the skeleton of a shape is sensitive to boundary noise. ### Mathematical exercises 1. Prove that dilation is monotone: if $A\subseteq C$, then $A\oplus B\subseteq C\oplus B$. 2. Prove the duality formula $$ (A\oplus B)^c=A^c\ominus \check{B}. $$ 3. Show that opening is anti-extensive: $A\circ B\subseteq A$. 4. Show that closing is extensive: $A\subseteq A\bullet B$. 5. Suppose $B$ is symmetric and contains the origin. Prove that $A\ominus B\subseteq A\subseteq A\oplus B$. 6. For a disk-shaped set $A$ and disk structuring element $B_r$, describe geometrically $A\oplus B_r$ and $A\ominus B_r$. 7. Let $A$ consist of two disks whose centers are distance $d$ apart. For a disk structuring element of radius $r$, determine when dilation causes the two components to merge. 8. Derive the centroid and covariance matrix of an ideal continuous ellipse centered at the origin. ### Python exercises 1. Generate a binary image with three components of different sizes. Use connected-component labeling to remove components below a size threshold. 2. Compare opening with square and disk-like structuring elements. 3. Compute the area, centroid, covariance matrix, and eccentricity of a binary mask. 4. Add random salt-and-pepper noise to a binary image and compare median filtering, opening, and closing. 5. Compute a distance transform and overlay the skeleton on the original image. 6. Implement binary dilation from scratch using only NumPy loops. Then compare with `scipy.ndimage.binary_dilation`. 7. Implement grayscale dilation using TensorFlow max pooling. 8. Create a small experiment showing how estimated area changes after dilation with increasing radius. ### AI and modeling exercises 1. Take a segmentation model output from any image dataset. Apply thresholding, opening, closing, and connected-component filtering. Report how IoU and Dice change. 2. Use test-time augmentation to obtain multiple predicted masks. Compute uncertainty intervals for area and eccentricity. 3. Design a shape-aware validation rule for medical segmentation masks. 4. Explain how morphology could be used to post-process masks produced by a promptable segmentation model. 5. Compare a CNN max-pooling layer with grayscale dilation. What is the same? What is different? 6. Propose a differentiable approximation to erosion and explain where it could be used in a neural network. ## 9.29 Chapter summary This chapter treated shapes as mathematical objects. A binary image is a set, an indicator function, and possibly a random field. Morphological operations such as dilation, erosion, opening, and closing manipulate sets through local probes called structuring elements. These operations have clean algebraic properties: monotonicity, duality, translation equivariance, extensivity or anti-extensivity, and idempotence. We then moved from morphology to shape analysis. Connected components turn masks into graphs. Distance transforms and skeletons describe the geometry of interiors. Moments, covariance matrices, compactness, eccentricity, and Fourier descriptors summarize shape numerically. Finally, we connected these classical tools to modern AI vision pipelines: segmentation post-processing, differentiable morphology, max pooling, shape-aware losses, and validation of foundation-model masks. The main message is that **computer vision does not end when a model produces a mask**. The mask is a mathematical object. Its geometry, topology, uncertainty, and statistical meaning must still be analyzed. ## References and further reading Classical mathematical morphology was developed by Matheron and Serra [@matheron1975; @serra1982]. A practical image-analysis treatment is given by Soille [@soille2003]. For general image processing and morphology operations, see Gonzalez and Woods [@gonzalez2018]. For connected components, segmentation, and shape analysis in a broad computer vision context, see Szeliski [@szeliski2022].