MATH 5010 Section 9 — Convergence and Limit Theorems

1. Why convergence matters

Random sequences have several kinds of limits

In ordinary calculus, a sequence $a_n$ either moves toward a number or it does not. In probability, $X_n$ is random, so there are several meaningful ways to say $X_n$ approaches $X$.

$$X_n \xrightarrow{P} X,\qquad X_n \xrightarrow{d} X,\qquad X_n \xrightarrow{L^1} X,\qquad X_n \xrightarrow{L^2} X.$$

Teaching message: convergence in probability controls the chance of a visible error; convergence in distribution controls the limiting CDF; convergence in mean controls average error; the CLT tells us the shape of normalized sums.

2. Four modes of convergence

Compare definitions

Mode	Definition	Intuition
$X_n\to X$ in probability	$P(\|X_n-X\|>\varepsilon)\to0$ for all $\varepsilon>0$	Large errors become rare.
$X_n\to X$ in distribution	$F_{X_n}(t)\to F_X(t)$ at continuity points of $F_X$	The shape of the distribution stabilizes.
$X_n\to X$ in $L^1$	$E\|X_n-X\|\to0$	Mean absolute error goes to zero.
$X_n\to X$ in $L^2$	$E[(X_n-X)^2]\to0$	Mean squared error goes to zero.

Common implication map

$$X_n\xrightarrow{L^2}X\Rightarrow X_n\xrightarrow{L^1}X\Rightarrow X_n\xrightarrow{P}X\Rightarrow X_n\xrightarrow{d}X.$$

The reverse implications usually fail. The rare-spike example below is a good warning: probability convergence does not automatically control moments.

A sequence can be almost always near zero but occasionally take very large values. Those rare large values can keep expectations or second moments away from zero.

3. Rare-spike example

$X_n=n^a$ with probability $1/n^b$

This generalizes the course example $X_n=n$ with probability $1/n^2$ and $0$ otherwise.

Spike size exponent $a$: 1.00Spike probability exponent $b$: 2.00Current $n$: 20

Formula

$$X_n=\begin{cases}n^a,&\text{with probability }n^{-b},\\0,&\text{with probability }1-n^{-b}. \end{cases}$$ $$P(|X_n|>\varepsilon)=n^{-b}\to0\quad(b>0),$$ $$E[X_n]=n^{a-b},\qquad E[X_n^2]=n^{2a-b}.$$

Probability vs. moments

Blue curve: $P(|X_n|>\varepsilon)$. Other curves show $E[X_n]$ and $E[X_n^2]$. When moments do not shrink, convergence in probability still may hold.

4. Convergence in distribution

Bernoulli CDF convergence

Let $X_n\sim\mathrm{Bernoulli}(p_n)$ with $p_n=1/2+1/n$. Compare it with $X\sim\mathrm{Bernoulli}(1/2)$.

$n$: 10

For $0\le t<1$, $F_{X_n}(t)=P(X_n=0)=\frac12-\frac1n\to\frac12=F_X(t)$.

Step CDFs

Convergence is checked only at continuity points of the limiting CDF. The Bernoulli limiting CDF jumps at $0$ and $1$.

5. Law of Large Numbers

Sample mean stabilizes

Choose a distribution and watch the running average $\bar X_n$ move toward $\mu$.

DistributionNumber of observations: 300

$$\bar X_n=\frac{1}{n}\sum_{i=1}^nX_i\xrightarrow{P}\mu.$$

Running mean path

6. Central Limit Theorem

Discrete sum approximation

Use the course example $P(X_i=0)=0.2$, $P(X_i=1)=0.5$, $P(X_i=2)=0.3$. Then $\mu=1.1$ and $\sigma=0.7$.

Sample size $n$: 50Cutoff $k$ for $P(S_n\le k)$: 60Monte Carlo repetitions: 5000

CLT formulas

$$S_n=\sum_{i=1}^nX_i,\qquad E[S_n]=n\mu,\qquad \mathrm{SD}(S_n)=\sigma\sqrt n.$$ $$P(S_n\le k)\approx \Phi\!\left(\frac{k-n\mu}{\sigma\sqrt n}\right),$$ $$P(S_n\le k)\approx \Phi\!\left(\frac{k+0.5-n\mu}{\sigma\sqrt n}\right)\quad\text{with continuity correction.}$$

Normalized sums

The histogram shows simulated $Z_n=(S_n-n\mu)/(\sigma\sqrt n)$ with the standard normal curve overlaid.

7. Quick checks

Self-check questions

Q1

If $X_n\to X$ in $L^2$, must $X_n\to X$ in probability?

Q2

In the rare-spike example $X_n=n$ with probability $1/n^2$, does $X_n\to0$ in probability?

Q3

Why use a $0.5$ continuity correction for $P(S_n\le k)$?