Modes of Convergence

There are several notions of convergence for sequences of random variables, each with different strengths and applications. Understanding their relationships is essential for rigorous probability theory.

Four Modes of Convergence

Definition

Let $X_n, X$ be random variables on a probability space $(\Omega, \mathcal{F}, P)$ .

Almost sure convergence ( $X_n \xrightarrow{a.s.} X$ ): $P(\lim_{n\to\infty} X_n = X) = 1$
Convergence in probability ( $X_n \xrightarrow{P} X$ ): For all $\epsilon > 0$ , $\lim_{n\to\infty} P(|X_n - X| > \epsilon) = 0$
Convergence in $L^p$ (mean) ( $X_n \xrightarrow{L^p} X$ ): $\lim_{n\to\infty} E[|X_n - X|^p] = 0$
Convergence in distribution ( $X_n \xrightarrow{d} X$ ): $\lim_{n\to\infty} F_{X_n}(x) = F_X(x)$ at all continuity points of $F_X$

Relationships

Theorem7.5Implications Between Modes

The following implications hold: $X_n \xrightarrow{a.s.} X \implies X_n \xrightarrow{P} X \implies X_n \xrightarrow{d} X$ $X_n \xrightarrow{L^p} X \implies X_n \xrightarrow{P} X \implies X_n \xrightarrow{d} X$ None of the reverse implications hold in general, though:

$X_n \xrightarrow{P} X$ implies there exists a subsequence with $X_{n_k} \xrightarrow{a.s.} X$
$X_n \xrightarrow{d} c$ (a constant) implies $X_n \xrightarrow{P} c$

ExampleConvergence in probability but not a.s.

The "typewriter sequence": on $[0,1]$ with Lebesgue measure, let $X_n = \mathbf{1}_{[k/2^m, (k+1)/2^m]}$ where $n = 2^m + k$ . Then $X_n \xrightarrow{P} 0$ (since the intervals shrink) but $X_n$ does not converge a.s. to $0$ (every point $\omega$ is eventually covered by infinitely many intervals).

Slutsky's Theorem

Theorem7.6Slutsky's Theorem

If $X_n \xrightarrow{d} X$ and $Y_n \xrightarrow{P} c$ (a constant), then:

$X_n + Y_n \xrightarrow{d} X + c$
$Y_n X_n \xrightarrow{d} cX$
$X_n / Y_n \xrightarrow{d} X/c$ (if $c \neq 0$ )

RemarkThe continuous mapping theorem

If $X_n \xrightarrow{d} X$ and $g$ is a continuous function, then $g(X_n) \xrightarrow{d} g(X)$ . Combined with Slutsky's theorem, this allows one to derive the limiting distribution of complex statistics from simpler ones — a technique used extensively in asymptotic statistics.