The Poisson Limit Theorem

The Poisson limit theorem (or law of rare events) states that the sum of many independent rare events converges in distribution to a Poisson distribution. This fundamental result explains why the Poisson distribution arises ubiquitously in applications and justifies the Poisson process as a model for random arrivals.

Statement of the theorem

Theorem2.1Poisson limit theorem (binomial approximation)

Let $X_n \sim \text{Binomial}(n, p_n)$ with $n p_n \to \lambda$ as $n \to \infty$ (where $\lambda > 0$ is fixed). Then

$\mathbb{P}(X_n = k) \to e^{-\lambda} \frac{\lambda^k}{k!} \quad \text{as } n \to \infty,$

for each $k = 0, 1, 2, \ldots$ . In other words, $X_n \xrightarrow{d} \text{Poisson}(\lambda)$ .

Intuition: Suppose $n$ is large and $p_n$ is small, with $n p_n \approx \lambda$ fixed. Then the binomial distribution (sum of $n$ Bernoulli trials, each with success probability $p_n$ ) is well-approximated by a Poisson distribution with parameter $\lambda$ .

ExampleDefects in manufacturing

A factory produces $n = 10{,}000$ items per day. Each item is defective with probability $p = 0.0005$ , independently. The number of defects $X$ is $\text{Binomial}(10{,}000, 0.0005)$ . Since $np = 5$ , we have $X \approx \text{Poisson}(5)$ :

$\mathbb{P}(X = 0) \approx e^{-5} \approx 0.0067, \quad \mathbb{P}(X = 1) \approx 5 e^{-5} \approx 0.0337.$

The Poisson approximation is much simpler than the binomial formula and highly accurate for large $n$ and small $p$ .

Proof

We prove convergence using the binomial probability mass function.

Step 1: Recall

$\mathbb{P}(X_n = k) = \binom{n}{k} p_n^k (1-p_n)^{n-k}.$

Step 2: Write

$\binom{n}{k} = \frac{n(n-1)\cdots(n-k+1)}{k!} = \frac{n^k}{k!} \left(1 - \frac{1}{n}\right)\left(1 - \frac{2}{n}\right) \cdots \left(1 - \frac{k-1}{n}\right).$

Step 3: Substitute $p_n = \lambda/n + o(1/n)$ (since $np_n \to \lambda$ ):

$\mathbb{P}(X_n = k) = \frac{n^k}{k!} \left(\frac{\lambda}{n}\right)^k (1-p_n)^{n-k} \prod_{\ell=0}^{k-1} \left(1 - \frac{\ell}{n}\right).$

Step 4: Simplify:

$\mathbb{P}(X_n = k) = \frac{\lambda^k}{k!} \left(1 - \frac{\lambda}{n}\right)^{n-k} \prod_{\ell=0}^{k-1} \left(1 - \frac{\ell}{n}\right) + o(1).$

Step 5: Take the limit as $n \to \infty$ :

$\lim_{n \to \infty} \left(1 - \frac{\lambda}{n}\right)^n = e^{-\lambda}, \quad \lim_{n \to \infty} \left(1 - \frac{\lambda}{n}\right)^{-k} = 1, \quad \prod_{\ell=0}^{k-1} \left(1 - \frac{\ell}{n}\right) \to 1.$

Hence,

$\lim_{n \to \infty} \mathbb{P}(X_n = k) = \frac{\lambda^k}{k!} e^{-\lambda}.$

General version: Poisson approximation for rare events

Theorem2.2General Poisson limit theorem

Let $X_1^{(n)}, X_2^{(n)}, \ldots, X_n^{(n)}$ be independent Bernoulli random variables with $\mathbb{P}(X_i^{(n)} = 1) = p_i^{(n)}$ . Suppose:

$\sum_{i=1}^n p_i^{(n)} \to \lambda$ as $n \to \infty$ .
$\max_{1 \leq i \leq n} p_i^{(n)} \to 0$ as $n \to \infty$ (individual probabilities vanish).

Then $S_n = \sum_{i=1}^n X_i^{(n)} \xrightarrow{d} \text{Poisson}(\lambda)$ .

This is the law of rare events: the sum of many independent, rare events (each with small probability) is approximately Poisson. The events need not be identically distributed.

ExampleTypos in a book

A book has $n = 500$ pages. On page $i$ , the probability of a typo is $p_i$ (which may vary by page, depending on complexity). Suppose $\sum_{i=1}^{500} p_i = 10$ and $\max_i p_i < 0.05$ . Then the total number of typos is approximately $\text{Poisson}(10)$ .

Connection to the Poisson process

Theorem2.3Poisson process from binomial approximation

Consider a time interval $[0, t]$ divided into $n$ subintervals of length $h = t/n$ . In each subinterval, an event occurs with probability $p_n = \lambda h + o(h) = \lambda t/n + o(1/n)$ , independently. As $n \to \infty$ , the number of events in $[0, t]$ converges in distribution to $\text{Poisson}(\lambda t)$ .

This theorem provides a discrete approximation to the Poisson process: by partitioning time into many small intervals and allowing at most one event per interval (with probability proportional to the interval length), we obtain a Poisson process in the limit.

RemarkHistorical context

The Poisson distribution was introduced by Siméon Denis Poisson in 1837 as a limiting case of the binomial distribution. The Poisson limit theorem was later generalized by von Mises (1919) and Prokhorov (1953) to the law of rare events for non-identically distributed summands.

Rate of convergence

Theorem2.4Total variation distance bound

Let $X \sim \text{Binomial}(n, p)$ with $\lambda = np$ . Then

$\|\mathcal{L}(X) - \text{Poisson}(\lambda)\|_{TV} \leq \min(p, 2p\lambda).$

Here, $\|\mu - \nu\|_{TV} = \frac{1}{2} \sum_k |\mu_k - \nu_k|$ is the total variation distance.

This bound shows that the approximation is accurate when $p$ is small (the "rare event" regime). For example, if $p \leq 0.01$ and $\lambda = np \leq 10$ , the error is at most $0.01$ .

ExampleNumerical comparison

For $n = 100$ , $p = 0.05$ , $\lambda = 5$ :

| $k$ | Binomial | Poisson | Error | |-----|----------|---------|-------| | 0 | 0.0059 | 0.0067 | 0.0008 | | 1 | 0.0312 | 0.0337 | 0.0025 | | 5 | 0.1800 | 0.1755 | 0.0045 | | 10 | 0.0167 | 0.0181 | 0.0014 |

The approximation is excellent across the range.

Applications

ExampleRadioactive decay

A radioactive substance has $N$ atoms. In a small time interval $[0, t]$ , each atom decays independently with probability $p = \lambda t / N$ (where $\lambda$ is the decay rate per atom). The number of decays is approximately $\text{Poisson}(\lambda t)$ for large $N$ and small $t$ .

This justifies modeling radioactive decay as a Poisson process: the number of decay events in any time interval $[0, t]$ is $\text{Poisson}(\lambda t)$ .

ExamplePhone calls

A telephone exchange receives calls from $n = 10{,}000$ subscribers. In a one-minute interval, each subscriber calls with probability $p = 0.001$ , independently. The total number of calls is approximately $\text{Poisson}(10)$ . This is the classical model for telephone traffic (Erlang, 1909).

Summary

The Poisson limit theorem explains why the Poisson distribution is ubiquitous:

Rare events: Sum of many independent, rare events $\to$ Poisson.
Binomial approximation: $\text{Binomial}(n, p) \approx \text{Poisson}(np)$ for large $n$ , small $p$ .
Poisson process: Discretizing time and taking limits yields the Poisson process.
Applications: Defects, arrivals, decays, calls, typos, accidents, etc.

The theorem is both a practical approximation tool and a conceptual foundation for continuous-time stochastic models.