Expectation and Variance - Main Theorem

Chebyshev's inequality provides a universal bound on the probability that a random variable deviates from its mean, requiring only knowledge of the variance.

Chebyshev's Inequality

Theorem

Let $X$ be a random variable with finite mean $\mu = E[X]$ and finite variance $\sigma^2 = \text{Var}(X)$ . Then for any $k > 0$ : $P(|X - \mu| \geq k) \leq \frac{\sigma^2}{k^2}$

Equivalently, in terms of standard deviations, for any $t > 0$ : $P(|X - \mu| \geq t\sigma) \leq \frac{1}{t^2}$

Proof: Let $k > 0$ . Define the event $A = \{|X - \mu| \geq k\}$ . Consider the indicator variable: $I_A = \begin{cases} 1 & \text{if } |X - \mu| \geq k \\ 0 & \text{otherwise} \end{cases}$

Note that $I_A \leq \frac{(X-\mu)^2}{k^2}$ since:

If $|X - \mu| \geq k$ , then $(X-\mu)^2 \geq k^2$ , so $\frac{(X-\mu)^2}{k^2} \geq 1 = I_A$
If $|X - \mu| < k$ , then $I_A = 0 \leq \frac{(X-\mu)^2}{k^2}$

Taking expectations: $P(A) = E[I_A] \leq E\left[\frac{(X-\mu)^2}{k^2}\right] = \frac{1}{k^2} E[(X-\mu)^2] = \frac{\sigma^2}{k^2}$ □

Interpretation and Applications

Chebyshev's inequality states that the probability of being far from the mean decreases as the square of the distance.

Setting $k = 2\sigma$ : $P(|X - \mu| \geq 2\sigma) \leq \frac{1}{4} = 25\%$

Therefore, at least 75% of the distribution lies within 2 standard deviations of the mean.

Setting $k = 3\sigma$ : $P(|X - \mu| \geq 3\sigma) \leq \frac{1}{9} \approx 11.1\%$

At least 88.9% lies within 3 standard deviations.

Example

Exam scores have mean 70 and standard deviation 10. What can we say about the proportion of scores between 50 and 90?

The interval $[50, 90]$ is $\mu \pm 2\sigma$ . By Chebyshev: $P(50 \leq X \leq 90) = P(|X - 70| \leq 20) \geq 1 - \frac{100}{400} = 0.75$

At least 75% of students scored between 50 and 90.

Comparison with Normal Distribution

For a normal distribution $\mathcal{N}(\mu, \sigma^2)$ :

68% lie within $\mu \pm \sigma$ (Chebyshev guarantees 0%)
95% lie within $\mu \pm 2\sigma$ (Chebyshev guarantees 75%)
99.7% lie within $\mu \pm 3\sigma$ (Chebyshev guarantees 89%)

Chebyshev's bounds are conservative but universal—they hold for any distribution.

One-Sided Chebyshev Inequality

Theorem

For any $k > 0$ : $P(X - \mu \geq k) \leq \frac{\sigma^2}{\sigma^2 + k^2}$

This provides a tighter bound when we only care about deviations in one direction.

Example

A manufacturing process produces items with mean weight 100g and standard deviation 5g. What's the probability an item weighs at least 115g?

Using one-sided Chebyshev with $k = 15$ : $P(X \geq 115) \leq \frac{25}{25 + 225} = \frac{25}{250} = 0.1 = 10\%$

Remark

Chebyshev's inequality is remarkably general—it requires only finite variance and makes no distributional assumptions. While not sharp for specific distributions, it provides a universally valid bound that's particularly useful when the exact distribution is unknown.