Expectation and Variance - Examples and Constructions

Computing expectations and variances for standard distributions reveals patterns and provides reference values for applications.

Common Discrete Distributions

Bernoulli $(p)$ : $E[X] = p, \quad \text{Var}(X) = p(1-p)$

Binomial $(n,p)$ : Sum of $n$ independent Bernoulli $(p)$ trials: $E[X] = np, \quad \text{Var}(X) = np(1-p)$

Geometric $(p)$ : Number of trials until first success: $E[X] = \frac{1}{p}, \quad \text{Var}(X) = \frac{1-p}{p^2}$

Poisson $(\lambda)$ : $E[X] = \lambda, \quad \text{Var}(X) = \lambda$

Remarkably, for Poisson the mean equals the variance!

Example

For Binomial $(100, 0.3)$ : $E[X] = 100 \times 0.3 = 30$ $\text{Var}(X) = 100 \times 0.3 \times 0.7 = 21$ $\sigma = \sqrt{21} \approx 4.58$

By Chebyshev: At least 75% of outcomes lie in $[30 - 2(4.58), 30 + 2(4.58)] = [20.84, 39.16]$ .

Common Continuous Distributions

Uniform $(a,b)$ : $E[X] = \frac{a+b}{2}, \quad \text{Var}(X) = \frac{(b-a)^2}{12}$

Exponential $(\lambda)$ : $E[X] = \frac{1}{\lambda}, \quad \text{Var}(X) = \frac{1}{\lambda^2}$

Normal $(\mu, \sigma^2)$ : $E[X] = \mu, \quad \text{Var}(X) = \sigma^2$

The parameters directly give the mean and variance!

Gamma $(\alpha, \lambda)$ : Generalizes exponential ( $\alpha = 1$ ): $E[X] = \frac{\alpha}{\lambda}, \quad \text{Var}(X) = \frac{\alpha}{\lambda^2}$

Example

Standardization: For any $X$ with mean $\mu$ and variance $\sigma^2$ , define: $Z = \frac{X - \mu}{\sigma}$

Then $E[Z] = 0$ and $\text{Var}(Z) = 1$ . This standardized variable has mean 0 and variance 1.

If $X \sim \mathcal{N}(\mu, \sigma^2)$ , then $Z \sim \mathcal{N}(0,1)$ (standard normal).

Computing Variance via MGF

Example

For $X \sim \text{Poisson}(\lambda)$ , the MGF is: $M_X(t) = e^{\lambda(e^t - 1)}$

Derivatives: $M_X'(t) = \lambda e^t \cdot e^{\lambda(e^t - 1)}$ $M_X''(t) = \lambda e^t \cdot e^{\lambda(e^t - 1)} \cdot (\lambda e^t + 1)$

At $t = 0$ : $E[X] = M_X'(0) = \lambda$ $E[X^2] = M_X''(0) = \lambda(\lambda + 1) = \lambda^2 + \lambda$ $\text{Var}(X) = E[X^2] - (E[X])^2 = \lambda^2 + \lambda - \lambda^2 = \lambda$

Sums of Independent Random Variables

Theorem: If $X_1, \ldots, X_n$ are independent: $E\left[\sum_{i=1}^n X_i\right] = \sum_{i=1}^n E[X_i]$ $\text{Var}\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \text{Var}(X_i)$

Example

If $X_1, \ldots, X_{100}$ are independent Exponential $(2)$ random variables: $S = \sum_{i=1}^{100} X_i$

has: $E[S] = 100 \times \frac{1}{2} = 50$ $\text{Var}(S) = 100 \times \frac{1}{4} = 25$ $\sigma_S = 5$

In fact, $S \sim \text{Gamma}(100, 2)$ .

Remark

Linearity of expectation holds universally, even for dependent variables. Variance additivity requires independence. These formulas are the workhorses of probability calculations—they allow us to build complex distributions from simple building blocks.