Expectation and Variance - Core Definitions

Expectation and variance are the two most important numerical summaries of a distribution, characterizing its center and spread respectively.

Expectation (Expected Value)

Definition

The expectation (or expected value or mean) of a random variable $X$ is:

Discrete case: $E[X] = \sum_x x \cdot p_X(x)$

Continuous case: $E[X] = \int_{-\infty}^{\infty} x \cdot f_X(x) \, dx$

provided the sum or integral converges absolutely.

The expectation represents the "average" value of $X$ over many independent repetitions. It is also denoted $\mu$ or $\mu_X$ .

Example

Fair Die: $X \in \{1,2,3,4,5,6\}$ with $p_X(k) = 1/6$ : $E[X] = \sum_{k=1}^6 k \cdot \frac{1}{6} = \frac{1+2+3+4+5+6}{6} = \frac{21}{6} = 3.5$

Example

Exponential Distribution: $X \sim \text{Exponential}(\lambda)$ with $f_X(x) = \lambda e^{-\lambda x}$ for $x \geq 0$ : $E[X] = \int_0^{\infty} x \cdot \lambda e^{-\lambda x} \, dx = \frac{1}{\lambda}$

Properties of Expectation

Linearity: For any constants $a, b$ and random variables $X, Y$ : $E[aX + b] = aE[X] + b$ $E[X + Y] = E[X] + E[Y]$

The linearity holds even if $X$ and $Y$ are dependent—no independence assumption needed!

Non-negativity: If $X \geq 0$ almost surely, then $E[X] \geq 0$ .

Monotonicity: If $X \leq Y$ almost surely, then $E[X] \leq E[Y]$ .

Variance

Definition

The variance of $X$ measures the spread of the distribution: $\text{Var}(X) = E[(X - \mu)^2] = E[X^2] - (E[X])^2$

The standard deviation is $\sigma = \sqrt{\text{Var}(X)}$ .

The variance is always non-negative: $\text{Var}(X) \geq 0$ , with equality if and only if $X$ is constant almost surely.

Example

For a fair die: $E[X^2] = \sum_{k=1}^6 k^2 \cdot \frac{1}{6} = \frac{1+4+9+16+25+36}{6} = \frac{91}{6}$ $\text{Var}(X) = \frac{91}{6} - (3.5)^2 = \frac{91}{6} - \frac{49}{4} = \frac{182 - 147}{12} = \frac{35}{12} \approx 2.917$

Properties of Variance

Shift invariance: $\text{Var}(X + b) = \text{Var}(X)$ (adding a constant doesn't change spread)

Scaling: $\text{Var}(aX) = a^2 \text{Var}(X)$

Independence: If $X$ and $Y$ are independent: $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

Remark

Expectation and variance are the first two moments of a distribution. Higher moments (skewness, kurtosis) provide additional shape information, but mean and variance suffice for many applications, especially with normal distributions which are completely determined by these two parameters.