Random Variables and Distributions - Key Properties

The distribution of a random variable encodes all probabilistic information about it. Understanding key properties of distributions is essential for both theoretical and applied work.

Properties of CDFs

The cumulative distribution function $F_X(x) = P(X \leq x)$ has several important properties that follow from the axioms of probability.

Theorem: For any random variable $X$ :

$0 \leq F_X(x) \leq 1$ for all $x$
If $x_1 < x_2$ , then $F_X(x_1) \leq F_X(x_2)$ (monotonicity)
$\lim_{x \to -\infty} F_X(x) = 0$ and $\lim_{x \to \infty} F_X(x) = 1$
$F_X$ is right-continuous: $\lim_{h \to 0^+} F_X(x+h) = F_X(x)$

From the CDF, we can compute various probabilities: $P(X > x) = 1 - F_X(x)$ $P(X < x) = \lim_{h \to 0^+} F_X(x-h) = F_X(x^-)$ $P(X = x) = F_X(x) - F_X(x^-) = \text{jump at } x$

Example

For a discrete random variable with PMF $p_X$ : $F_X(x) = \sum_{k: k \leq x} p_X(k)$

The CDF is a step function with jumps of size $p_X(k)$ at each value $k$ where $p_X(k) > 0$ .

Properties of PDFs

For continuous random variables with PDF $f_X$ :

Normalization: $\int_{-\infty}^{\infty} f_X(x) \, dx = 1$

Non-negativity: $f_X(x) \geq 0$ for all $x$

Relationship to CDF: $F_X(x) = \int_{-\infty}^x f_X(t) \, dt$ and $f_X(x) = \frac{d}{dx} F_X(x)$ (where the derivative exists)

Key Insight: Unlike probabilities, $f_X(x)$ can exceed 1. It represents probability density, not probability. Only integrals of $f_X$ give probabilities.

Example

The PDF $f_X(x) = 2x$ for $0 \leq x \leq 1$ (and 0 elsewhere) has $f_X(0.9) = 1.8 > 1$ , which is perfectly valid. The total integral is: $\int_0^1 2x \, dx = [x^2]_0^1 = 1$ ✓

Transformations of Random Variables

Theorem

If $X$ is a continuous random variable with PDF $f_X$ and $Y = g(X)$ where $g$ is strictly monotonic and differentiable, then $Y$ has PDF: $f_Y(y) = f_X(g^{-1}(y)) \left|\frac{d}{dy} g^{-1}(y)\right|$

For discrete random variables, if $Y = g(X)$ : $p_Y(y) = \sum_{x: g(x) = y} p_X(x)$

Example

If $X \sim \text{Uniform}(0,1)$ and $Y = -\ln X$ , then $Y \sim \text{Exponential}(1)$ .

Starting with $f_X(x) = 1$ for $0 < x < 1$ :

Inverse: $x = e^{-y}$ , so $\frac{dx}{dy} = -e^{-y}$
PDF of $Y$ : $f_Y(y) = 1 \cdot |-e^{-y}| = e^{-y}$ for $y > 0$

Quantile Function

Definition

The quantile function (inverse CDF) is: $F_X^{-1}(p) = \inf\{x: F_X(x) \geq p\}$

for $0 < p < 1$ . The value $F_X^{-1}(p)$ is the $p$ -th quantile.

Special quantiles:

Median: $F_X^{-1}(0.5)$
First quartile: $F_X^{-1}(0.25)$
Third quartile: $F_X^{-1}(0.75)$

Remark

The quantile function inverts the CDF and is fundamental in generating random samples. The probability integral transform states that if $U \sim \text{Uniform}(0,1)$ , then $F_X^{-1}(U)$ has distribution $F_X$ .