Joint and Conditional Distributions - Main Theorem

The convolution formula provides a method for computing the distribution of sums of independent random variables.

Convolution Theorem

Theorem

If $X$ and $Y$ are independent continuous random variables with PDFs $f_X$ and $f_Y$ , then $Z = X + Y$ has PDF: $f_Z(z) = \int_{-\infty}^{\infty} f_X(x) f_Y(z-x) \, dx = \int_{-\infty}^{\infty} f_Y(y) f_X(z-y) \, dy$

This is the convolution of $f_X$ and $f_Y$ , written $f_Z = f_X * f_Y$ .

Proof: For any $z$ : $F_Z(z) = P(X + Y \leq z) = \int\int_{x+y \leq z} f_{X,Y}(x,y) \, dx \, dy$

By independence, $f_{X,Y}(x,y) = f_X(x)f_Y(y)$ : $= \int_{-\infty}^{\infty} f_X(x) \left[\int_{-\infty}^{z-x} f_Y(y) \, dy\right] dx = \int_{-\infty}^{\infty} f_X(x) F_Y(z-x) \, dx$

Differentiating with respect to $z$ : $f_Z(z) = \int_{-\infty}^{\infty} f_X(x) f_Y(z-x) \, dx$ □

Example

Sum of Two Uniforms: $X, Y \sim \text{Uniform}(0,1)$ independent. Find PDF of $Z = X + Y$ .

$f_Z(z) = \int_{-\infty}^{\infty} f_X(x) f_Y(z-x) \, dx$

For $0 < z < 1$ : $f_X(x) = 1$ for $0 < x < z$ , $f_Y(z-x) = 1$ for $0 < z-x < 1$ : $f_Z(z) = \int_0^z 1 \cdot 1 \, dx = z$

For $1 < z < 2$ : Both conditions give $z-1 < x < 1$ : $f_Z(z) = \int_{z-1}^1 1 \cdot 1 \, dx = 2 - z$

Result: $f_Z(z) = \begin{cases} z & 0 < z < 1 \\ 2-z & 1 < z < 2 \end{cases}$ (triangular distribution)

Discrete Convolution

For discrete independent $X$ and $Y$ : $p_Z(z) = \sum_x p_X(x) p_Y(z-x) = \sum_y p_Y(y) p_X(z-y)$

Example

Sum of two dice: $X, Y \in \{1,2,3,4,5,6\}$ with $p_X(k) = p_Y(k) = 1/6$ .

For $Z = X + Y = 7$ : $p_Z(7) = \sum_{k=1}^6 p_X(k) p_Y(7-k) = \sum_{k=1}^6 \frac{1}{6} \cdot \frac{1}{6} \cdot \mathbb{1}\{1 \leq 7-k \leq 6\}$ $= 6 \times \frac{1}{36} = \frac{1}{6}$

Six ways: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1).

Remark

Convolution is computationally intensive but conceptually fundamental. For practical calculations with many sums, MGFs or characteristic functions are often more efficient. However, convolution provides geometric intuition: we're "sliding" one PDF along the other and integrating overlaps.