Joint and Conditional Distributions - Core Definitions

When analyzing multiple random variables simultaneously, joint distributions capture their collective behavior and dependencies.

Joint Distributions

Definition

For discrete random variables $X$ and $Y$ , the joint PMF is: $p_{X,Y}(x,y) = P(X = x, Y = y)$

For continuous random variables, the joint PDF $f_{X,Y}(x,y)$ satisfies: $P((X,Y) \in A) = \iint_A f_{X,Y}(x,y) \, dx \, dy$

The joint CDF is: $F_{X,Y}(x,y) = P(X \leq x, Y \leq y)$

Normalization: $\sum_x \sum_y p_{X,Y}(x,y) = 1$ (discrete) or $\int\int f_{X,Y}(x,y) \, dx \, dy = 1$ (continuous)

Example

Toss two dice. Let $X$ = first die, $Y$ = second die.

Joint PMF: $p_{X,Y}(i,j) = 1/36$ for $i,j \in \{1,2,3,4,5,6\}$

Probability both dice show even: $P(X \text{ even}, Y \text{ even}) = \sum_{i \in \{2,4,6\}} \sum_{j \in \{2,4,6\}} \frac{1}{36} = \frac{9}{36} = \frac{1}{4}$

Marginal Distributions

Definition

The marginal distributions are obtained by summing/integrating over the other variable:

Discrete: $p_X(x) = \sum_y p_{X,Y}(x,y)$ , $p_Y(y) = \sum_x p_{X,Y}(x,y)$

Continuous: $f_X(x) = \int f_{X,Y}(x,y) \, dy$ , $f_Y(y) = \int f_{X,Y}(x,y) \, dx$

The marginal distribution of $X$ ignores $Y$ , giving the distribution of $X$ alone.

Example

Uniform on Triangle: $(X,Y)$ uniform on triangle $\{(x,y): 0 \leq x \leq 1, 0 \leq y \leq x\}$ .

Joint PDF: $f_{X,Y}(x,y) = 2$ for $0 \leq y \leq x \leq 1$ (area = 1/2, so density = 2)

Marginal of $X$ : $f_X(x) = \int_0^x 2 \, dy = 2x, \quad 0 \leq x \leq 1$

Marginal of $Y$ : $f_Y(y) = \int_y^1 2 \, dx = 2(1-y), \quad 0 \leq y \leq 1$

Conditional Distributions

Definition

The conditional PMF of $X$ given $Y = y$ is: $p_{X|Y}(x|y) = \frac{p_{X,Y}(x,y)}{p_Y(y)}, \quad p_Y(y) > 0$

The conditional PDF is: $f_{X|Y}(x|y) = \frac{f_{X,Y}(x,y)}{f_Y(y)}, \quad f_Y(y) > 0$

Conditional distributions represent the distribution of $X$ given knowledge of $Y = y$ .

Example

For the triangle example: $f_{X|Y}(x|y) = \frac{f_{X,Y}(x,y)}{f_Y(y)} = \frac{2}{2(1-y)} = \frac{1}{1-y}, \quad y \leq x \leq 1$

Given $Y = y$ , $X$ is uniform on $[y, 1]$ !

Remark

Joint distributions encode all information about multiple random variables. Marginals describe individual behavior, while conditionals describe how one variable behaves given knowledge of another. Together, these concepts form the foundation for multivariate probability theory.