Joint and Conditional Distributions - Examples and Constructions

Joint distributions model real-world scenarios where multiple measurements or observations occur simultaneously.

Multinomial Distribution

Definition

Extension of binomial to $k$ categories. If $n$ independent trials result in category $i$ with probability $p_i$ (where $\sum p_i = 1$ ), the counts $(X_1, \ldots, X_k)$ follow: $P(X_1=n_1, \ldots, X_k=n_k) = \frac{n!}{n_1! \cdots n_k!} p_1^{n_1} \cdots p_k^{n_k}$

where $\sum n_i = n$ .

Example

Roll a die 12 times. Let $X_i$ = count of outcome $i$ .

Probability of getting each face exactly twice: $P(X_1=2, \ldots, X_6=2) = \frac{12!}{(2!)^6} \left(\frac{1}{6}\right)^{12} \approx 0.00344$

Bivariate Transformations

For $U = g_1(X,Y)$ and $V = g_2(X,Y)$ with invertible transformation: $f_{U,V}(u,v) = f_{X,Y}(x,y) |J|$

where $J$ is the Jacobian determinant: $J = \begin{vmatrix} \frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v} \end{vmatrix}$

Example

Polar Coordinates: $X, Y \sim \mathcal{N}(0,1)$ independent.

Transform to $(R, \Theta)$ where $X = R\cos\Theta, Y = R\sin\Theta$ : $f_{R,\Theta}(r, \theta) = f_{X,Y}(r\cos\theta, r\sin\theta) \cdot r$

Since $f_{X,Y}(x,y) = \frac{1}{2\pi}e^{-(x^2+y^2)/2}$ and $x^2 + y^2 = r^2$ : $f_{R,\Theta}(r,\theta) = \frac{r}{2\pi}e^{-r^2/2}$

Factoring: $R$ and $\Theta$ are independent! $R \sim \text{Rayleigh}$ , $\Theta \sim \text{Uniform}(0,2\pi)$ .

Order Statistics

For IID sample $X_1, \ldots, X_n$ , the joint PDF of order statistics $(X_{(1)}, \ldots, X_{(n)})$ is: $f_{(1),\ldots,(n)}(x_1, \ldots, x_n) = n! \prod_{i=1}^n f(x_i)$

for $x_1 < x_2 < \cdots < x_n$ .

Example

For $n=3$ uniform $(0,1)$ variables: $f_{(1),(2),(3)}(x,y,z) = 6 \text{ for } 0 < x < y < z < 1$

Joint density is constant over the simplex!

Copulas

Definition

A copula is a joint distribution function on $[0,1]^2$ with uniform marginals. It separates dependence structure from marginal distributions.

For any joint CDF $F_{X,Y}$ : $F_{X,Y}(x,y) = C(F_X(x), F_Y(y))$

where $C$ is the copula function.

Common copulas: Gaussian, t-, Archimedean (Clayton, Gumbel, Frank).

Remark

Copulas enable flexible modeling by separating "what" variables we're modeling (marginals) from "how" they're related (dependence structure). This is powerful in finance for modeling joint tail risk.