Likelihood Ratio Tests

The generalized likelihood ratio test (GLRT) extends the Neyman-Pearson approach to composite hypotheses, providing a universal testing methodology with well-understood asymptotic behavior.

The Generalized Likelihood Ratio

Definition

For testing $H_0: \theta \in \Theta_0$ against $H_1: \theta \in \Theta_1 = \Theta \setminus \Theta_0$ , the generalized likelihood ratio statistic is $\Lambda = \frac{\sup_{\theta \in \Theta_0} L(\theta; \mathbf{x})}{\sup_{\theta \in \Theta} L(\theta; \mathbf{x})} = \frac{L(\hat{\theta}_0; \mathbf{x})}{L(\hat{\theta}; \mathbf{x})}$ where $\hat{\theta}_0$ is the MLE under $H_0$ and $\hat{\theta}$ is the unrestricted MLE. Note $0 \leq \Lambda \leq 1$ , with small values suggesting $H_0$ is implausible.

Wilks' Theorem

Theorem9.6Wilks' Theorem

Under $H_0$ and regularity conditions, the statistic $-2\log\Lambda$ converges in distribution to a chi-squared distribution: $-2\log\Lambda \xrightarrow{d} \chi^2_r \quad \text{as } n \to \infty$ where $r = \dim(\Theta) - \dim(\Theta_0)$ is the number of constraints imposed by $H_0$ . The GLRT at asymptotic level $\alpha$ rejects when $-2\log\Lambda > \chi^2_{r, \alpha}$ .

ExampleTesting normality parameters

For $X_i \sim N(\mu, \sigma^2)$ , test $H_0: \mu = \mu_0$ (with $\sigma^2$ unknown):

$\Theta = \{(\mu, \sigma^2) : \mu \in \mathbb{R}, \sigma^2 > 0\}$ has dimension $2$
$\Theta_0 = \{(\mu_0, \sigma^2) : \sigma^2 > 0\}$ has dimension $1$
So $r = 1$ and $-2\log\Lambda \sim \chi^2_1$ under $H_0$

Computing: $-2\log\Lambda = n\log\left(1 + \frac{T^2}{n-1}\right)$ where $T = \frac{\bar{X}-\mu_0}{S/\sqrt{n}}$ . For large $n$ , this is approximately $T^2 \sim \chi^2_1$ , recovering the $t$ -test.

Applications

ExampleANOVA as a likelihood ratio test

One-way ANOVA tests $H_0: \mu_1 = \mu_2 = \cdots = \mu_k$ for $k$ groups. The GLRT statistic can be shown to be a monotone function of the $F$ -statistic $F = \frac{\text{MS}_{\text{between}}}{\text{MS}_{\text{within}}}$ , with $-2\log\Lambda \xrightarrow{d} \chi^2_{k-1}$ .

RemarkModel selection

Likelihood ratio tests extend naturally to model comparison. The Akaike Information Criterion (AIC $= -2\ell(\hat{\theta}) + 2p$ ) and Bayesian Information Criterion (BIC $= -2\ell(\hat{\theta}) + p\log n$ ) penalize the log-likelihood by model complexity, providing automatic model selection criteria derived from likelihood principles.