The Neyman-Pearson Lemma

The Neyman-Pearson lemma characterizes the most powerful test for simple hypotheses, providing the theoretical foundation for optimal hypothesis testing.

The Lemma

Theorem9.4Neyman-Pearson Lemma

Consider testing $H_0: \theta = \theta_0$ against $H_1: \theta = \theta_1$ (simple vs. simple). Among all tests with significance level $\leq \alpha$ , the likelihood ratio test that rejects $H_0$ when $\Lambda(\mathbf{x}) = \frac{L(\theta_1; \mathbf{x})}{L(\theta_0; \mathbf{x})} > k$ where $k$ is chosen so that $P_{\theta_0}(\Lambda > k) = \alpha$ , is the most powerful test. That is, it maximizes the power $P_{\theta_1}(\text{reject } H_0)$ among all level- $\alpha$ tests.

The intuition is straightforward: reject $H_0$ when the data is much more likely under $H_1$ than under $H_0$ , as measured by the likelihood ratio.

Application

ExampleNormal mean, one-sided

Test $H_0: \mu = 0$ vs. $H_1: \mu = 1$ for $X_1, \ldots, X_n \sim N(\mu, 1)$ : $\Lambda = \frac{\prod \frac{1}{\sqrt{2\pi}}e^{-(x_i-1)^2/2}}{\prod \frac{1}{\sqrt{2\pi}}e^{-x_i^2/2}} = \exp\left(\sum x_i - \frac{n}{2}\right)$ So $\Lambda > k$ iff $\bar{X} > k'$ for some constant $k'$ . The Neyman-Pearson test is: reject if $\bar{X} > z_\alpha/\sqrt{n}$ . This is the uniformly most powerful (UMP) test for $H_0: \mu = 0$ vs. $H_1: \mu > 0$ (for any $\mu > 0$ ).

Uniformly Most Powerful Tests

Theorem9.5UMP Tests for Monotone Likelihood Ratio

If the family $\{f(x;\theta)\}$ has a monotone likelihood ratio in a statistic $T(\mathbf{x})$ — meaning $\Lambda(\mathbf{x}) = f(\mathbf{x}; \theta_1)/f(\mathbf{x}; \theta_0)$ is a non-decreasing function of $T$ for $\theta_1 > \theta_0$ — then the test "reject $H_0: \theta \leq \theta_0$ when $T > c$ " is uniformly most powerful at level $\alpha$ against $H_1: \theta > \theta_0$ .

RemarkLimitations

For two-sided alternatives ( $H_1: \theta \neq \theta_0$ ), UMP tests generally do not exist (except in special cases). The Neyman-Pearson framework motivates the generalized likelihood ratio test $\Lambda = \sup_{\theta \in \Theta_0} L(\theta) / \sup_{\theta \in \Theta} L(\theta)$ , which provides a practical testing method even when optimality cannot be guaranteed.