ProofComplete

Proof of the Neyman-Pearson Lemma

We prove the Neyman-Pearson lemma, establishing that the likelihood ratio test is the most powerful test for simple hypotheses.


Proof

Theorem (Neyman-Pearson): Let f0=f(x;ΞΈ0)f_0 = f(\mathbf{x}; \theta_0) and f1=f(x;ΞΈ1)f_1 = f(\mathbf{x}; \theta_1) be the densities under H0H_0 and H1H_1. The test Ο•βˆ—\phi^* that rejects H0H_0 when f1(x)/f0(x)>kf_1(\mathbf{x})/f_0(\mathbf{x}) > k (where kk is chosen to give size Ξ±\alpha) is the most powerful level-Ξ±\alpha test.

Step 1: Define the tests.

Let Ο•βˆ—\phi^* be the likelihood ratio test with rejection region Rβˆ—={x:f1(x)>kβ‹…f0(x)}R^* = \{\mathbf{x} : f_1(\mathbf{x}) > k \cdot f_0(\mathbf{x})\} and size Ξ±βˆ—=∫Rβˆ—f0 dx≀α\alpha^* = \int_{R^*} f_0\,d\mathbf{x} \leq \alpha.

Let Ο•\phi be any other test with rejection region RR and size Ξ±Ο•=∫Rf0 dx≀α\alpha_\phi = \int_R f_0\,d\mathbf{x} \leq \alpha.

We want to show Ξ²1(Ο•βˆ—)β‰₯Ξ²1(Ο•)\beta_1(\phi^*) \geq \beta_1(\phi), i.e., ∫Rβˆ—f1 dxβ‰₯∫Rf1 dx\int_{R^*} f_1\,d\mathbf{x} \geq \int_R f_1\,d\mathbf{x}.

Step 2: Decompose the power difference.

∫Rβˆ—f1βˆ’βˆ«Rf1=∫Rβˆ—βˆ–Rf1βˆ’βˆ«Rβˆ–Rβˆ—f1\int_{R^*} f_1 - \int_R f_1 = \int_{R^* \setminus R} f_1 - \int_{R \setminus R^*} f_1

On Rβˆ—βˆ–RR^* \setminus R: since x∈Rβˆ—\mathbf{x} \in R^*, we have f1(x)>kβ‹…f0(x)f_1(\mathbf{x}) > k \cdot f_0(\mathbf{x}), so ∫Rβˆ—βˆ–Rf1>k∫Rβˆ—βˆ–Rf0\int_{R^* \setminus R} f_1 > k \int_{R^* \setminus R} f_0

On Rβˆ–Rβˆ—R \setminus R^*: since xβˆ‰Rβˆ—\mathbf{x} \notin R^*, we have f1(x)≀kβ‹…f0(x)f_1(\mathbf{x}) \leq k \cdot f_0(\mathbf{x}), so ∫Rβˆ–Rβˆ—f1≀k∫Rβˆ–Rβˆ—f0\int_{R \setminus R^*} f_1 \leq k \int_{R \setminus R^*} f_0

Step 3: Combine the inequalities.

∫Rβˆ—f1βˆ’βˆ«Rf1>k(∫Rβˆ—βˆ–Rf0βˆ’βˆ«Rβˆ–Rβˆ—f0)=k(∫Rβˆ—f0βˆ’βˆ«Rf0)=k(Ξ±βˆ—βˆ’Ξ±Ο•)\int_{R^*} f_1 - \int_R f_1 > k\left(\int_{R^* \setminus R} f_0 - \int_{R \setminus R^*} f_0\right) = k\left(\int_{R^*} f_0 - \int_R f_0\right) = k(\alpha^* - \alpha_\phi)

Step 4: Use the size constraint.

Since Ξ±βˆ—=Ξ±\alpha^* = \alpha (by construction of kk) and αϕ≀α\alpha_\phi \leq \alpha, we have Ξ±βˆ—βˆ’Ξ±Ο•β‰₯0\alpha^* - \alpha_\phi \geq 0, so: ∫Rβˆ—f1βˆ’βˆ«Rf1β‰₯kβ‹…0=0\int_{R^*} f_1 - \int_R f_1 \geq k \cdot 0 = 0

Therefore the power of Ο•βˆ—\phi^* is at least as large as the power of any other level-Ξ±\alpha test Ο•\phi: PΞΈ1(Ο•βˆ—Β rejects)β‰₯PΞΈ1(ϕ rejects)P_{\theta_1}(\phi^* \text{ rejects}) \geq P_{\theta_1}(\phi \text{ rejects})

This holds for every level-Ξ±\alpha test Ο•\phi, so Ο•βˆ—\phi^* is the most powerful level-Ξ±\alpha test. β–‘\square

β– 

RemarkRandomized tests

When the distribution of Ξ›\Lambda under H0H_0 is discrete, there may be no kk giving exactly size Ξ±\alpha. In this case, a randomized test is used: reject with probability Ξ³\gamma when Ξ›=k\Lambda = k (on the boundary). The Neyman-Pearson lemma extends to include this randomization, and the resulting test is still most powerful.