Confidence Intervals

A confidence interval provides a range of plausible values for an unknown parameter, quantifying the uncertainty inherent in estimation from random samples.

Definition

A $100(1-\alpha)\%$ confidence interval for a parameter $\theta$ is a random interval $[L(X_1,\ldots,X_n), U(X_1,\ldots,X_n)]$ such that $P(L \leq \theta \leq U) = 1 - \alpha$ for all $\theta \in \Theta$ . The value $1 - \alpha$ is the confidence level (typically $0.90$ , $0.95$ , or $0.99$ ). The half-width $U - L$ is the margin of error.

The correct interpretation is frequentist: if we repeat the sampling procedure many times, approximately $100(1-\alpha)\%$ of the resulting intervals will contain the true $\theta$ .

Common Confidence Intervals

ExampleCI for the mean (known variance)

For $X_i \sim N(\mu, \sigma^2)$ with $\sigma$ known, an exact $95\%$ CI for $\mu$ is: $\bar{X} \pm 1.96 \frac{\sigma}{\sqrt{n}}$ This follows from $\frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$ .

ExampleCI for the mean (unknown variance)

When $\sigma$ is unknown, replace $\sigma$ by $S$ (sample standard deviation). Then $\frac{\bar{X} - \mu}{S/\sqrt{n}} \sim t_{n-1}$ (Student's $t$ -distribution with $n-1$ degrees of freedom), giving: $\bar{X} \pm t_{n-1, \alpha/2} \frac{S}{\sqrt{n}}$ For $n = 25$ and $95\%$ confidence: $t_{24, 0.025} = 2.064$ (slightly wider than the $z$ -interval).

Properties

Theorem8.3Sample Size Determination

To achieve a margin of error $E$ for a $100(1-\alpha)\%$ CI for the mean with known $\sigma$ , the required sample size is $n = \left\lceil \left(\frac{z_{\alpha/2} \sigma}{E}\right)^2 \right\rceil$ Halving the margin of error requires quadrupling the sample size.

RemarkConfidence intervals vs. credible intervals

A frequentist confidence interval says "if we repeated this experiment, $95\%$ of intervals would contain $\theta$ ." It does not say " $\theta$ is in this interval with probability $95\%$ ." The Bayesian analogue, a credible interval, does have this interpretation: given the posterior distribution $\pi(\theta | \text{data})$ , a $95\%$ credible interval is $[a, b]$ with $\int_a^b \pi(\theta | \text{data})\,d\theta = 0.95$ .