Weierstrass Approximation Theorem

The Weierstrass approximation theorem establishes that polynomials are dense in the space of continuous functions, providing the theoretical foundation for polynomial approximation methods.

TheoremWeierstrass Approximation Theorem

Let $f \in C[a,b]$ be continuous on a closed interval. For any $\epsilon > 0$ , there exists a polynomial $p(x)$ such that: $|f(x) - p(x)| < \epsilon \quad \text{for all } x \in [a,b]$

Equivalently, polynomials are uniformly dense in $C[a,b]$ : every continuous function can be approximated arbitrarily closely by polynomials with respect to the uniform norm $\|f\|_\infty = \max_{x \in [a,b]} |f(x)|$ .

This existential result guarantees approximation is possible but doesn't specify the degree required or provide a construction. Constructive proofs use Bernstein polynomials, Chebyshev series, or other explicit approximations.

ExampleBernstein Polynomial Approximation

The Bernstein polynomial of degree $n$ for $f \in C[0,1]$ is: $B_n(f; x) = \sum_{k=0}^n f\left(\frac{k}{n}\right) \binom{n}{k} x^k (1-x)^{n-k}$

For $f(x) = x^2$ on $[0,1]$ : $B_3(f; x) = 0 \cdot \binom{3}{0}(1-x)^3 + \frac{1}{9}\binom{3}{1}x(1-x)^2 + \frac{4}{9}\binom{3}{2}x^2(1-x) + 1 \cdot \binom{3}{3}x^3$ $= \frac{1}{3}x(1-x)^2 + \frac{4}{3}x^2(1-x) + x^3 = x^2 + \frac{1}{3}x(1-x) \to x^2 \text{ as } n \to \infty$

Bernstein polynomials converge monotonically and preserve shape properties (positivity, monotonicity), making them useful in computer graphics.

Remark

Convergence Rate: For $f \in C^k[a,b]$ , the Weierstrass theorem doesn't specify convergence rate. Specific constructions achieve:

Bernstein polynomials: $O(1/\sqrt{n})$ for general $C[0,1]$ , $O(1/n)$ for Lipschitz functions
Chebyshev approximation: $O(1/n^k)$ for $C^k$ functions, exponential for analytic functions
Minimax approximation: Optimal but expensive to compute

The theorem extends to multivariate functions via tensor products, though the "curse of dimensionality" causes approximation complexity to grow exponentially with dimension.

The Stone-Weierstrass theorem generalizes to algebras of functions: if an algebra separates points and contains constants, its closure includes all continuous functions. This underlies Fourier series, wavelet approximations, and neural network universal approximation theorems.

Practical implications include:

Function libraries: Mathematical software uses polynomial or rational approximations for elementary functions ( $\sin$ , $\exp$ , etc.)
Numerical integration: Polynomial approximation of integrands justifies quadrature rules
Data compression: Approximating signals by polynomials or splines reduces storage requirements

While the theorem guarantees existence, finding good approximations requires additional theory: orthogonal polynomials, best approximation, and convergence analysis provide practical guidance for choosing approximation spaces and degrees.