ProofComplete

Proof: Newton's Method Convergence

We prove that Newton's method converges quadratically to simple roots, establishing the fundamental convergence rate for this widely-used algorithm.

ProofQuadratic Convergence of Newton's Method

Given: f∈C2[a,b]f \in C^2[a,b] with f(r)=0f(r) = 0, fβ€²(r)β‰ 0f'(r) \neq 0 for r∈(a,b)r \in (a,b).

To Prove: There exists Ξ΄>0\delta > 0 such that if ∣x0βˆ’r∣<Ξ΄|x_0 - r| < \delta, then Newton's iteration converges quadratically with: lim⁑nβ†’βˆžβˆ£xn+1βˆ’r∣∣xnβˆ’r∣2=∣fβ€²β€²(r)∣2∣fβ€²(r)∣\lim_{n \to \infty} \frac{|x_{n+1} - r|}{|x_n - r|^2} = \frac{|f''(r)|}{2|f'(r)|}

Proof:

Define the error en=xnβˆ’re_n = x_n - r. Newton's iteration is: xn+1=xnβˆ’f(xn)fβ€²(xn)x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}

Therefore: en+1=xn+1βˆ’r=xnβˆ’f(xn)fβ€²(xn)βˆ’r=enβˆ’f(xn)fβ€²(xn)e_{n+1} = x_{n+1} - r = x_n - \frac{f(x_n)}{f'(x_n)} - r = e_n - \frac{f(x_n)}{f'(x_n)}

Since f(r)=0f(r) = 0, Taylor's theorem with remainder gives: f(xn)=f(r)+fβ€²(r)(xnβˆ’r)+fβ€²β€²(ΞΎn)2(xnβˆ’r)2f(x_n) = f(r) + f'(r)(x_n - r) + \frac{f''(\xi_n)}{2}(x_n - r)^2 =fβ€²(r)en+fβ€²β€²(ΞΎn)2en2= f'(r)e_n + \frac{f''(\xi_n)}{2}e_n^2

where ΞΎn\xi_n is between xnx_n and rr.

Substituting into the error equation: en+1=enβˆ’fβ€²(r)en+fβ€²β€²(ΞΎn)2en2fβ€²(xn)e_{n+1} = e_n - \frac{f'(r)e_n + \frac{f''(\xi_n)}{2}e_n^2}{f'(x_n)}

=enβˆ’fβ€²(r)fβ€²(xn)enβˆ’fβ€²β€²(ΞΎn)2fβ€²(xn)en2= e_n - \frac{f'(r)}{f'(x_n)}e_n - \frac{f''(\xi_n)}{2f'(x_n)}e_n^2

=en(1βˆ’fβ€²(r)fβ€²(xn))βˆ’fβ€²β€²(ΞΎn)2fβ€²(xn)en2= e_n\left(1 - \frac{f'(r)}{f'(x_n)}\right) - \frac{f''(\xi_n)}{2f'(x_n)}e_n^2

Now, since fβ€²f' is continuous and fβ€²(r)β‰ 0f'(r) \neq 0, there exists Ξ΄1>0\delta_1 > 0 such that ∣xnβˆ’r∣<Ξ΄1|x_n - r| < \delta_1 implies fβ€²(xn)f'(x_n) has the same sign as fβ€²(r)f'(r) and ∣fβ€²(xn)∣β‰₯∣fβ€²(r)∣/2|f'(x_n)| \geq |f'(r)|/2.

For ∣xnβˆ’r∣<Ξ΄1|x_n - r| < \delta_1, as xnβ†’rx_n \to r, we have fβ€²(xn)β†’fβ€²(r)f'(x_n) \to f'(r), so: 1βˆ’fβ€²(r)fβ€²(xn)β†’01 - \frac{f'(r)}{f'(x_n)} \to 0

More precisely, by continuity: fβ€²(r)βˆ’fβ€²(xn)fβ€²(xn)=O(∣xnβˆ’r∣)\frac{f'(r) - f'(x_n)}{f'(x_n)} = O(|x_n - r|)

The dominant term is: en+1=βˆ’fβ€²β€²(ΞΎn)2fβ€²(xn)en2+O(en3)e_{n+1} = -\frac{f''(\xi_n)}{2f'(x_n)}e_n^2 + O(e_n^3)

Since fβ€²β€²f'' and fβ€²f' are continuous, fβ€²β€²(ΞΎn)β†’fβ€²β€²(r)f''(\xi_n) \to f''(r) and fβ€²(xn)β†’fβ€²(r)f'(x_n) \to f'(r) as nβ†’βˆžn \to \infty. Therefore: ∣en+1∣∣en∣2β†’βˆ£fβ€²β€²(r)∣2∣fβ€²(r)∣\frac{|e_{n+1}|}{|e_n|^2} \to \frac{|f''(r)|}{2|f'(r)|}

This establishes quadratic convergence. The error bound follows: ∣en+1βˆ£β‰€M2m∣en∣2|e_{n+1}| \leq \frac{M}{2m}|e_n|^2 where M=sup⁑∣fβ€²β€²(x)∣M = \sup|f''(x)| and m=inf⁑∣fβ€²(x)∣m = \inf|f'(x)| on a neighborhood of rr.

β– 
Remark

Key Insights:

  1. The proof requires fβ€²(r)β‰ 0f'(r) \neq 0 (simple root). When fβ€²(r)=0f'(r) = 0 (multiple root), the first-order term doesn't vanish, yielding only linear convergence.
  2. The convergence constant ∣fβ€²β€²(r)∣/(2∣fβ€²(r)∣)|f''(r)|/(2|f'(r)|) depends on local curvature. Larger ∣fβ€²β€²βˆ£|f''| or smaller ∣fβ€²βˆ£|f'| increases the constant, requiring closer initial guesses.
  3. The quadratic convergence is asymptotic: it holds once xnx_n is sufficiently close to rr. Global convergence is not guaranteed.

This proof technique extends to multidimensional Newton's method, where the Jacobian matrix replaces fβ€²f' and the Hessian tensor relates to fβ€²β€²f'', yielding similar quadratic convergence near non-singular roots.