TheoremComplete

QR Algorithm Convergence Theorem

Theorem6.2Convergence of the Shifted QR Algorithm

Let A∈Rn×nA \in \mathbb{R}^{n \times n} be a symmetric matrix in tridiagonal form with nonzero subdiagonal entries. The QR algorithm with Wilkinson shift produces a sequence AkA_k such that the bottom subdiagonal entry βk=an,n−1(k)\beta_k = a_{n,n-1}^{(k)} satisfies ∣βk+1∣≤c∣βk∣3|\beta_{k+1}| \leq c |\beta_k|^3 for a constant cc depending on the eigenvalue gap. In particular, the convergence is globally and at least cubically convergent: the bottom eigenvalue is isolated in O(log⁡log⁡(1/ε))O(\log\log(1/\varepsilon)) steps to achieve accuracy ε\varepsilon.


Proof Sketch

Proof

Setup. Consider the tridiagonal matrix TkT_k with bottom-right 2×22 \times 2 block (αn−1ββαn)\begin{pmatrix} \alpha_{n-1} & \beta \\ \beta & \alpha_n \end{pmatrix}. The Wilkinson shift σ\sigma is the eigenvalue of this block closer to αn\alpha_n.

Key estimate. Write δ=(αn−1−αn)/2\delta = (\alpha_{n-1} - \alpha_n)/2 and μ=αn−σ=−δ+sign(δ)δ2+β2\mu = \alpha_n - \sigma = -\delta + \mathrm{sign}(\delta)\sqrt{\delta^2 + \beta^2}. Then ∣μ∣≤β2/(2∣δ∣)|\mu| \leq \beta^2 / (2|\delta|) when ∣δ∣≫∣β∣|\delta| \gg |\beta|, and ∣μ∣≤∣β∣|\mu| \leq |\beta| always.

One QR step. After the shifted QR step Tk−σI=QRT_k - \sigma I = QR, Tk+1=RQ+σIT_{k+1} = RQ + \sigma I, the new subdiagonal entry satisfies ∣β′∣=∣β∣∏j=1n−2∣λj−σ∣∣dj∣|\beta'| = |\beta| \prod_{j=1}^{n-2} \frac{|\lambda_j - \sigma|}{|d_j|} where djd_j are related to the QR factorization. The Wilkinson shift guarantees that (Tk−σI)(T_k - \sigma I) has its smallest singular value at the bottom, causing ∣β′∣/∣β∣|\beta'|/|\beta| to be small.

Cubic convergence. For symmetric tridiagonal matrices, Wilkinson proved that ∣βk+1∣=O(∣βk∣3)|\beta_{k+1}| = O(|\beta_k|^3) using the identity relating the shift quality to the subdiagonal entry. The key insight is that ∣αn−σk∣=O(βk2)|\alpha_n - \sigma_k| = O(\beta_k^2) (the shift approximates the eigenvalue quadratically well), and this quadratic approximation combined with the inverse-iteration-like nature of shifted QR yields cubic convergence of βk→0\beta_k \to 0.

Global convergence. The monotonicity ∑iβi2\sum_i \beta_i^2 is non-increasing under QR steps (related to the Wilkinson monotonicity theorem) ensures global convergence. Once ∣βk∣|\beta_k| is small enough for the cubic estimate to dominate, convergence is rapid. □\square

■

ExampleOverall Complexity of Symmetric Eigenvalue Problem

For an n×nn \times n symmetric matrix:

  1. Tridiagonalization via Householder: 43n3\frac{4}{3}n^3 flops
  2. QR iterations with Wilkinson shift: ∼2n\sim 2n total QR steps (cubic convergence means ∼2\sim 2 steps per eigenvalue), each costing O(n)O(n), total O(n2)O(n^2)
  3. Total: 43n3+O(n2)≈43n3\frac{4}{3}n^3 + O(n^2) \approx \frac{4}{3}n^3 flops for all eigenvalues

For eigenvalues and eigenvectors: 43n3+O(n3)=O(n3)\frac{4}{3}n^3 + O(n^3) = O(n^3) (accumulating orthogonal transformations dominates).

RemarkNon-Symmetric Case

For non-symmetric matrices, the QR algorithm (with double shifts) converges in practice but lacks a rigorous global convergence proof. The Hessenberg QR step costs O(n2)O(n^2) and typically requires O(n)O(n) total iterations. Exceptional shifts are used when convergence stalls. The total cost is O(n3)O(n^3) for computing all eigenvalues of a general n×nn \times n matrix.