TheoremComplete

QR Algorithm Convergence Theorem

Theorem6.2Convergence of the Shifted QR Algorithm

Let ARn×nA \in \mathbb{R}^{n \times n} be a symmetric matrix in tridiagonal form with nonzero subdiagonal entries. The QR algorithm with Wilkinson shift produces a sequence AkA_k such that the bottom subdiagonal entry βk=an,n1(k)\beta_k = a_{n,n-1}^{(k)} satisfies βk+1cβk3|\beta_{k+1}| \leq c |\beta_k|^3 for a constant cc depending on the eigenvalue gap. In particular, the convergence is globally and at least cubically convergent: the bottom eigenvalue is isolated in O(loglog(1/ε))O(\log\log(1/\varepsilon)) steps to achieve accuracy ε\varepsilon.


Proof Sketch

Proof

Setup. Consider the tridiagonal matrix TkT_k with bottom-right 2×22 \times 2 block (αn1ββαn)\begin{pmatrix} \alpha_{n-1} & \beta \\ \beta & \alpha_n \end{pmatrix}. The Wilkinson shift σ\sigma is the eigenvalue of this block closer to αn\alpha_n.

Key estimate. Write δ=(αn1αn)/2\delta = (\alpha_{n-1} - \alpha_n)/2 and μ=αnσ=δ+sign(δ)δ2+β2\mu = \alpha_n - \sigma = -\delta + \mathrm{sign}(\delta)\sqrt{\delta^2 + \beta^2}. Then μβ2/(2δ)|\mu| \leq \beta^2 / (2|\delta|) when δβ|\delta| \gg |\beta|, and μβ|\mu| \leq |\beta| always.

One QR step. After the shifted QR step TkσI=QRT_k - \sigma I = QR, Tk+1=RQ+σIT_{k+1} = RQ + \sigma I, the new subdiagonal entry satisfies β=βj=1n2λjσdj|\beta'| = |\beta| \prod_{j=1}^{n-2} \frac{|\lambda_j - \sigma|}{|d_j|} where djd_j are related to the QR factorization. The Wilkinson shift guarantees that (TkσI)(T_k - \sigma I) has its smallest singular value at the bottom, causing β/β|\beta'|/|\beta| to be small.

Cubic convergence. For symmetric tridiagonal matrices, Wilkinson proved that βk+1=O(βk3)|\beta_{k+1}| = O(|\beta_k|^3) using the identity relating the shift quality to the subdiagonal entry. The key insight is that αnσk=O(βk2)|\alpha_n - \sigma_k| = O(\beta_k^2) (the shift approximates the eigenvalue quadratically well), and this quadratic approximation combined with the inverse-iteration-like nature of shifted QR yields cubic convergence of βk0\beta_k \to 0.

Global convergence. The monotonicity iβi2\sum_i \beta_i^2 is non-increasing under QR steps (related to the Wilkinson monotonicity theorem) ensures global convergence. Once βk|\beta_k| is small enough for the cubic estimate to dominate, convergence is rapid. \square


ExampleOverall Complexity of Symmetric Eigenvalue Problem

For an n×nn \times n symmetric matrix:

  1. Tridiagonalization via Householder: 43n3\frac{4}{3}n^3 flops
  2. QR iterations with Wilkinson shift: 2n\sim 2n total QR steps (cubic convergence means 2\sim 2 steps per eigenvalue), each costing O(n)O(n), total O(n2)O(n^2)
  3. Total: 43n3+O(n2)43n3\frac{4}{3}n^3 + O(n^2) \approx \frac{4}{3}n^3 flops for all eigenvalues

For eigenvalues and eigenvectors: 43n3+O(n3)=O(n3)\frac{4}{3}n^3 + O(n^3) = O(n^3) (accumulating orthogonal transformations dominates).

RemarkNon-Symmetric Case

For non-symmetric matrices, the QR algorithm (with double shifts) converges in practice but lacks a rigorous global convergence proof. The Hessenberg QR step costs O(n2)O(n^2) and typically requires O(n)O(n) total iterations. Exceptional shifts are used when convergence stalls. The total cost is O(n3)O(n^3) for computing all eigenvalues of a general n×nn \times n matrix.