ProofComplete

Proof of Cholesky Factorization Existence

Every symmetric positive definite matrix admits a unique Cholesky factorization. This factorization is numerically stable without pivoting and costs half the operations of general LU decomposition.


Statement

Theorem5.3Cholesky Factorization Theorem

Let A∈Rn×nA \in \mathbb{R}^{n \times n} be symmetric positive definite (SPD). Then there exists a unique lower triangular matrix LL with positive diagonal entries such that A=LLTA = LL^T. Moreover, ℓii=aii−∑k=1i−1ℓik2>0\ell_{ii} = \sqrt{a_{ii} - \sum_{k=1}^{i-1}\ell_{ik}^2} > 0 for all ii.


Proof

Proof

Existence by induction. For n=1n = 1: A=(a11)A = (a_{11}) with a11>0a_{11} > 0, so L=(a11)L = (\sqrt{a_{11}}).

For the inductive step, partition A=(An−1ccTann)A = \begin{pmatrix} A_{n-1} & c \\ c^T & a_{nn} \end{pmatrix} where An−1A_{n-1} is (n−1)×(n−1)(n-1) \times (n-1). Since any principal submatrix of an SPD matrix is SPD, An−1A_{n-1} is SPD. By the induction hypothesis, An−1=Ln−1Ln−1TA_{n-1} = L_{n-1} L_{n-1}^T.

Seek L=(Ln−10ℓTℓnn)L = \begin{pmatrix} L_{n-1} & 0 \\ \ell^T & \ell_{nn} \end{pmatrix} so that LLT=(Ln−1Ln−1TLn−1ℓℓTLn−1TℓTℓ+ℓnn2)=ALL^T = \begin{pmatrix} L_{n-1}L_{n-1}^T & L_{n-1}\ell \\ \ell^T L_{n-1}^T & \ell^T\ell + \ell_{nn}^2 \end{pmatrix} = A.

From Ln−1ℓ=cL_{n-1}\ell = c, we get ℓ=Ln−1−1c\ell = L_{n-1}^{-1}c (unique, since Ln−1L_{n-1} is nonsingular).

From ℓTℓ+ℓnn2=ann\ell^T\ell + \ell_{nn}^2 = a_{nn}, we need ℓnn2=ann−ℓTℓ=ann−cT(An−1)−1c\ell_{nn}^2 = a_{nn} - \ell^T\ell = a_{nn} - c^T(A_{n-1})^{-1}c.

Positivity of ℓnn2\ell_{nn}^2. The quantity ann−cTAn−1−1ca_{nn} - c^T A_{n-1}^{-1} c is the Schur complement of An−1A_{n-1} in AA. For SPD AA, all Schur complements are positive definite. Concretely: for any α≠0\alpha \neq 0, let v=(−An−1−1cαα)v = \begin{pmatrix} -A_{n-1}^{-1}c\alpha \\ \alpha \end{pmatrix}. Then vTAv=α2(ann−cTAn−1−1c)>0v^T A v = \alpha^2(a_{nn} - c^T A_{n-1}^{-1}c) > 0 since AA is positive definite and v≠0v \neq 0.

Therefore ℓnn=ann−cTAn−1−1c>0\ell_{nn} = \sqrt{a_{nn} - c^T A_{n-1}^{-1}c} > 0, completing the existence proof.

Uniqueness. Suppose A=L1L1T=L2L2TA = L_1 L_1^T = L_2 L_2^T with L1,L2L_1, L_2 lower triangular with positive diagonals. Then L2−1L1=L2TL1−TL_2^{-1}L_1 = L_2^T L_1^{-T}. The left side is lower triangular and the right side is upper triangular, so both equal a diagonal matrix DD. From L1=L2DL_1 = L_2 D: A=L2DDTL2T=L2D2L2TA = L_2 D D^T L_2^T = L_2 D^2 L_2^T. Comparing with A=L2L2TA = L_2 L_2^T gives D2=ID^2 = I, so D=±ID = \pm I on each diagonal entry. Since L1L_1 and L2L_2 have positive diagonals, D=ID = I, hence L1=L2L_1 = L_2.

Explicit formula. The diagonal entries satisfy ℓii2=aii−∑k=1i−1ℓik2\ell_{ii}^2 = a_{ii} - \sum_{k=1}^{i-1} \ell_{ik}^2. Since aii>0a_{ii} > 0 (diagonal of SPD matrix) and the construction guarantees positivity at each step, ℓii>0\ell_{ii} > 0 for all ii. □\square

■

RemarkNumerical Stability

Cholesky factorization is backward stable without pivoting: L^L^T=A+E\hat{L}\hat{L}^T = A + E with ∣E∣≤(n+1)εmach∣L^∣∣L^T∣|E| \leq (n+1)\varepsilon_{\text{mach}} |\hat{L}||\hat{L}^T|. This is because the positive definiteness prevents the growth factors that necessitate pivoting in general LU. Moreover, attempting Cholesky on a non-SPD matrix will produce a non-positive value under the square root, providing a diagnostic for positive definiteness.

ExampleOperation Count

The Cholesky algorithm computes ℓij=(aij−∑k=1j−1ℓikℓjk)/ℓjj\ell_{ij} = (a_{ij} - \sum_{k=1}^{j-1}\ell_{ik}\ell_{jk})/\ell_{jj} for i>ji > j and ℓjj=(ajj−∑k=1j−1ℓjk2)1/2\ell_{jj} = (a_{jj} - \sum_{k=1}^{j-1}\ell_{jk}^2)^{1/2}. The dominant cost is ∑j=1n∑i=j+1n(j−1)≈n3/6\sum_{j=1}^n \sum_{i=j+1}^n (j-1) \approx n^3/6 multiplications, for a total of n3/3n^3/3 flops. This is exactly half the 2n3/32n^3/3 cost of LU factorization, since Cholesky exploits symmetry.