ProofComplete

Proof of Cholesky Factorization Existence

Every symmetric positive definite matrix admits a unique Cholesky factorization. This factorization is numerically stable without pivoting and costs half the operations of general LU decomposition.


Statement

Theorem5.3Cholesky Factorization Theorem

Let ARn×nA \in \mathbb{R}^{n \times n} be symmetric positive definite (SPD). Then there exists a unique lower triangular matrix LL with positive diagonal entries such that A=LLTA = LL^T. Moreover, ii=aiik=1i1ik2>0\ell_{ii} = \sqrt{a_{ii} - \sum_{k=1}^{i-1}\ell_{ik}^2} > 0 for all ii.


Proof

Proof

Existence by induction. For n=1n = 1: A=(a11)A = (a_{11}) with a11>0a_{11} > 0, so L=(a11)L = (\sqrt{a_{11}}).

For the inductive step, partition A=(An1ccTann)A = \begin{pmatrix} A_{n-1} & c \\ c^T & a_{nn} \end{pmatrix} where An1A_{n-1} is (n1)×(n1)(n-1) \times (n-1). Since any principal submatrix of an SPD matrix is SPD, An1A_{n-1} is SPD. By the induction hypothesis, An1=Ln1Ln1TA_{n-1} = L_{n-1} L_{n-1}^T.

Seek L=(Ln10Tnn)L = \begin{pmatrix} L_{n-1} & 0 \\ \ell^T & \ell_{nn} \end{pmatrix} so that LLT=(Ln1Ln1TLn1TLn1TT+nn2)=ALL^T = \begin{pmatrix} L_{n-1}L_{n-1}^T & L_{n-1}\ell \\ \ell^T L_{n-1}^T & \ell^T\ell + \ell_{nn}^2 \end{pmatrix} = A.

From Ln1=cL_{n-1}\ell = c, we get =Ln11c\ell = L_{n-1}^{-1}c (unique, since Ln1L_{n-1} is nonsingular).

From T+nn2=ann\ell^T\ell + \ell_{nn}^2 = a_{nn}, we need nn2=annT=anncT(An1)1c\ell_{nn}^2 = a_{nn} - \ell^T\ell = a_{nn} - c^T(A_{n-1})^{-1}c.

Positivity of nn2\ell_{nn}^2. The quantity anncTAn11ca_{nn} - c^T A_{n-1}^{-1} c is the Schur complement of An1A_{n-1} in AA. For SPD AA, all Schur complements are positive definite. Concretely: for any α0\alpha \neq 0, let v=(An11cαα)v = \begin{pmatrix} -A_{n-1}^{-1}c\alpha \\ \alpha \end{pmatrix}. Then vTAv=α2(anncTAn11c)>0v^T A v = \alpha^2(a_{nn} - c^T A_{n-1}^{-1}c) > 0 since AA is positive definite and v0v \neq 0.

Therefore nn=anncTAn11c>0\ell_{nn} = \sqrt{a_{nn} - c^T A_{n-1}^{-1}c} > 0, completing the existence proof.

Uniqueness. Suppose A=L1L1T=L2L2TA = L_1 L_1^T = L_2 L_2^T with L1,L2L_1, L_2 lower triangular with positive diagonals. Then L21L1=L2TL1TL_2^{-1}L_1 = L_2^T L_1^{-T}. The left side is lower triangular and the right side is upper triangular, so both equal a diagonal matrix DD. From L1=L2DL_1 = L_2 D: A=L2DDTL2T=L2D2L2TA = L_2 D D^T L_2^T = L_2 D^2 L_2^T. Comparing with A=L2L2TA = L_2 L_2^T gives D2=ID^2 = I, so D=±ID = \pm I on each diagonal entry. Since L1L_1 and L2L_2 have positive diagonals, D=ID = I, hence L1=L2L_1 = L_2.

Explicit formula. The diagonal entries satisfy ii2=aiik=1i1ik2\ell_{ii}^2 = a_{ii} - \sum_{k=1}^{i-1} \ell_{ik}^2. Since aii>0a_{ii} > 0 (diagonal of SPD matrix) and the construction guarantees positivity at each step, ii>0\ell_{ii} > 0 for all ii. \square


RemarkNumerical Stability

Cholesky factorization is backward stable without pivoting: L^L^T=A+E\hat{L}\hat{L}^T = A + E with E(n+1)εmachL^L^T|E| \leq (n+1)\varepsilon_{\text{mach}} |\hat{L}||\hat{L}^T|. This is because the positive definiteness prevents the growth factors that necessitate pivoting in general LU. Moreover, attempting Cholesky on a non-SPD matrix will produce a non-positive value under the square root, providing a diagnostic for positive definiteness.

ExampleOperation Count

The Cholesky algorithm computes ij=(aijk=1j1ikjk)/jj\ell_{ij} = (a_{ij} - \sum_{k=1}^{j-1}\ell_{ik}\ell_{jk})/\ell_{jj} for i>ji > j and jj=(ajjk=1j1jk2)1/2\ell_{jj} = (a_{jj} - \sum_{k=1}^{j-1}\ell_{jk}^2)^{1/2}. The dominant cost is j=1ni=j+1n(j1)n3/6\sum_{j=1}^n \sum_{i=j+1}^n (j-1) \approx n^3/6 multiplications, for a total of n3/3n^3/3 flops. This is exactly half the 2n3/32n^3/3 cost of LU factorization, since Cholesky exploits symmetry.