Mathematics Lecture Notes

Theorem8.5Spectral Theorem (real symmetric case)

Let $A \in M_{n \times n}(\mathbb{R})$ with $A^T = A$ . Then there exists an orthogonal matrix $Q$ ( $Q^TQ = I$ ) such that $Q^TAQ = \operatorname{diag}(\lambda_1, \ldots, \lambda_n)$ , where all $\lambda_i \in \mathbb{R}$ .

ProofLemma 1: Eigenvalues of symmetric matrices are real

Let $A = A^T$ (with real entries) and $Av = \lambda v$ for some $v \in \mathbb{C}^n$ , $v \neq 0$ . We show $\lambda \in \mathbb{R}$ .

Consider $v^* A v$ (where $v^* = \bar{v}^T$ ):

$v^* A v = v^* (\lambda v) = \lambda (v^* v) = \lambda \|v\|^2.$

Also, since $A = A^T = \bar{A}$ (real matrix) and $A^* = \bar{A}^T = A^T = A$ :

$v^* A v = v^* A^* v = (Av)^* v = (\lambda v)^* v = \bar{\lambda} (v^* v) = \bar{\lambda} \|v\|^2.$

Comparing: $\lambda \|v\|^2 = \bar{\lambda} \|v\|^2$ . Since $\|v\|^2 > 0$ : $\lambda = \bar{\lambda}$ , so $\lambda \in \mathbb{R}$ . $\blacksquare$

■

ExampleVerifying real eigenvalues

$A = \begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}$ : $p(\lambda) = \lambda^2 - 5\lambda = \lambda(\lambda - 5)$ . Eigenvalues: $0, 5$ . Both real ✓.

$A = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$ : eigenvalues $1, -1$ . Real ✓.

Compare with the non-symmetric $B = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ ( $B^T \neq B$ ): eigenvalues $\pm i$ . Not real. Symmetry is essential.

ProofLemma 2: Orthogonality of eigenvectors for distinct eigenvalues

Let $A = A^T$ , $Av_1 = \lambda_1 v_1$ , $Av_2 = \lambda_2 v_2$ , $\lambda_1 \neq \lambda_2$ .

$\lambda_1 (v_1 \cdot v_2) = (Av_1) \cdot v_2 = v_1^T A^T v_2 = v_1^T A v_2 = v_1 \cdot (Av_2) = \lambda_2 (v_1 \cdot v_2).$

So $(\lambda_1 - \lambda_2)(v_1 \cdot v_2) = 0$ . Since $\lambda_1 \neq \lambda_2$ : $v_1 \cdot v_2 = 0$ . $\blacksquare$

■

ExampleChecking orthogonality

$A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ , eigenvalues $3, 1$ , eigenvectors $(1,1)$ and $(1,-1)$ .

$(1,1) \cdot (1,-1) = 0$ ✓ (orthogonal).

ProofLemma 3: Eigenspaces of symmetric matrices are invariant

Let $A = A^T$ and $\lambda$ be an eigenvalue. Then $E_\lambda^\perp$ is also invariant under $A$ .

If $v \in E_\lambda^\perp$ (meaning $v \cdot w = 0$ for all $w \in E_\lambda$ ), then for any $w \in E_\lambda$ :

$(Av) \cdot w = v \cdot (A^Tw) = v \cdot (Aw) = v \cdot (\lambda w) = \lambda (v \cdot w) = 0.$

So $Av \in E_\lambda^\perp$ . This means $A$ maps $E_\lambda^\perp$ to itself, so $A$ restricts to a linear map on $E_\lambda^\perp$ . $\blacksquare$

■

ExampleChecking invariance of orthogonal complement

$A = \begin{pmatrix} 3 & 1 & 0 \\ 1 & 3 & 0 \\ 0 & 0 & 5 \end{pmatrix}$ , eigenvalue $\lambda = 5$ with $E_5 = \operatorname{span}\{(0,0,1)\}$ .

$E_5^\perp = \operatorname{span}\{(1,0,0), (0,1,0)\}$ (the $xy$ -plane).

$A(1,0,0) = (3,1,0) \in E_5^\perp$ ✓ and $A(0,1,0) = (1,3,0) \in E_5^\perp$ ✓. The restriction is $\begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}$ (still symmetric).

ProofProof of the Spectral Theorem by induction

Induction on $n$ . The case $n = 1$ is trivial ( $A = (\lambda_1)$ is already diagonal).

Inductive step. Assume the theorem holds for all symmetric matrices of size less than $n$ . Let $A \in M_{n \times n}(\mathbb{R})$ be symmetric.

Step 1: Find an eigenvalue. The characteristic polynomial $p_A(\lambda) = \det(A - \lambda I)$ is a real polynomial of degree $n$ . Over $\mathbb{C}$ , it has at least one root $\lambda_1$ . By Lemma 1, $\lambda_1 \in \mathbb{R}$ .

Step 2: Find an eigenvector. Since $\lambda_1$ is real, $(A - \lambda_1 I)v = 0$ has a nonzero real solution $v_1 \in \mathbb{R}^n$ . Normalize: $q_1 = v_1 / \|v_1\|$ , so $\|q_1\| = 1$ .

Step 3: Reduce to a smaller matrix. Let $W = \operatorname{span}\{q_1\}^\perp = \{v \in \mathbb{R}^n : v \cdot q_1 = 0\}$ , which has dimension $n - 1$ .

By Lemma 3, $A$ maps $W$ to itself: if $v \in W$ , then $Av \in W$ . The restriction $A|_W: W \to W$ is a linear operator on an $(n-1)$ -dimensional space, and it is symmetric with respect to the standard inner product restricted to $W$ .

Step 4: Choose a basis for $W$ . Extend $q_1$ to an orthonormal basis $\{q_1, w_2, \ldots, w_n\}$ of $\mathbb{R}^n$ (using Gram--Schmidt on any extension). Let $\tilde{Q}_1 = (q_1 \mid w_2 \mid \cdots \mid w_n)$ . Then:

$\tilde{Q}_1^T A \tilde{Q}_1 = \begin{pmatrix} \lambda_1 & 0 \\ 0 & A' \end{pmatrix},$

where $A' \in M_{(n-1) \times (n-1)}(\mathbb{R})$ is the matrix of $A|_W$ with respect to $\{w_2, \ldots, w_n\}$ . The zeros in the first row and column arise because $Aq_1 = \lambda_1 q_1$ and $q_1 \perp w_j$ for $j \geq 2$ :

$(\tilde{Q}_1^T A \tilde{Q}_1)_{1j} = q_1^T A w_j = (Aq_1)^T w_j = \lambda_1 q_1^T w_j = 0$ for $j \geq 2$ .

By symmetry: $(\tilde{Q}_1^T A \tilde{Q}_1)_{j1} = 0$ for $j \geq 2$ .

Step 5: Apply induction. $A'$ is symmetric (since $A$ is, and the restriction preserves symmetry). By the inductive hypothesis, there exists an orthogonal matrix $Q'$ such that $(Q')^T A' Q' = \operatorname{diag}(\lambda_2, \ldots, \lambda_n)$ .

Step 6: Combine. Let $Q_2 = \begin{pmatrix} 1 & 0 \\ 0 & Q' \end{pmatrix}$ (orthogonal). Then $Q = \tilde{Q}_1 Q_2$ is orthogonal and:

$Q^T A Q = Q_2^T \begin{pmatrix} \lambda_1 & 0 \\ 0 & A' \end{pmatrix} Q_2 = \begin{pmatrix} \lambda_1 & 0 \\ 0 & (Q')^T A' Q' \end{pmatrix} = \operatorname{diag}(\lambda_1, \ldots, \lambda_n). \quad \blacksquare$

■

ExampleWalking through the proof for 3x3

$A = \begin{pmatrix} 2 & 1 & 0 \\ 1 & 3 & 1 \\ 0 & 1 & 2 \end{pmatrix}$ .

Step 1-2: Characteristic polynomial: $p(\lambda) = -\lambda^3 + 7\lambda^2 - 14\lambda + 8 = -(\lambda - 1)(\lambda - 2)(\lambda - 4)$ .

Eigenvalues: $1, 2, 4$ . Eigenvector for $\lambda = 4$ : solve $(A - 4I)v = 0$ . $(A-4I) = \begin{pmatrix} -2 & 1 & 0 \\ 1 & -1 & 1 \\ 0 & 1 & -2 \end{pmatrix}$ . Row reduce to get $v_1 = (1, 2, 1)$ , $q_1 = \frac{1}{\sqrt{6}}(1, 2, 1)$ .

Step 3-4: $W = q_1^\perp$ has dimension $2$ . Find ONB for $W$ : $w_2 = \frac{1}{\sqrt{2}}(1, 0, -1)$ , $w_3 = \frac{1}{\sqrt{3}}(-1, 1, -1)$ (orthogonal to $q_1$ and to each other).

$\tilde{Q}_1 = \begin{pmatrix} 1/\sqrt{6} & 1/\sqrt{2} & -1/\sqrt{3} \\ 2/\sqrt{6} & 0 & 1/\sqrt{3} \\ 1/\sqrt{6} & -1/\sqrt{2} & -1/\sqrt{3} \end{pmatrix}$ .

$\tilde{Q}_1^T A \tilde{Q}_1 = \begin{pmatrix} 4 & 0 & 0 \\ 0 & a' & b' \\ 0 & b' & c' \end{pmatrix}$ where $A' = \begin{pmatrix} a' & b' \\ b' & c' \end{pmatrix}$ is the restriction.

Computing $A'$ : $w_2^T A w_2 = \frac{1}{2}(1,0,-1)\begin{pmatrix} 2 \\ 2 \\ -1 \end{pmatrix} = \frac{1}{2}(2+1) = \frac{3}{2}$ ... let me just note the eigenvalues of $A'$ must be $1$ and $2$ (the remaining eigenvalues of $A$ ).

Step 5-6: Diagonalize $A'$ (a $2 \times 2$ symmetric matrix) to get $Q'$ , then combine.

ProofAlternative: via Schur decomposition

Every complex matrix $A$ has a Schur decomposition: $A = UTU^*$ where $U$ is unitary and $T$ is upper triangular with the eigenvalues on the diagonal.

If $A = A^*$ (Hermitian), then $T = U^*AU$ satisfies $T^* = (U^*AU)^* = U^*A^*U = U^*AU = T$ . An upper triangular matrix that is also Hermitian must be diagonal (the off-diagonal entries $t_{ij}$ with $i < j$ must equal $\overline{t_{ji}} = 0$ ). So $T = \Lambda = \operatorname{diag}(\lambda_1, \ldots, \lambda_n)$ .

For the real case: the real Schur decomposition gives $A = QTQ^T$ with $Q$ orthogonal and $T$ quasi-upper-triangular (block upper triangular with $1 \times 1$ and $2 \times 2$ blocks on the diagonal). If $A = A^T$ , symmetry forces $T$ to be symmetric, hence diagonal. So $A = Q\Lambda Q^T$ . $\blacksquare$

■

ExampleSchur decomposition of a symmetric matrix is diagonal

$A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ .

Schur decomposition over $\mathbb{R}$ : since $A$ is symmetric, the Schur form $T$ is diagonal. $T = \begin{pmatrix} 3 & 0 \\ 0 & 1 \end{pmatrix}$ with $Q = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}$ .

Compare with a non-symmetric matrix $B = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}$ : the Schur form is $T = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}$ (still upper triangular, not diagonal).

ExampleFull verification for 2x2

$A = \begin{pmatrix} 4 & 2 \\ 2 & 1 \end{pmatrix}$ , eigenvalues $5, 0$ .

$v_1 = (2, 1)$ for $\lambda = 5$ , $v_2 = (-1, 2)$ for $\lambda = 0$ .

$Q = \frac{1}{\sqrt{5}}\begin{pmatrix} 2 & -1 \\ 1 & 2 \end{pmatrix}$ , $Q^TQ = I$ ✓.

$Q^TAQ = \frac{1}{5}\begin{pmatrix} 2 & 1 \\ -1 & 2 \end{pmatrix}\begin{pmatrix} 4 & 2 \\ 2 & 1 \end{pmatrix}\begin{pmatrix} 2 & -1 \\ 1 & 2 \end{pmatrix} = \begin{pmatrix} 5 & 0 \\ 0 & 0 \end{pmatrix}$ ✓.

ExampleVerification with repeated eigenvalue

$A = \begin{pmatrix} 3 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 5 \end{pmatrix}$ : eigenvalue $3$ has $m_a = 2$ , $E_3 = \operatorname{span}\{e_1, e_2\}$ . Any ONB of $E_3$ works (e.g., $e_1, e_2$ ). $Q = I$ , $Q^TAQ = A = \operatorname{diag}(3, 3, 5)$ ✓.

A non-diagonal example with repeated eigenvalue: $A = \begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{pmatrix}$ . Eigenvalues: $4$ (once) and $1$ (twice). Eigenvector for $4$ : $(1,1,1)/\sqrt{3}$ . Eigenspace for $1$ : $\{(x,y,z) : x+y+z = 0\}$ , a $2$ -dimensional space. Choose ONB: $(1,-1,0)/\sqrt{2}$ and $(1,1,-2)/\sqrt{6}$ .

ExampleVerification for a negative definite matrix

$A = \begin{pmatrix} -5 & 2 \\ 2 & -2 \end{pmatrix}$ . Eigenvalues: $\frac{-7 \pm \sqrt{9+16}}{2} = \frac{-7 \pm 5}{2}$ . So $\lambda_1 = -1$ and $\lambda_2 = -6$ . Both negative (negative definite ✓).

Eigenvectors: $(2, 1)/\sqrt{5}$ for $\lambda = -1$ and $(-1, 2)/\sqrt{5}$ for $\lambda = -6$ .

$Q = \frac{1}{\sqrt{5}}\begin{pmatrix} 2 & -1 \\ 1 & 2 \end{pmatrix}$ , $Q^TAQ = \operatorname{diag}(-1, -6)$ ✓.

ExampleSpectral theorem for a 4x4 matrix

$A = I + ee^T$ where $e = (1, 1, 1, 1)^T$ . Then $A = I + ee^T$ has eigenvalues: $1$ (with multiplicity $3$ , eigenspace $e^\perp$ ) and $5$ (with multiplicity $1$ , eigenspace $\operatorname{span}\{e\}$ ).

$\operatorname{diag}(5, 1, 1, 1)$ in the basis $\{e/2, \ldots\}$ (any ONB starting with $e/2$ ).

ExampleOrthogonal diagonalization fails without symmetry

$A = \begin{pmatrix} 1 & 1 \\ 0 & 2 \end{pmatrix}$ : not symmetric. Eigenvalues $1, 2$ , eigenvectors $(1, 0)$ and $(1, 1)$ .

$(1, 0) \cdot (1, 1) = 1 \neq 0$ . The eigenvectors are not orthogonal.

$A$ is diagonalizable (distinct eigenvalues), but not orthogonally diagonalizable. Orthogonal diagonalizability requires $A = A^T$ .

ExampleNon-diagonalizable non-symmetric matrix

$A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ : $A^T \neq A$ , and $A$ is not even diagonalizable (single eigenvalue $1$ with $m_g = 1$ ). This cannot happen for symmetric matrices -- the spectral theorem guarantees full diagonalizability.

RemarkTwo proofs, one theorem

The Spectral Theorem admits two clean proofs:

Inductive proof: Peel off one eigenvector at a time. The key is that the orthogonal complement of an eigenspace is invariant under a symmetric operator, allowing the induction to proceed.

Schur decomposition proof: Start with the existence of the Schur form (upper triangular, unitarily similar to $A$ ). Symmetry forces the upper triangular matrix to be diagonal.

Both proofs use the same essential fact: real eigenvalues (from $\lambda = \bar{\lambda}$ ) and orthogonality of eigenspaces (from $A = A^T$ ). The result is the most perfect form a matrix can take: a real diagonal matrix in an orthonormal eigenbasis.

Math Notes

Proof of Spectral Theorem

Statement

Preliminary lemmas

Main proof

Worked example of the inductive proof

Alternative proof via Schur decomposition

Verification examples

Why the theorem fails for non-symmetric matrices

Summary