The Spectral Theorem
The spectral theorem is the crown jewel of linear algebra, establishing that symmetric matrices have complete orthonormal eigenvector bases. This result underlies countless applications across mathematics and science.
Let be a real symmetric matrix. Then:
- All eigenvalues of are real
- Eigenvectors corresponding to distinct eigenvalues are orthogonal
- has an orthonormal basis consisting of eigenvectors of
- is orthogonally diagonalizable: where:
- is orthogonal with
- contains the eigenvalues
- Columns of are orthonormal eigenvectors
This decomposition is unique up to reordering eigenvalues and sign choices for eigenvectors.
Let be an Hermitian matrix (i.e., ). Then:
- All eigenvalues of are real
- has an orthonormal basis of eigenvectors
- is unitarily diagonalizable: where is unitary and is real diagonal
The spectral theorem guarantees maximal diagonalizability: unlike general matrices (which may require Jordan form), symmetric/Hermitian matrices always have enough eigenvectors for a complete orthonormal basis.
Two symmetric matrices and are simultaneously diagonalizable by the same orthogonal matrix if and only if (they commute).
If , there exists orthogonal such that both and are diagonal.
Given data matrix (mean-centered), the covariance matrix is , which is symmetric PSD.
The spectral theorem gives where:
- Eigenvectors (columns of ) are the principal components—directions of maximum variance
- Eigenvalues measure variance along each principal component
- The transformation rotates data to principal axes
PCA reduces dimensionality by keeping only eigenvectors with largest eigenvalues.
For symmetric matrix and nonzero , the Rayleigh quotient is:
Let and be the smallest and largest eigenvalues. Then:
Moreover, achieves its maximum when is the corresponding eigenvector.
The spectral theorem is foundational because it reveals the geometric structure of symmetric matrices: they are exactly those linear transformations that can be understood purely through orthogonal scaling along perpendicular axes. This decomposition appears everywhere: in vibrating systems (normal modes), quantum mechanics (observables), statistics (decorrelation), and optimization (quadratic forms). The theorem's elegance—that symmetry implies such strong spectral properties—is one of the most beautiful results in mathematics.