Symmetric and Hermitian Matrices

Symmetric and Hermitian matrices possess exceptional spectral properties: real eigenvalues and orthogonal eigenvectors. These properties make them ubiquitous in applications from physics to statistics.

DefinitionSymmetric Matrix

A real $n \times n$ matrix $A$ is symmetric if $A^T = A$ , i.e., $a_{ij} = a_{ji}$ for all $i, j$ .

A complex $n \times n$ matrix $A$ is Hermitian (or self-adjoint) if $A^* = A$ , where $A^* = \overline{A^T}$ is the conjugate transpose. For Hermitian matrices, $a_{ij} = \overline{a_{ji}}$ .

Symmetric matrices are the real case of Hermitian matrices. The spectral theorem establishes that these matrices have especially nice eigenvalue decompositions.

ExampleSymmetric Matrices

$A = \begin{bmatrix} 2 & 1 & 0 \\ 1 & 3 & -1 \\ 0 & -1 & 2 \end{bmatrix}$ is symmetric since $A^T = A$ .

$H = \begin{bmatrix} 2 & 1+i \\ 1-i & 3 \end{bmatrix}$ is Hermitian since $h_{12} = 1+i = \overline{h_{21}} = \overline{1-i}$ .

TheoremReal Eigenvalues of Symmetric Matrices

Every eigenvalue of a real symmetric matrix (or complex Hermitian matrix) is real.

Proof sketch: Let $A\mathbf{v} = \lambda\mathbf{v}$ with $\mathbf{v} \neq \mathbf{0}$ . Take conjugate transpose: $\mathbf{v}^*A^*\mathbf{v} = \overline{\lambda}\mathbf{v}^*\mathbf{v}$

Since $A^* = A$ : $\mathbf{v}^*A\mathbf{v} = \overline{\lambda}\mathbf{v}^*\mathbf{v}$

But also $\mathbf{v}^*A\mathbf{v} = \mathbf{v}^*(\lambda\mathbf{v}) = \lambda\mathbf{v}^*\mathbf{v}$ .

Thus $\lambda\mathbf{v}^*\mathbf{v} = \overline{\lambda}\mathbf{v}^*\mathbf{v}$ . Since $\mathbf{v}^*\mathbf{v} = \|\mathbf{v}\|^2 > 0$ , we have $\lambda = \overline{\lambda}$ , so $\lambda \in \mathbb{R}$ . ∎

TheoremOrthogonality of Eigenvectors

Eigenvectors of a symmetric matrix corresponding to distinct eigenvalues are orthogonal.

Proof: Let $A\mathbf{v}_1 = \lambda_1\mathbf{v}_1$ and $A\mathbf{v}_2 = \lambda_2\mathbf{v}_2$ with $\lambda_1 \neq \lambda_2$ .

Compute: $\langle A\mathbf{v}_1, \mathbf{v}_2 \rangle = \langle \lambda_1\mathbf{v}_1, \mathbf{v}_2 \rangle = \lambda_1\langle \mathbf{v}_1, \mathbf{v}_2 \rangle$

Also: $\langle A\mathbf{v}_1, \mathbf{v}_2 \rangle = \langle \mathbf{v}_1, A^T\mathbf{v}_2 \rangle = \langle \mathbf{v}_1, A\mathbf{v}_2 \rangle = \lambda_2\langle \mathbf{v}_1, \mathbf{v}_2 \rangle$

Thus $(\lambda_1 - \lambda_2)\langle \mathbf{v}_1, \mathbf{v}_2 \rangle = 0$ . Since $\lambda_1 \neq \lambda_2$ , we have $\langle \mathbf{v}_1, \mathbf{v}_2 \rangle = 0$ . ∎

DefinitionOrthogonal and Unitary Matrices

A real matrix $Q$ is orthogonal if $Q^TQ = I$ , equivalently $Q^T = Q^{-1}$ .

A complex matrix $U$ is unitary if $U^*U = I$ , equivalently $U^* = U^{-1}$ .

Orthogonal/unitary matrices preserve inner products and norms: $\langle Q\mathbf{x}, Q\mathbf{y} \rangle = \langle \mathbf{x}, \mathbf{y} \rangle$ and $\|Q\mathbf{x}\| = \|\mathbf{x}\|$ .

Remark

The reality of eigenvalues and orthogonality of eigenvectors make symmetric matrices ideal for applications. In physics, observables in quantum mechanics are represented by Hermitian operators. In statistics, covariance matrices are symmetric positive semi-definite. The spectral theorem formalizes these special properties.