Orthogonal and Orthonormal Sets

Orthogonality is the generalization of perpendicularity to abstract inner product spaces. Orthonormal bases provide the most convenient coordinate systems.

DefinitionOrthogonal and Orthonormal Sets

A set $\{\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k\}$ in an inner product space is:

Orthogonal if $\langle \mathbf{v}_i, \mathbf{v}_j \rangle = 0$ for all $i \neq j$
Orthonormal if it is orthogonal and $\|\mathbf{v}_i\| = 1$ for all $i$

In other words, orthonormal means: $\langle \mathbf{v}_i, \mathbf{v}_j \rangle = \delta_{ij} = \begin{cases} 1 & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases}$

TheoremOrthogonal Sets are Independent

Any orthogonal set of nonzero vectors is linearly independent.

Proof: Suppose $c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0}$ . Taking inner product with $\mathbf{v}_i$ : $\langle c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k, \mathbf{v}_i \rangle = \langle \mathbf{0}, \mathbf{v}_i \rangle = 0$

By linearity and orthogonality: $c_i\langle \mathbf{v}_i, \mathbf{v}_i \rangle = 0$

Since $\mathbf{v}_i \neq \mathbf{0}$ , we have $\langle \mathbf{v}_i, \mathbf{v}_i \rangle > 0$ , so $c_i = 0$ . ∎

DefinitionOrthonormal Basis

An orthonormal basis for an inner product space $V$ is a basis consisting of orthonormal vectors.

If $\mathcal{B} = \{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ is an orthonormal basis, then any $\mathbf{v} \in V$ has the simple expansion: $\mathbf{v} = \sum_{i=1}^n \langle \mathbf{v}, \mathbf{e}_i \rangle \mathbf{e}_i$

The coordinates are simply the inner products: $c_i = \langle \mathbf{v}, \mathbf{e}_i \rangle$ .

ExampleComputing with Orthonormal Basis

The standard basis $\{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\}$ for $\mathbb{R}^3$ is orthonormal under the dot product.

For $\mathbf{v} = \begin{pmatrix} 2 \\ 3 \\ 1 \end{pmatrix}$ : $\mathbf{v} = 2\mathbf{e}_1 + 3\mathbf{e}_2 + 1\mathbf{e}_3$

The coefficients are $\langle \mathbf{v}, \mathbf{e}_1 \rangle = 2$ , $\langle \mathbf{v}, \mathbf{e}_2 \rangle = 3$ , $\langle \mathbf{v}, \mathbf{e}_3 \rangle = 1$ .

DefinitionOrthogonal Complement

For a subspace $W$ of inner product space $V$ , the orthogonal complement is: $W^\perp = \{\mathbf{v} \in V : \langle \mathbf{v}, \mathbf{w} \rangle = 0 \text{ for all } \mathbf{w} \in W\}$

$W^\perp$ is a subspace, and for finite-dimensional $V$ : $V = W \oplus W^\perp$ and $\dim(W) + \dim(W^\perp) = \dim(V)$ .

TheoremParseval's Identity

If $\{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ is an orthonormal basis for $V$ , then for any $\mathbf{v} \in V$ : $\|\mathbf{v}\|^2 = \sum_{i=1}^n |\langle \mathbf{v}, \mathbf{e}_i \rangle|^2$

This generalizes the Pythagorean theorem to $n$ dimensions.

Remark

Orthonormal bases are computationally ideal: finding coordinates requires only inner products (no solving systems), and the norm formula is a simple sum of squares. The Gram-Schmidt process (studied next) provides an algorithm to construct orthonormal bases from arbitrary bases.