ProofComplete

Proof of the Replacement Theorem

We prove the Replacement Theorem: if VV is spanned by a set GG with ∣G∣=n|G| = n, and L={v1,…,vm}L = \{v_1, \ldots, v_m\} is linearly independent, then m≤nm \leq n and there exists H⊆GH \subseteq G with ∣H∣=n−m|H| = n - m such that span(L∪H)=V\text{span}(L \cup H) = V.


Proof by induction on mm

Proof

We induct on m=∣L∣m = |L|.

Base case (m=0m = 0): L=∅L = \varnothing, and we take H=GH = G. Then ∣H∣=n=n−0|H| = n = n - 0 and span(∅∪G)=span(G)=V\text{span}(\varnothing \cup G) = \text{span}(G) = V.

Inductive step: Assume the result holds for m−1m - 1 independent vectors. Suppose L={v1,…,vm}L = \{v_1, \ldots, v_m\} is linearly independent.

By the inductive hypothesis applied to {v1,…,vm−1}\{v_1, \ldots, v_{m-1}\}, there exist m−1m - 1 elements of GG that can be "replaced," leaving a subset H′⊆GH' \subseteq G with ∣H′∣=n−(m−1)=n−m+1|H'| = n - (m-1) = n - m + 1 such that

span({v1,…,vm−1}∪H′)=V.\text{span}(\{v_1, \ldots, v_{m-1}\} \cup H') = V.

Write H′={g1,g2,…,gn−m+1}H' = \{g_1, g_2, \ldots, g_{n-m+1}\}.

Since vm∈V=span({v1,…,vm−1}∪H′)v_m \in V = \text{span}(\{v_1, \ldots, v_{m-1}\} \cup H'), we can write

vm=a1v1+⋯+am−1vm−1+b1g1+⋯+bn−m+1gn−m+1.v_m = a_1 v_1 + \cdots + a_{m-1} v_{m-1} + b_1 g_1 + \cdots + b_{n-m+1} g_{n-m+1}.

Claim: At least one bj≠0b_j \neq 0.

Proof of claim: If all bj=0b_j = 0, then vm=a1v1+⋯+am−1vm−1v_m = a_1 v_1 + \cdots + a_{m-1} v_{m-1}, contradicting the linear independence of {v1,…,vm}\{v_1, \ldots, v_m\}.

So some bj≠0b_j \neq 0. Without loss of generality, say b1≠0b_1 \neq 0. Then we can solve for g1g_1:

g1=1b1(vm−a1v1−⋯−am−1vm−1−b2g2−⋯−bn−m+1gn−m+1).g_1 = \frac{1}{b_1}\left(v_m - a_1 v_1 - \cdots - a_{m-1} v_{m-1} - b_2 g_2 - \cdots - b_{n-m+1} g_{n-m+1}\right).

This shows g1∈span({v1,…,vm,g2,…,gn−m+1})g_1 \in \text{span}(\{v_1, \ldots, v_m, g_2, \ldots, g_{n-m+1}\}).

Therefore:

V=span({v1,…,vm−1}∪H′)=span({v1,…,vm,g2,…,gn−m+1}).V = \text{span}(\{v_1, \ldots, v_{m-1}\} \cup H') = \text{span}(\{v_1, \ldots, v_m, g_2, \ldots, g_{n-m+1}\}).

Set H={g2,…,gn−m+1}⊆GH = \{g_2, \ldots, g_{n-m+1}\} \subseteq G with ∣H∣=n−m|H| = n - m. Then span(L∪H)=V\text{span}(L \cup H) = V.

Bounding mm: We need ∣H′∣=n−m+1≥1|H'| = n - m + 1 \geq 1, i.e., m≤nm \leq n. If m>nm > n, then at the step where ∣H′∣|H'| becomes 00, we would have V=span{v1,…,vn}V = \text{span}\{v_1, \ldots, v_n\}, and vn+1v_{n+1} would be a linear combination of v1,…,vnv_1, \ldots, v_n, contradicting independence.

■

Worked examples illustrating the proof

ExampleTracing the proof in R^3

G={e1,e2,e3}G = \{e_1, e_2, e_3\} spans R3\mathbb{R}^3, n=3n = 3. Let L={v1,v2}L = \{v_1, v_2\} with v1=(1,1,0)v_1 = (1, 1, 0), v2=(0,1,1)v_2 = (0, 1, 1).

Step 1 (m=1m = 1): We write v1=1⋅e1+1⋅e2+0⋅e3v_1 = 1 \cdot e_1 + 1 \cdot e_2 + 0 \cdot e_3. The coefficient of e1e_1 is 1≠01 \neq 0, so replace e1e_1:

R3=span{v1,e2,e3}.\mathbb{R}^3 = \text{span}\{v_1, e_2, e_3\}.

Step 2 (m=2m = 2): We write v2=(0,1,1)v_2 = (0, 1, 1) in terms of {v1,e2,e3}\{v_1, e_2, e_3\}:

v2=0⋅v1+1⋅e2+1⋅e3.v_2 = 0 \cdot v_1 + 1 \cdot e_2 + 1 \cdot e_3.

We need a nonzero coefficient on e2e_2 or e3e_3. The coefficient of e2e_2 is 1≠01 \neq 0, so replace e2e_2:

R3=span{v1,v2,e3}.\mathbb{R}^3 = \text{span}\{v_1, v_2, e_3\}.

Result: H={e3}H = \{e_3\}, ∣H∣=1=3−2|H| = 1 = 3 - 2.

ExampleTracing the proof in P_2(R)

G={1,x,x2}G = \{1, x, x^2\} spans P2(R)P_2(\mathbb{R}), n=3n = 3. Let L={1+x+x2}L = \{1 + x + x^2\}, so m=1m = 1.

v1=1+x+x2=1⋅1+1⋅x+1⋅x2v_1 = 1 + x + x^2 = 1 \cdot 1 + 1 \cdot x + 1 \cdot x^2. All coefficients are nonzero. Choose to replace 11:

P2(R)=span{1+x+x2,x,x2}.P_2(\mathbb{R}) = \text{span}\{1 + x + x^2, x, x^2\}.

Indeed: 1=(1+x+x2)−x−x21 = (1 + x + x^2) - x - x^2.

H={x,x2}H = \{x, x^2\}, ∣H∣=2=3−1|H| = 2 = 3 - 1.

ExampleWhy m > n is impossible

Try to find 4 independent vectors in R3\mathbb{R}^3. Say v1,v2,v3,v4v_1, v_2, v_3, v_4 are independent. After replacing 3 vectors from G={e1,e2,e3}G = \{e_1, e_2, e_3\}:

R3=span{v1,v2,v3}.\mathbb{R}^3 = \text{span}\{v_1, v_2, v_3\}.

Then v4∈span{v1,v2,v3}v_4 \in \text{span}\{v_1, v_2, v_3\}, contradicting independence of {v1,v2,v3,v4}\{v_1, v_2, v_3, v_4\}.

ExampleFreedom in choosing replacements

In the proof, when vm=∑aivi+∑bjgjv_m = \sum a_i v_i + \sum b_j g_j, we can replace any gjg_j with bj≠0b_j \neq 0. For v1=(1,2,3)v_1 = (1, 2, 3) and G={e1,e2,e3}G = \{e_1, e_2, e_3\}:

v1=1⋅e1+2⋅e2+3⋅e3.v_1 = 1 \cdot e_1 + 2 \cdot e_2 + 3 \cdot e_3.

All three coefficients are nonzero, so we could replace any one of e1e_1, e2e_2, or e3e_3. Each choice gives a valid spanning set.

ExampleThe proof is constructive

Unlike many existence proofs in mathematics, the Replacement Theorem's proof is fully constructive: it provides an explicit algorithm to find HH. Process v1,v2,…,vmv_1, v_2, \ldots, v_m one at a time, each time replacing one element of GG based on a nonzero coefficient. This algorithm is essentially Gaussian elimination.

ExampleCorollary: dimension is well-defined

If β1\beta_1 and β2\beta_2 are two bases for VV, then β1\beta_1 is independent and β2\beta_2 is spanning, so ∣β1∣≤∣β2∣|\beta_1| \leq |\beta_2| by the Replacement Theorem. By symmetry, ∣β2∣≤∣β1∣|\beta_2| \leq |\beta_1|. Hence ∣β1∣=∣β2∣|\beta_1| = |\beta_2|.

This is the most important consequence: the number dim⁡V\dim V does not depend on the choice of basis.

ExampleCorollary: maximal independent = basis

An independent set LL with ∣L∣=dim⁡V|L| = \dim V is a basis. The Replacement Theorem gives H⊆GH \subseteq G with ∣H∣=n−n=0|H| = n - n = 0, so span(L)=V\text{span}(L) = V.

ExampleCorollary: minimal spanning = basis

A spanning set GG with ∣G∣=dim⁡V|G| = \dim V is a basis. If GG were dependent, we could remove an element and still span, giving a spanning set of size n−1<nn - 1 < n. But any independent subset (e.g., a basis) has nn elements, contradicting n≤n−1n \leq n - 1.

ExampleCorollary: independent sets extend to bases

If L={v1,…,vm}L = \{v_1, \ldots, v_m\} is independent and m<n=dim⁡Vm < n = \dim V, then there exist g1,…,gn−m∈Gg_1, \ldots, g_{n-m} \in G such that {v1,…,vm,g1,…,gn−m}\{v_1, \ldots, v_m, g_1, \ldots, g_{n-m}\} is a basis for VV.

ExampleLattice-theoretic viewpoint

The Replacement Theorem can be seen as a statement about the lattice of subspaces: any maximal chain from {0}\{0\} to VV (a composition series) has the same length, dim⁡V\dim V. This is a special case of the Jordan-Holder theorem in group theory.

ExampleInfinite-dimensional extension

In infinite-dimensional spaces, the analogous theorem holds with Zorn's lemma replacing induction. Any linearly independent set can be extended to a (Hamel) basis, and all bases have the same cardinality.

ExampleCorollary: subspace dimension bound

If W⊆VW \subseteq V is a subspace of a finite-dimensional space VV, then dim⁡W≤dim⁡V\dim W \leq \dim V. Any basis of WW is an independent set in VV, and by the Replacement Theorem, its size cannot exceed ∣G∣=dim⁡V|G| = \dim V. Moreover, dim⁡W=dim⁡V\dim W = \dim V implies W=VW = V (extend a basis of WW -- but no extension is needed since it already has dim⁡V\dim V vectors, so it spans VV).


RemarkSummary

The Replacement Theorem is the cornerstone of finite-dimensional linear algebra. It ensures:

  • Dimension is well-defined.
  • Independent sets are "small" (≤dim⁡V\leq \dim V elements).
  • Spanning sets are "large" (≥dim⁡V\geq \dim V elements).
  • Bases are precisely the independent sets (or spanning sets) of size dim⁡V\dim V.