ProofComplete

Proof of the Replacement Theorem

We prove the Replacement Theorem: if VV is spanned by a set GG with G=n|G| = n, and L={v1,,vm}L = \{v_1, \ldots, v_m\} is linearly independent, then mnm \leq n and there exists HGH \subseteq G with H=nm|H| = n - m such that span(LH)=V\text{span}(L \cup H) = V.


Proof by induction on mm

Proof

We induct on m=Lm = |L|.

Base case (m=0m = 0): L=L = \varnothing, and we take H=GH = G. Then H=n=n0|H| = n = n - 0 and span(G)=span(G)=V\text{span}(\varnothing \cup G) = \text{span}(G) = V.

Inductive step: Assume the result holds for m1m - 1 independent vectors. Suppose L={v1,,vm}L = \{v_1, \ldots, v_m\} is linearly independent.

By the inductive hypothesis applied to {v1,,vm1}\{v_1, \ldots, v_{m-1}\}, there exist m1m - 1 elements of GG that can be "replaced," leaving a subset HGH' \subseteq G with H=n(m1)=nm+1|H'| = n - (m-1) = n - m + 1 such that

span({v1,,vm1}H)=V.\text{span}(\{v_1, \ldots, v_{m-1}\} \cup H') = V.

Write H={g1,g2,,gnm+1}H' = \{g_1, g_2, \ldots, g_{n-m+1}\}.

Since vmV=span({v1,,vm1}H)v_m \in V = \text{span}(\{v_1, \ldots, v_{m-1}\} \cup H'), we can write

vm=a1v1++am1vm1+b1g1++bnm+1gnm+1.v_m = a_1 v_1 + \cdots + a_{m-1} v_{m-1} + b_1 g_1 + \cdots + b_{n-m+1} g_{n-m+1}.

Claim: At least one bj0b_j \neq 0.

Proof of claim: If all bj=0b_j = 0, then vm=a1v1++am1vm1v_m = a_1 v_1 + \cdots + a_{m-1} v_{m-1}, contradicting the linear independence of {v1,,vm}\{v_1, \ldots, v_m\}.

So some bj0b_j \neq 0. Without loss of generality, say b10b_1 \neq 0. Then we can solve for g1g_1:

g1=1b1(vma1v1am1vm1b2g2bnm+1gnm+1).g_1 = \frac{1}{b_1}\left(v_m - a_1 v_1 - \cdots - a_{m-1} v_{m-1} - b_2 g_2 - \cdots - b_{n-m+1} g_{n-m+1}\right).

This shows g1span({v1,,vm,g2,,gnm+1})g_1 \in \text{span}(\{v_1, \ldots, v_m, g_2, \ldots, g_{n-m+1}\}).

Therefore:

V=span({v1,,vm1}H)=span({v1,,vm,g2,,gnm+1}).V = \text{span}(\{v_1, \ldots, v_{m-1}\} \cup H') = \text{span}(\{v_1, \ldots, v_m, g_2, \ldots, g_{n-m+1}\}).

Set H={g2,,gnm+1}GH = \{g_2, \ldots, g_{n-m+1}\} \subseteq G with H=nm|H| = n - m. Then span(LH)=V\text{span}(L \cup H) = V.

Bounding mm: We need H=nm+11|H'| = n - m + 1 \geq 1, i.e., mnm \leq n. If m>nm > n, then at the step where H|H'| becomes 00, we would have V=span{v1,,vn}V = \text{span}\{v_1, \ldots, v_n\}, and vn+1v_{n+1} would be a linear combination of v1,,vnv_1, \ldots, v_n, contradicting independence.


Worked examples illustrating the proof

ExampleTracing the proof in R^3

G={e1,e2,e3}G = \{e_1, e_2, e_3\} spans R3\mathbb{R}^3, n=3n = 3. Let L={v1,v2}L = \{v_1, v_2\} with v1=(1,1,0)v_1 = (1, 1, 0), v2=(0,1,1)v_2 = (0, 1, 1).

Step 1 (m=1m = 1): We write v1=1e1+1e2+0e3v_1 = 1 \cdot e_1 + 1 \cdot e_2 + 0 \cdot e_3. The coefficient of e1e_1 is 101 \neq 0, so replace e1e_1:

R3=span{v1,e2,e3}.\mathbb{R}^3 = \text{span}\{v_1, e_2, e_3\}.

Step 2 (m=2m = 2): We write v2=(0,1,1)v_2 = (0, 1, 1) in terms of {v1,e2,e3}\{v_1, e_2, e_3\}:

v2=0v1+1e2+1e3.v_2 = 0 \cdot v_1 + 1 \cdot e_2 + 1 \cdot e_3.

We need a nonzero coefficient on e2e_2 or e3e_3. The coefficient of e2e_2 is 101 \neq 0, so replace e2e_2:

R3=span{v1,v2,e3}.\mathbb{R}^3 = \text{span}\{v_1, v_2, e_3\}.

Result: H={e3}H = \{e_3\}, H=1=32|H| = 1 = 3 - 2.

ExampleTracing the proof in P_2(R)

G={1,x,x2}G = \{1, x, x^2\} spans P2(R)P_2(\mathbb{R}), n=3n = 3. Let L={1+x+x2}L = \{1 + x + x^2\}, so m=1m = 1.

v1=1+x+x2=11+1x+1x2v_1 = 1 + x + x^2 = 1 \cdot 1 + 1 \cdot x + 1 \cdot x^2. All coefficients are nonzero. Choose to replace 11:

P2(R)=span{1+x+x2,x,x2}.P_2(\mathbb{R}) = \text{span}\{1 + x + x^2, x, x^2\}.

Indeed: 1=(1+x+x2)xx21 = (1 + x + x^2) - x - x^2.

H={x,x2}H = \{x, x^2\}, H=2=31|H| = 2 = 3 - 1.

ExampleWhy m > n is impossible

Try to find 4 independent vectors in R3\mathbb{R}^3. Say v1,v2,v3,v4v_1, v_2, v_3, v_4 are independent. After replacing 3 vectors from G={e1,e2,e3}G = \{e_1, e_2, e_3\}:

R3=span{v1,v2,v3}.\mathbb{R}^3 = \text{span}\{v_1, v_2, v_3\}.

Then v4span{v1,v2,v3}v_4 \in \text{span}\{v_1, v_2, v_3\}, contradicting independence of {v1,v2,v3,v4}\{v_1, v_2, v_3, v_4\}.

ExampleFreedom in choosing replacements

In the proof, when vm=aivi+bjgjv_m = \sum a_i v_i + \sum b_j g_j, we can replace any gjg_j with bj0b_j \neq 0. For v1=(1,2,3)v_1 = (1, 2, 3) and G={e1,e2,e3}G = \{e_1, e_2, e_3\}:

v1=1e1+2e2+3e3.v_1 = 1 \cdot e_1 + 2 \cdot e_2 + 3 \cdot e_3.

All three coefficients are nonzero, so we could replace any one of e1e_1, e2e_2, or e3e_3. Each choice gives a valid spanning set.

ExampleThe proof is constructive

Unlike many existence proofs in mathematics, the Replacement Theorem's proof is fully constructive: it provides an explicit algorithm to find HH. Process v1,v2,,vmv_1, v_2, \ldots, v_m one at a time, each time replacing one element of GG based on a nonzero coefficient. This algorithm is essentially Gaussian elimination.

ExampleCorollary: dimension is well-defined

If β1\beta_1 and β2\beta_2 are two bases for VV, then β1\beta_1 is independent and β2\beta_2 is spanning, so β1β2|\beta_1| \leq |\beta_2| by the Replacement Theorem. By symmetry, β2β1|\beta_2| \leq |\beta_1|. Hence β1=β2|\beta_1| = |\beta_2|.

This is the most important consequence: the number dimV\dim V does not depend on the choice of basis.

ExampleCorollary: maximal independent = basis

An independent set LL with L=dimV|L| = \dim V is a basis. The Replacement Theorem gives HGH \subseteq G with H=nn=0|H| = n - n = 0, so span(L)=V\text{span}(L) = V.

ExampleCorollary: minimal spanning = basis

A spanning set GG with G=dimV|G| = \dim V is a basis. If GG were dependent, we could remove an element and still span, giving a spanning set of size n1<nn - 1 < n. But any independent subset (e.g., a basis) has nn elements, contradicting nn1n \leq n - 1.

ExampleCorollary: independent sets extend to bases

If L={v1,,vm}L = \{v_1, \ldots, v_m\} is independent and m<n=dimVm < n = \dim V, then there exist g1,,gnmGg_1, \ldots, g_{n-m} \in G such that {v1,,vm,g1,,gnm}\{v_1, \ldots, v_m, g_1, \ldots, g_{n-m}\} is a basis for VV.

ExampleLattice-theoretic viewpoint

The Replacement Theorem can be seen as a statement about the lattice of subspaces: any maximal chain from {0}\{0\} to VV (a composition series) has the same length, dimV\dim V. This is a special case of the Jordan-Holder theorem in group theory.

ExampleInfinite-dimensional extension

In infinite-dimensional spaces, the analogous theorem holds with Zorn's lemma replacing induction. Any linearly independent set can be extended to a (Hamel) basis, and all bases have the same cardinality.

ExampleCorollary: subspace dimension bound

If WVW \subseteq V is a subspace of a finite-dimensional space VV, then dimWdimV\dim W \leq \dim V. Any basis of WW is an independent set in VV, and by the Replacement Theorem, its size cannot exceed G=dimV|G| = \dim V. Moreover, dimW=dimV\dim W = \dim V implies W=VW = V (extend a basis of WW -- but no extension is needed since it already has dimV\dim V vectors, so it spans VV).


RemarkSummary

The Replacement Theorem is the cornerstone of finite-dimensional linear algebra. It ensures:

  • Dimension is well-defined.
  • Independent sets are "small" (dimV\leq \dim V elements).
  • Spanning sets are "large" (dimV\geq \dim V elements).
  • Bases are precisely the independent sets (or spanning sets) of size dimV\dim V.