ConceptComplete

Linear Independence

Linear independence captures the idea that no vector in a set is redundant -- none can be written as a linear combination of the others.


Definition

Definition1.7Linear independence

A set of vectors {v1,v2,,vn}\{v_1, v_2, \ldots, v_n\} in a vector space VV over FF is linearly independent if the equation

a1v1+a2v2++anvn=0a_1 v_1 + a_2 v_2 + \cdots + a_n v_n = 0

implies a1=a2==an=0a_1 = a_2 = \cdots = a_n = 0.

A set that is not linearly independent is called linearly dependent.

RemarkEquivalent formulation

{v1,,vn}\{v_1, \ldots, v_n\} is linearly dependent if and only if some viv_i can be written as a linear combination of the other vectors. That is, there exists an index ii such that

vi=jicjvjv_i = \sum_{j \neq i} c_j v_j

for some scalars cjFc_j \in F.

RemarkThe empty set and singletons
  • The empty set is linearly independent (vacuously).
  • {v}\{v\} is linearly independent if and only if v0v \neq 0. (If av=0av = 0 and v0v \neq 0, then a=0a = 0.)

Examples

ExampleStandard basis vectors

In Rn\mathbb{R}^n, the standard basis vectors e1=(1,0,,0)e_1 = (1, 0, \ldots, 0), e2=(0,1,,0)e_2 = (0, 1, \ldots, 0), \ldots, en=(0,0,,1)e_n = (0, 0, \ldots, 1) are linearly independent.

If a1e1++anen=0a_1 e_1 + \cdots + a_n e_n = 0, then (a1,a2,,an)=(0,0,,0)(a_1, a_2, \ldots, a_n) = (0, 0, \ldots, 0).

ExampleTwo vectors in R^2

{(1,2),(3,4)}\{(1, 2), (3, 4)\} is linearly independent. Suppose a(1,2)+b(3,4)=(0,0)a(1, 2) + b(3, 4) = (0, 0):

a+3b=0,2a+4b=0.a + 3b = 0, \quad 2a + 4b = 0.

From the first equation, a=3ba = -3b. Substituting: 6b+4b=2b=0-6b + 4b = -2b = 0, so b=0b = 0 and a=0a = 0.

ExampleA linearly dependent set

{(1,2,3),(4,5,6),(5,7,9)}\{(1, 2, 3), (4, 5, 6), (5, 7, 9)\} is linearly dependent because

(1,2,3)+(4,5,6)=(5,7,9).(1, 2, 3) + (4, 5, 6) = (5, 7, 9).

Equivalently, 1(1,2,3)+1(4,5,6)+(1)(5,7,9)=(0,0,0)1 \cdot (1,2,3) + 1 \cdot (4,5,6) + (-1) \cdot (5,7,9) = (0,0,0), a nontrivial relation.

ExampleIndependent polynomials

{1,x,x2,,xn}\{1, x, x^2, \ldots, x^n\} is linearly independent in P(F)P(F). If a0+a1x++anxn=0a_0 + a_1 x + \cdots + a_n x^n = 0 as a polynomial function, then all coefficients must be zero (when FF is infinite, or by comparing coefficients formally).

Examplesin and cos are independent

{sinx,cosx}\{\sin x, \cos x\} is linearly independent in C(R)C(\mathbb{R}). If asinx+bcosx=0a\sin x + b\cos x = 0 for all xx:

  • Setting x=0x = 0: b=0b = 0.
  • Setting x=π/2x = \pi/2: a=0a = 0.
ExampleDistinct exponentials are independent

{ex,e2x}\{e^x, e^{2x}\} is linearly independent in C(R)C^\infty(\mathbb{R}). If aex+be2x=0ae^x + be^{2x} = 0 for all xx, then dividing by exe^x gives a+bex=0a + be^x = 0 for all xx, which forces b=0b = 0 (let xx \to \infty) and then a=0a = 0.

More generally, {eλ1x,eλ2x,,eλnx}\{e^{\lambda_1 x}, e^{\lambda_2 x}, \ldots, e^{\lambda_n x}\} is linearly independent whenever λ1,,λn\lambda_1, \ldots, \lambda_n are distinct.

ExampleIndependent matrices

In M2×2(R)M_{2 \times 2}(\mathbb{R}), the matrices

A1=(1000),A2=(0110),A3=(0001)A_1 = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \quad A_2 = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, \quad A_3 = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}

are linearly independent: aA1+bA2+cA3=(abbc)=0aA_1 + bA_2 + cA_3 = \begin{pmatrix} a & b \\ b & c \end{pmatrix} = 0 implies a=b=c=0a = b = c = 0.

ExampleFunctional independence via evaluation

{1,x}\{1, |x|\} is linearly independent in F(R,R)F(\mathbb{R}, \mathbb{R}). If a1+bx=0a \cdot 1 + b|x| = 0 for all xx:

  • x=0x = 0: a=0a = 0.
  • x=1x = 1: b=0b = 0.
ExampleAny set containing 0 is dependent

If 0{v1,,vn}0 \in \{v_1, \ldots, v_n\}, the set is linearly dependent. Say v1=0v_1 = 0. Then 1v1+0v2++0vn=01 \cdot v_1 + 0 \cdot v_2 + \cdots + 0 \cdot v_n = 0 is a nontrivial relation.

ExampleSupersets of dependent sets are dependent

If {v1,,vn}\{v_1, \ldots, v_n\} is linearly dependent and wVw \in V, then {v1,,vn,w}\{v_1, \ldots, v_n, w\} is also linearly dependent: the existing nontrivial relation extends with coefficient 00 for ww.

ExampleSubsets of independent sets are independent

If {v1,,vn}\{v_1, \ldots, v_n\} is linearly independent, then every nonempty subset is linearly independent. (Any nontrivial relation in the subset would extend to a nontrivial relation in the full set.)

ExampleIndependence depends on the field

{1,i}\{1, i\} is linearly independent in C\mathbb{C} viewed as a vector space over R\mathbb{R}, but linearly dependent in C\mathbb{C} over C\mathbb{C} (since i=i1i = i \cdot 1). The field matters.

ExampleWronskian test for functions

For nn times differentiable functions f1,,fnf_1, \ldots, f_n, the Wronskian is

W(f1,,fn)(x)=det(f1(x)fn(x)f1(x)fn(x)f1(n1)(x)fn(n1)(x)).W(f_1, \ldots, f_n)(x) = \det \begin{pmatrix} f_1(x) & \cdots & f_n(x) \\ f_1'(x) & \cdots & f_n'(x) \\ \vdots & \ddots & \vdots \\ f_1^{(n-1)}(x) & \cdots & f_n^{(n-1)}(x) \end{pmatrix}.

If W0W \neq 0 at some point, then f1,,fnf_1, \ldots, f_n are linearly independent.

For example, W(sinx,cosx)=sinx(sinx)cosxcosx=10W(\sin x, \cos x) = \sin x \cdot (-\sin x) - \cos x \cdot \cos x = -1 \neq 0.


Key properties

Theorem1.5Dependence and span

Let S={v1,,vn}S = \{v_1, \ldots, v_n\} be linearly dependent and vn0v_n \neq 0. If vkv_k is a vector that can be expressed as a linear combination of the preceding vectors v1,,vk1v_1, \ldots, v_{k-1}, then

span(S)=span(S{vk}).\text{span}(S) = \text{span}(S \setminus \{v_k\}).

In other words, removing a "redundant" vector does not change the span.

RemarkLooking ahead

Linear independence and spanning are dual requirements. A set that is both linearly independent and spanning is a basis, the central concept that connects the algebraic structure of a vector space to its "size" (dimension).