ConceptComplete

Inner Product

An inner product endows a vector space with geometric structure: lengths, angles, and distances. While a bare vector space only has addition and scalar multiplication, an inner product space can measure the "size" of vectors and the "angle" between them, connecting algebra to geometry.


Definition

Definition6.1Inner product (real case)

Let VV be a real vector space. An inner product on VV is a function ,:V×VR\langle \cdot, \cdot \rangle : V \times V \to \mathbb{R} satisfying:

  1. Linearity in the first argument: αu+βv,w=αu,w+βv,w\langle \alpha u + \beta v, w \rangle = \alpha \langle u, w \rangle + \beta \langle v, w \rangle.
  2. Symmetry: u,v=v,u\langle u, v \rangle = \langle v, u \rangle.
  3. Positive definiteness: v,v0\langle v, v \rangle \geq 0, with equality if and only if v=0v = 0.

A real vector space equipped with an inner product is called a real inner product space (or Euclidean space when finite-dimensional).

Definition6.2Inner product (complex case)

Let VV be a complex vector space. An inner product on VV is a function ,:V×VC\langle \cdot, \cdot \rangle : V \times V \to \mathbb{C} satisfying:

  1. Linearity in the first argument: αu+βv,w=αu,w+βv,w\langle \alpha u + \beta v, w \rangle = \alpha \langle u, w \rangle + \beta \langle v, w \rangle.
  2. Conjugate symmetry: u,v=v,u\langle u, v \rangle = \overline{\langle v, u \rangle}.
  3. Positive definiteness: v,v0\langle v, v \rangle \geq 0 (it is real by conjugate symmetry), with equality iff v=0v = 0.

A complex vector space with an inner product is called a unitary space (or pre-Hilbert space).

RemarkConjugate-linearity in the second argument

In the complex case, conjugate symmetry together with linearity in the first argument implies conjugate-linearity in the second:

u,αv+βw=αˉu,v+βˉu,w.\langle u, \alpha v + \beta w \rangle = \bar{\alpha} \langle u, v \rangle + \bar{\beta} \langle u, w \rangle.

Some references (especially in physics) use the opposite convention: linear in the second argument, conjugate-linear in the first.


Standard examples

ExampleThe dot product on R^n

The standard inner product on Rn\mathbb{R}^n is the dot product:

x,y=xy=i=1nxiyi=xTy.\langle x, y \rangle = x \cdot y = \sum_{i=1}^n x_i y_i = x^T y.

For x=(1,2,3)x = (1, 2, 3) and y=(4,1,2)y = (4, -1, 2): x,y=42+6=8\langle x, y \rangle = 4 - 2 + 6 = 8.

This satisfies all three axioms: linearity is clear, symmetry gives xy=yxx \cdot y = y \cdot x, and x,x=x12++xn20\langle x, x \rangle = x_1^2 + \cdots + x_n^2 \geq 0 with equality iff x=0x = 0.

ExampleThe standard inner product on C^n

On Cn\mathbb{C}^n:

x,y=i=1nxiyi=xy,\langle x, y \rangle = \sum_{i=1}^n x_i \overline{y_i} = x^* y,

where x=xˉTx^* = \bar{x}^T is the conjugate transpose.

For x=(1,i)x = (1, i) and y=(2,3i)y = (2, 3i): x,y=12ˉ+i3i=2+i(3i)=2+3=5\langle x, y \rangle = 1 \cdot \bar{2} + i \cdot \overline{3i} = 2 + i(-3i) = 2 + 3 = 5.

Note: x,x=x12++xn20\langle x, x \rangle = |x_1|^2 + \cdots + |x_n|^2 \geq 0, so positive definiteness holds.

ExampleWeighted inner product

On Rn\mathbb{R}^n with positive weights w1,,wn>0w_1, \ldots, w_n > 0:

x,yw=i=1nwixiyi.\langle x, y \rangle_w = \sum_{i=1}^n w_i x_i y_i.

For w=(2,3)w = (2, 3) on R2\mathbb{R}^2: (1,1),(1,1)w=211+31(1)=1\langle (1, 1), (1, -1) \rangle_w = 2 \cdot 1 \cdot 1 + 3 \cdot 1 \cdot (-1) = -1.

The vectors (1,1)(1, 1) and (1,1)(1, -1) are orthogonal in the standard inner product (11=01 - 1 = 0) but not in this weighted one.

ExampleInner product on C[a,b]

On the space of continuous functions C[a,b]C[a, b]:

f,g=abf(x)g(x)dx.\langle f, g \rangle = \int_a^b f(x) g(x) \, dx.

For f(x)=xf(x) = x and g(x)=x2g(x) = x^2 on [0,1][0, 1]: f,g=01x3dx=14\langle f, g \rangle = \int_0^1 x^3 \, dx = \frac{1}{4}.

For f(x)=sin(πx)f(x) = \sin(\pi x) and g(x)=sin(2πx)g(x) = \sin(2\pi x) on [0,1][0, 1]: f,g=01sin(πx)sin(2πx)dx=0\langle f, g \rangle = \int_0^1 \sin(\pi x) \sin(2\pi x) \, dx = 0 (orthogonality of Fourier modes).


Norm and distance

Definition6.3Norm induced by an inner product

The norm (or length) of a vector vv in an inner product space is:

v=v,v.\|v\| = \sqrt{\langle v, v \rangle}.

The distance between uu and vv is d(u,v)=uvd(u, v) = \|u - v\|.

ExampleNorm in R^3

For v=(3,4,0)R3v = (3, 4, 0) \in \mathbb{R}^3 with the standard inner product:

v=9+16+0=5\|v\| = \sqrt{9 + 16 + 0} = 5.

(1,1,1)=3\|(1, 1, 1)\| = \sqrt{3}, (1,0,0)=1\|(1, 0, 0)\| = 1.

ExampleNorm of functions

With f,g=01f(x)g(x)dx\langle f, g \rangle = \int_0^1 f(x) g(x) \, dx:

x=01x2dx=1/3=13\|x\| = \sqrt{\int_0^1 x^2 \, dx} = \sqrt{1/3} = \frac{1}{\sqrt{3}}.

1=011dx=1\|1\| = \sqrt{\int_0^1 1 \, dx} = 1.

sin(πx)=01sin2(πx)dx=1/2=12\|\sin(\pi x)\| = \sqrt{\int_0^1 \sin^2(\pi x) \, dx} = \sqrt{1/2} = \frac{1}{\sqrt{2}}.


Properties of the norm

TheoremProperties of the induced norm

For all u,vVu, v \in V and αF\alpha \in F:

  1. v0\|v\| \geq 0, with v=0\|v\| = 0 iff v=0v = 0.
  2. αv=αv\|\alpha v\| = |\alpha| \cdot \|v\| (homogeneity).
  3. u+vu+v\|u + v\| \leq \|u\| + \|v\| (triangle inequality).
  4. u,vuv|\langle u, v \rangle| \leq \|u\| \cdot \|v\| (Cauchy--Schwarz inequality).
ExampleTriangle inequality in R^2

u=(3,0)u = (3, 0), v=(0,4)v = (0, 4): u+v=(3,4)=53+4=u+v=7\|u + v\| = \|(3, 4)\| = 5 \leq 3 + 4 = \|u\| + \|v\| = 7. The inequality is strict because uu and vv are not parallel.

u=(1,2)u = (1, 2), v=(2,4)v = (2, 4): u+v=(3,6)=35=35\|u + v\| = \|(3, 6)\| = 3\sqrt{5} = 3\sqrt{5} and u+v=5+25=35\|u\| + \|v\| = \sqrt{5} + 2\sqrt{5} = 3\sqrt{5}. Equality holds because v=2uv = 2u (they are parallel).


Angles

Definition6.4Angle between vectors

In a real inner product space, the angle θ\theta between nonzero vectors u,vu, v is defined by:

cosθ=u,vuv,0θπ.\cos\theta = \frac{\langle u, v \rangle}{\|u\| \cdot \|v\|}, \quad 0 \leq \theta \leq \pi.

This is well-defined by the Cauchy--Schwarz inequality, which guarantees cosθ1|\cos\theta| \leq 1.

ExampleAngle between vectors in R^3

u=(1,0,0)u = (1, 0, 0), v=(1,1,0)v = (1, 1, 0): cosθ=12\cos\theta = \frac{1}{\sqrt{2}}, so θ=π/4=45°\theta = \pi/4 = 45°.

u=(1,0,0)u = (1, 0, 0), v=(0,1,0)v = (0, 1, 0): cosθ=0\cos\theta = 0, so θ=π/2=90°\theta = \pi/2 = 90° (perpendicular).

u=(1,1,1)u = (1, 1, 1), v=(1,1,1)v = (-1, -1, -1): cosθ=33=1\cos\theta = \frac{-3}{3} = -1, so θ=π=180°\theta = \pi = 180° (anti-parallel).

ExampleAngle between functions

With f,g=01f(x)g(x)dx\langle f, g \rangle = \int_0^1 f(x)g(x)\,dx, the angle between f(x)=1f(x) = 1 and g(x)=xg(x) = x:

cosθ=1,x1x=1/211/3=32\cos\theta = \frac{\langle 1, x \rangle}{\|1\| \cdot \|x\|} = \frac{1/2}{1 \cdot 1/\sqrt{3}} = \frac{\sqrt{3}}{2}, so θ=π/6=30°\theta = \pi/6 = 30°.


Inner products via matrices

TheoremInner products on R^n via positive definite matrices

Every inner product on Rn\mathbb{R}^n has the form x,y=xTMy\langle x, y \rangle = x^T M y for a unique symmetric positive definite matrix MM. Conversely, every symmetric positive definite MM defines an inner product this way.

ExampleInner product from a positive definite matrix

M=(2113)M = \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix} is positive definite (eigenvalues 5±52>0\frac{5 \pm \sqrt{5}}{2} > 0).

x,y=xTMy\langle x, y \rangle = x^T M y. For x=(1,0)x = (1, 0), y=(0,1)y = (0, 1): x,y=(1,0)(2113)(01)=1\langle x, y \rangle = (1, 0) \begin{pmatrix} 2 & 1 \\ 1 & 3 \end{pmatrix} \begin{pmatrix} 0 \\ 1 \end{pmatrix} = 1.

e12=e1,e1=2\|e_1\|^2 = \langle e_1, e_1 \rangle = 2, e22=3\|e_2\|^2 = 3. The standard basis vectors are not unit vectors in this inner product.

ExampleWhen a bilinear form is NOT an inner product

M=(1221)M = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} has eigenvalues 33 and 1-1. Since 1<0-1 < 0, MM is not positive definite.

(1,1),(1,1)=(1,1)(1221)(11)=(1,1)(11)=2<0\langle (1, -1), (1, -1) \rangle = (1, -1) \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} 1 \\ -1 \end{pmatrix} = (1, -1)\begin{pmatrix} -1 \\ 1 \end{pmatrix} = -2 < 0.

This fails positive definiteness, so it is not an inner product.


Parallelogram law and polarization

TheoremParallelogram law

In any inner product space: u+v2+uv2=2u2+2v2\|u + v\|^2 + \|u - v\|^2 = 2\|u\|^2 + 2\|v\|^2.

This characterizes inner product spaces among normed spaces: a norm satisfies the parallelogram law if and only if it comes from an inner product.

ExampleParallelogram law verification

u=(1,0)u = (1, 0), v=(0,1)v = (0, 1) in R2\mathbb{R}^2:

u+v2+uv2=(1,1)2+(1,1)2=2+2=4\|u + v\|^2 + \|u - v\|^2 = \|(1,1)\|^2 + \|(1,-1)\|^2 = 2 + 2 = 4. 2u2+2v2=2+2=42\|u\|^2 + 2\|v\|^2 = 2 + 2 = 4 ✓.

TheoremPolarization identity (real case)

The inner product can be recovered from the norm:

u,v=14(u+v2uv2).\langle u, v \rangle = \frac{1}{4}(\|u + v\|^2 - \|u - v\|^2).


Summary

RemarkThe inner product as geometric structure

An inner product transforms a bare vector space into a geometric space:

  • Norm: measures length, v=v,v\|v\| = \sqrt{\langle v, v \rangle}.
  • Distance: d(u,v)=uvd(u,v) = \|u - v\| gives metric space structure.
  • Angle: cosθ=u,v/(uv)\cos\theta = \langle u, v \rangle / (\|u\| \|v\|).
  • Orthogonality: uvu \perp v iff u,v=0\langle u, v \rangle = 0.
  • Every inner product on Rn\mathbb{R}^n arises from a symmetric positive definite matrix.
  • The Cauchy--Schwarz inequality is the fundamental inequality governing the interaction between the inner product and the norm.