Tensor Product Definition

The tensor product constructs a new vector space from two given spaces, capturing all possible bilinear relationships. It provides the universal framework for multilinear algebra.

DefinitionTensor Product

Given vector spaces $V$ and $W$ over field $\mathbb{F}$ , the tensor product $V \otimes W$ is a vector space together with a bilinear map $\otimes: V \times W \to V \otimes W$ satisfying the universal property:

For any vector space $U$ and bilinear map $\phi: V \times W \to U$ , there exists a unique linear map $\tilde{\phi}: V \otimes W \to U$ making the diagram commute: $\phi(\mathbf{v}, \mathbf{w}) = \tilde{\phi}(\mathbf{v} \otimes \mathbf{w})$

Elements $\mathbf{v} \otimes \mathbf{w}$ are called simple tensors or pure tensors. General elements of $V \otimes W$ are finite sums $\sum_i \mathbf{v}_i \otimes \mathbf{w}_i$ .

DefinitionTensor Product Relations

Simple tensors satisfy bilinearity relations:

$(\mathbf{v}_1 + \mathbf{v}_2) \otimes \mathbf{w} = \mathbf{v}_1 \otimes \mathbf{w} + \mathbf{v}_2 \otimes \mathbf{w}$
$\mathbf{v} \otimes (\mathbf{w}_1 + \mathbf{w}_2) = \mathbf{v} \otimes \mathbf{w}_1 + \mathbf{v} \otimes \mathbf{w}_2$
$(c\mathbf{v}) \otimes \mathbf{w} = c(\mathbf{v} \otimes \mathbf{w}) = \mathbf{v} \otimes (c\mathbf{w})$

These force $\mathbf{0} \otimes \mathbf{w} = \mathbf{0}$ and $\mathbf{v} \otimes \mathbf{0} = \mathbf{0}$ .

ExampleTensor Product of $\mathbb{R}^2$ with Itself

$\mathbb{R}^2 \otimes \mathbb{R}^2$ has dimension $2 \times 2 = 4$ . With standard bases $\{\mathbf{e}_1, \mathbf{e}_2\}$ , a basis is: $\{\mathbf{e}_1 \otimes \mathbf{e}_1, \mathbf{e}_1 \otimes \mathbf{e}_2, \mathbf{e}_2 \otimes \mathbf{e}_1, \mathbf{e}_2 \otimes \mathbf{e}_2\}$

General element: $\sum_{i,j} c_{ij}(\mathbf{e}_i \otimes \mathbf{e}_j)$ corresponds to a $2 \times 2$ matrix $[c_{ij}]$ .

TheoremDimension of Tensor Product

If $\dim(V) = m$ and $\dim(W) = n$ , then: $\dim(V \otimes W) = mn$

If $\{\mathbf{v}_1, \ldots, \mathbf{v}_m\}$ and $\{\mathbf{w}_1, \ldots, \mathbf{w}_n\}$ are bases, then $\{\mathbf{v}_i \otimes \mathbf{w}_j : 1 \leq i \leq m, 1 \leq j \leq n\}$ is a basis for $V \otimes W$ .

Remark

The tensor product is subtle: not every element is a simple tensor. For instance, $\mathbf{e}_1 \otimes \mathbf{e}_2 + \mathbf{e}_2 \otimes \mathbf{e}_1$ cannot be written as $\mathbf{v} \otimes \mathbf{w}$ . The tensor product naturally models multilinear phenomena: in physics, stress tensors combine force and area; in machine learning, outer products build weight matrices. The universal property makes tensor products the "correct" construction for multilinearity.