Taylor's Theorem (Multivariable)

Taylor's theorem extends to multivariable functions, approximating $f(\mathbf{x})$ near $\mathbf{a}$ using derivatives. The first-order approximation uses the gradient, and the second-order uses the Hessian. This is essential for optimization (Newton's method), error analysis, and differential geometry.

Statement

Theorem9.1Taylor's Theorem (Second Order)

Let $f : \mathbb{R}^n \to \mathbb{R}$ be twice continuously differentiable. Then for $\mathbf{x}$ near $\mathbf{a}$ ,

$f(\mathbf{x}) = f(\mathbf{a}) + \nabla f(\mathbf{a}) \cdot (\mathbf{x} - \mathbf{a}) + \frac{1}{2}(\mathbf{x} - \mathbf{a})^T H f(\mathbf{a}) (\mathbf{x} - \mathbf{a}) + o(\|\mathbf{x} - \mathbf{a}\|^2),$

where $Hf(\mathbf{a})$ is the Hessian matrix.

Applications

ExampleNewton's method for optimization

To minimize $f(\mathbf{x})$ , Newton's method iterates

$\mathbf{x}_{n+1} = \mathbf{x}_n - [Hf(\mathbf{x}_n)]^{-1} \nabla f(\mathbf{x}_n).$

This uses the second-order Taylor approximation to find the minimum of the quadratic approximation.

Summary

Multivariable Taylor's theorem:

First-order: $f(\mathbf{x}) \approx f(\mathbf{a}) + \nabla f \cdot (\mathbf{x} - \mathbf{a})$ .
Second-order: adds Hessian term for curvature.
Applications: optimization (Newton's method), error analysis.

See Extrema for optimization.