Extrema of Multivariable Functions

Finding extrema (maxima and minima) of multivariable functions is central to optimization, economics, and physics. Critical points occur where the gradient vanishes, and the second derivative test (using the Hessian matrix) classifies them as local maxima, minima, or saddle points. Constrained optimization uses Lagrange multipliers.

Critical points

Definition9.1Critical point

A point $\mathbf{a}$ is a critical point of $f : \mathbb{R}^n \to \mathbb{R}$ if $\nabla f(\mathbf{a}) = \mathbf{0}$ or $\nabla f(\mathbf{a})$ does not exist.

Theorem9.1First derivative test

If $f$ has a local extremum (maximum or minimum) at an interior point $\mathbf{a}$ and $f$ is differentiable at $\mathbf{a}$ , then $\nabla f(\mathbf{a}) = \mathbf{0}$ .

ExampleCritical points of f(x, y) = x² - y²

$\nabla f = (2x, -2y) = \mathbf{0}$ gives $(x, y) = (0, 0)$ as the only critical point. However, $(0, 0)$ is a saddle point, not an extremum.

Second derivative test

Definition9.2Hessian matrix

The Hessian matrix of $f$ at $\mathbf{a}$ is

$Hf(\mathbf{a}) = \begin{pmatrix} \frac{\partial^2 f}{\partial x_1^2} & \cdots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\ \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n \partial x_1} & \cdots & \frac{\partial^2 f}{\partial x_n^2} \end{pmatrix}_{\mathbf{a}}.$

Theorem9.2Second derivative test

Let $\mathbf{a}$ be a critical point of $f$ with $\nabla f(\mathbf{a}) = \mathbf{0}$ . Let $H = Hf(\mathbf{a})$ .

If $H$ is positive definite, then $\mathbf{a}$ is a local minimum.
If $H$ is negative definite, then $\mathbf{a}$ is a local maximum.
If $H$ has both positive and negative eigenvalues, then $\mathbf{a}$ is a saddle point.
If $H$ is indefinite (test inconclusive), higher-order tests are needed.

Examplef(x, y) = x² + y²

$\nabla f = (2x, 2y) = \mathbf{0}$ at $(0, 0)$ . The Hessian is

$H = \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix},$

which is positive definite (eigenvalues $2, 2 > 0$ ). Thus $(0, 0)$ is a local minimum.

Lagrange multipliers

Theorem9.3Lagrange multipliers

To optimize $f(\mathbf{x})$ subject to the constraint $g(\mathbf{x}) = 0$ , solve

$\nabla f = \lambda \nabla g$

for $\mathbf{x}$ and $\lambda$ (the Lagrange multiplier). The solutions are candidates for constrained extrema.

ExampleMaximize xy subject to x² + y² = 1

Lagrange condition: $(y, x) = \lambda (2x, 2y)$ . This gives $y = 2\lambda x$ and $x = 2\lambda y$ , so $x = 4\lambda^2 x$ . If $x \neq 0$ , then $\lambda^2 = 1/4$ , so $\lambda = \pm 1/2$ .

For $\lambda = 1/2$ : $y = x$ , so $2x^2 = 1$ , giving $(x, y) = (\pm 1/\sqrt{2}, \pm 1/\sqrt{2})$ with $f = 1/2$ (maximum).

For $\lambda = -1/2$ : $y = -x$ , giving $f = -1/2$ (minimum).

Summary

Extrema of multivariable functions:

Critical points: $\nabla f = \mathbf{0}$ .
Second derivative test: Hessian determines local behavior.
Lagrange multipliers for constrained optimization.

See Implicit Function Theorem for related results.