Method of Lagrange Multipliers

The method of Lagrange multipliers provides an elegant and systematic approach to constrained optimization, reducing the problem of extremizing a function subject to constraints to a system of algebraic equations.

The Theorem

Theorem8.7Lagrange Multipliers

Let $f, g_1, \ldots, g_k : \mathbb{R}^n \to \mathbb{R}$ be $C^1$ functions with $k < n$ . Suppose $\mathbf{a}$ is a local extremum of $f$ subject to the constraints $g_1(\mathbf{x}) = \cdots = g_k(\mathbf{x}) = 0$ , and that the gradients $\nabla g_1(\mathbf{a}), \ldots, \nabla g_k(\mathbf{a})$ are linearly independent. Then there exist scalars $\lambda_1, \ldots, \lambda_k$ (called Lagrange multipliers) such that $\nabla f(\mathbf{a}) = \lambda_1 \nabla g_1(\mathbf{a}) + \cdots + \lambda_k \nabla g_k(\mathbf{a})$

The geometric interpretation is clear: at a constrained extremum, the gradient of $f$ is a linear combination of the constraint gradients. Equivalently, $\nabla f$ is perpendicular to the constraint surface at $\mathbf{a}$ , meaning there is no direction along the constraint surface in which $f$ can increase.

Applications

ExampleOptimization on a sphere

Maximize $f(x, y, z) = x + 2y + 3z$ subject to $g(x, y, z) = x^2 + y^2 + z^2 - 1 = 0$ .

The Lagrange condition $\nabla f = \lambda \nabla g$ gives: $1 = 2\lambda x, \quad 2 = 2\lambda y, \quad 3 = 2\lambda z$ So $x = \frac{1}{2\lambda}$ , $y = \frac{1}{\lambda}$ , $z = \frac{3}{2\lambda}$ . Substituting into the constraint: $\frac{1}{4\lambda^2} + \frac{1}{\lambda^2} + \frac{9}{4\lambda^2} = 1 \implies \frac{14}{4\lambda^2} = 1 \implies \lambda = \pm\frac{\sqrt{14}}{2}$ Maximum value: $f = \frac{1}{\sqrt{14}}(1 + 2 + \frac{9}{1}) = \sqrt{14}$ at $\left(\frac{1}{\sqrt{14}}, \frac{2}{\sqrt{14}}, \frac{3}{\sqrt{14}}\right)$ .

ExampleInequality constraint: AM-GM

To prove the AM-GM inequality, minimize $f(x_1, \ldots, x_n) = \frac{x_1 + \cdots + x_n}{n}$ subject to $g = x_1 x_2 \cdots x_n - c^n = 0$ for $x_i > 0$ . The Lagrange condition gives $\frac{1}{n} = \lambda \prod_{j \neq i} x_j$ for each $i$ , implying all $x_i$ are equal: $x_1 = \cdots = x_n = c$ . This establishes $\frac{x_1 + \cdots + x_n}{n} \geq (x_1 \cdots x_n)^{1/n}$ .

RemarkThe multiplier has economic meaning

The Lagrange multiplier $\lambda$ has the interpretation $\lambda = \frac{df^*}{dc}$ where $f^*$ is the optimal value and $c$ is the constraint level. In economics, if $f$ represents utility and $g$ a budget constraint, then $\lambda$ measures the marginal utility of income (the "shadow price" of relaxing the constraint by one unit).