14.8 - Lagrange Multipliers

Consider the problem of maximizing or minimizing some function $f(x,y)$, subject to some constraint $g(x,y)=k$

Graph with Matplotlib

Python

import numpy as np import matplotlib.pyplot as plt from mpl_toolkits.mplot3d import Axes3D def f(x, y): return x**2 + y**2 # Set up grid for plotting x = np.linspace(-1.5, 1.5, 400) y = np.linspace(-1.5, 1.5, 400) X, Y = np.meshgrid(x, y) Z = f(X, Y) fig = plt.figure(figsize=(6, 4)) ax = fig.add_subplot(111, projection='3d') ax.plot_surface(X, Y, Z, rstride=5, cstride=5, alpha=0.3, cmap='coolwarm') theta = np.linspace(0, 2 * np.pi, 100) r = 1 # Radius of the constraint (constant for regular curve) x_constraint = r * np.cos(theta) y_constraint = r * np.sin(theta) z_constraint = np.zeros_like(x_constraint) ax.plot(x_constraint, y_constraint, z_constraint, color='red', lw=2, label='Constraint curve') def squiggled_constraint(theta, base_r=1, amplitude=0.1, frequency=5): # Adjust radius based on sinusoidal squiggle r_squiggled = base_r + amplitude * np.sin(frequency * theta) x_squiggled = r_squiggled * np.cos(theta) y_squiggled = r_squiggled * np.sin(theta) return x_squiggled, y_squiggled x_squiggled, y_squiggled = squiggled_constraint(theta) z_squiggled = f(x_squiggled, y_squiggled) ax.plot(x_squiggled, y_squiggled, z_squiggled, color='orange', lw=3, label='Squiggled lifted constraint curve') max_idx = np.argmax(z_squiggled) # Index of the maximum z value min_idx = np.argmin(z_squiggled) # Index of the minimum z value ax.scatter(x_squiggled[max_idx], y_squiggled[max_idx], z_squiggled[max_idx], color='green', s=100) ax.text(x_squiggled[max_idx], y_squiggled[max_idx], z_squiggled[max_idx] + 0.2, 'Maximum', color='green', fontsize=12) ax.scatter(x_squiggled[min_idx], y_squiggled[min_idx], z_squiggled[min_idx], color='blue', s=100) ax.text(x_squiggled[min_idx], y_squiggled[min_idx], z_squiggled[min_idx] + 0.2, 'Minimum', color='blue', fontsize=12) ax.set_xlabel('X') ax.set_ylabel('Y') ax.set_zlabel('Z') ax.set_title('3D Surface with Squiggled Constraint Curve and Max/Min Points') ax.view_init(elev=30, azim=120) plt.legend() plt.show()

Output

Important

$g(x,y)=k$ is a contour/level curve of some unknown constraint surface $z=f(x,y)$.

---

Solving This Problem

1. We parameterize the constraint curve $g(x,y)=k$ by

$\vec{r}(t)=⟨x(t),y(t)⟩$

By the Key Formula from Sec. 14.6,

$\nabla g\cdot {\vec{r}}^{\prime }(t)=0$

for all t in the parameter domain.

2. The curve C whose optimal value we seek is obtained by lifting $\vec{r}(t)$ to the surface $z=f(x,y)$.

This means we can parameterize C by

$\vec{c}(t)=⟨x(t),y(t),f(x(t),y(t))⟩$

Observe

The max/min values along $\vec{c}$ may occur when the z-component of ${\vec{c}}^{\prime }s$ tangent vector equals 0.

Why?

In complete analogy with Calc 1, we want to look where $\vec{c}$ has a horizontal tangent line. In ${\mathbb{R}}^3$

, this means we are looking for points where the tangent vector $c^{\prime }(t)$ is horizontal.

A tangent vector is horizontal if its z-component is zero. Let’s compute the tangent vector $c^{\prime }(t)$:

$$c^{\prime }(t)=⟨x^{\prime }(t),y^{\prime }(t),\frac{d}{dt}f(x(t),y(t))⟩$$

For this vector to be horizontal, we must have:

$$\frac{d}{dt}f(x(t),y(t))=0$$

By the Chain Rule for paths, we know:

$$\frac{d}{dt}f(x(t),y(t))=\frac{\partial f}{\partial x}\frac{\partial x}{\partial t}+\frac{\partial f}{\partial y}\frac{\partial y}{\partial t}$$

This can be written compactly as a dot product:

$$\nabla f(x(t),y(t))\cdot r^{\prime }(t)=0$$

---

The Geometric Conclusion

Let’s summarize what we know about an optimal point $(x_0,y_0)$ on the constraint curve. Let this point correspond to $t=t_0$​ in our parameterization.

1. Because the point is on a level curve of g, the gradient of g is orthogonal to the direction of the curve:

   $$\nabla g(x_0,y_0)\cdot {\vec{r}}^{\prime }(t_0)=0$$

2. Because the point is a local max or min of f along the curve, the gradient of $f$ is also orthogonal to the direction of the curve:

$$\nabla f(x_0,y_0)\cdot {\vec{r}}^{\prime }(t_0)=0$$

Both gradients, $\nabla f$ and $\nabla g$, are orthogonal to the very same tangent vector $r^{\prime }(t_0)$ at that point. In ${\mathbb{R}}^2$, this means that the two gradient vectors must be parallel.

The Lagrange Multiplier

If two vectors are parallel, then one must be a scalar multiple of the other. We call this scalar multiplier λ (lambda), the Lagrange Multiplier.

$$\nabla f(x_0,y_0)=\lambda \nabla g(x_0,y_0)$$

This single vector equation gives us the theoretical foundation for solving constrained optimization problems.

---

The Method of Lagrange Multipliers

To find the maximum and minimum values of $f(x,y)$ subject to the constraint $g(x,y)=k$:

1. Find all values of $x$, $y$, and $\lambda$ such that

$$\nabla f(x,y)=\lambda \nabla g(x,y)$$

and

$$g(x,y)=k$$

2. The first equation, $\nabla f=\lambda \nabla g$, provides a system of two component equations:
   • $f_x(x,y)=\lambda g_x(x,y)$
   • $f_y(x,y)=\lambda g_y(x,y)$
3. Evaluate f at all points $(x,y)$ you obtain from step 1. The largest of these values is the maximum value, and the smallest is the minimum value.

Summary: The System to Solve

In total, you are solving a system of three equations with three unknowns $(x,y,\lambda )$:

1. $f_x=\lambda g_x$
2. $f_y=\lambda g_y$
3. $g(x,y)=k$