Raising the bar

Lagrange method: necessary condition

Consider the problem:

(1) maximize the objective function $latex f(x,y)$ subject to the equality constraint $latex g(x,y)=0.$

Lagrange's idea: add a new, artificial variable $latex \lambda$ and consider a new function of three variables $latex L(x,y,\lambda )=f(x,y)+\lambda g(x,y).$ The solution of the constrained problem (1) should be equivalent to the solution of the unconstrained problem

(2) maximize $latex L(x,y,\lambda )$.

$latex L(x,y,\lambda )$ is called a Lagrangian. Recall the implicit function existence condition

(3) $latex \frac{\partial g}{\partial y}\neq 0.$

Under this condition we can employ a useful trick: when the implicit function exists, we can differentiate the restriction $latex g(x,y(x))=0$ to obtain

(4) $latex \frac{\partial g}{\partial x}+\frac{\partial g}{\partial y}y^\prime(x)=0.$

Simple way to solve (1)

Assuming (3), we can find $latex y=y(x)$ from the restriction and plug $latex y(x)$ into the objective function to obtain a function of one variable

$latex \phi (x)=f(x,y(x)).$

It's enough to find the extremes of this function (we don't need to use the constraint). At an extremum we necessarily have the first order condition

(5) $latex \frac{d\phi }{dx}=\frac{\partial f}{\partial x}+\frac{\partial f}{\partial y}y^\prime(x)=0.$

Lagrange went one step further

(4)+(5) is a linear system of equations. To make this clear, let us introduce a matrix and a vector

$latex A=\left(\begin{array}{cc}
\frac{\partial g}{\partial x} & \frac{\partial g}{\partial y}\\
\frac{\partial f}{\partial x} & \frac{\partial f}{\partial y}
\end{array}\right) ,~\ Y=\left(\begin{array}{c}1\\
y^\prime(x)\end{array}\right).$

Then (4)+(5) becomes $latex AY=0.$ This is a homogeneous system (the right side is zero) and it has a nonzero solution $latex Y$ (at least its first component is not zero). The matrix theory tells us that this is possible only if the determinant of the system is zero: $latex \det A=0$, which happens only if the second row is proportional to the first:

$latex \left(\begin{array}{cc}
\frac{\partial f}{\partial x} & \frac{\partial f}{\partial y}\end{array}\right) =c\left(
\begin{array}{cc}
\frac{\partial g}{\partial x} & \frac{\partial g}{\partial y}
\end{array}\right).$

One vector equation is equivalent to two scalar ones: $latex \frac{
\partial f}{\partial x}=c\frac{\partial g}{\partial x},$ $latex \frac{\partial f}{
\partial y}=c\frac{\partial g}{\partial y}.$ Denoting $latex \lambda =-c,$ we obtain two first order conditions for the Lagrangian:

(6) $latex \frac{\partial L}{\partial x}=\frac{\partial f}{\partial x}+\lambda
\frac{\partial g}{\partial x}=0,$ $latex \frac{\partial L}{\partial y}=\frac{
\partial f}{\partial y}+\lambda \frac{\partial g}{\partial y}=0.$

The third one is just the constraint:

(7) $latex \frac{\partial L}{\partial \lambda }=g(x,y)=0.$

We have proved the following result:

Theorem. Let the implicit function existence condition be satisfied. Then there exists a number $latex \lambda $ such that the solution of the constrained problem (1) satisfies first order conditions (6)+(7) for the Lagrangian (it must be a critical point).