Lagrange method: necessary condition
Consider the problem:
(1) maximize the objective function subject to the equality constraint
Lagrange's idea: add a new, artificial variable and consider a new function of three variables The solution of the constrained problem (1) should be equivalent to the solution of the unconstrained problem
(2) maximize .
is called a Lagrangian. Recall the implicit function existence condition
Under this condition we can employ a useful trick: when the implicit function exists, we can differentiate the restriction to obtain
Simple way to solve (1)
Assuming (3), we can find from the restriction and plug into the objective function to obtain a function of one variable
It's enough to find the extremes of this function (we don't need to use the constraint). At an extremum we necessarily have the first order condition
Lagrange went one step further
(4)+(5) is a linear system of equations. To make this clear, let us introduce a matrix and a vector
Then (4)+(5) becomes This is a homogeneous system (the right side is zero) and it has a nonzero solution (at least its first component is not zero). The matrix theory tells us that this is possible only if the determinant of the system is zero: , which happens only if the second row is proportional to the first: One vector equation is equivalent to two scalar ones: Denoting we obtain two first order conditions for the Lagrangian:
The third one is just the constraint:
We have proved the following result:
Theorem. Let the implicit function existence condition be satisfied. Then there exists a number such that the solution of the constrained problem (1) satisfies first order conditions (6)+(7) for the Lagrangian (it must be a critical point).