Importance of implicit function theorem for optimization
The ultimate goal is to study the Lagrange method for optimization with equality constraints. However, this is impossible to do without the implicit function theorem. For background, on the way to the Lagrange method, we first consider a home-made method for solving the problem:
maximize subject to
Geometry. The constraint defines a surface in the 3D space. The intersection of that surface with the surface gives us the curve that we need to minimize.
Implicit function theorem
Example 1. We solve this problem by reducing it to a one-dimensional case.
Namely, the constraint can be solved for giving
Geometrically, the constraint defines a straight line on the plane . The set of points in the 3D space is a vertical plane through the line because the constraint does not contain any restrictions on and, for each , can be arbitrary. This vertical plane cuts the surface along a curve which we need to minimize.
Plugging (1) in , we have a function of one variable:
Obviously, this is minimized at , which gives Thus, the point is the solution.
Let's think again about the solution presented. Did you see an implicit function in that solution? Solving an equation for means exactly finding as an implicit function of . In our case it was easy to find (1). We need a condition that would guarantee the existence of an implicit function in the general case.
Example 2. Let Can we find from Obviously, no, because is not in the equation. We need a condition that makes sure that indeed depends on in a nontrivial way.
is such a condition, according to the next theorem.
Implicit function theorem. If (2) holds at some point then the equation
defines as an implicit function of , in some neighborhood of
Example 3. The equation
describes a circle of radius 1 centered at the origin. In this case Thus, when we can solve (4) for For we have and for we have
Remark. Condition (2) also allows us to find the derivative When the implicit function determined by (3) exists, (3) in fact becomes
Differentiating both sides with respect to and using the 2D version of the chain rule we get
Because of (2), from here we can find the derivative
In particular, for Example 3 for we have which could be found directly from our solution for