Cauchy-Schwarz inequality and optimization 2
Here we consider a problem from mt3042 Optimization Guide by M. Baltovic:
Example 8.1. Given the initial state for some
and a discount factor
we need to maximize the discounted stream of future rewards
(1)
The maximization is over actions (you can think of them as consumptions). The initial state
is the initial wealth; at each step, the consumption is chosen between zero and the wealth left from the previous period,
This reduces the amount available for consumption in the next period:
(2)
Step 1. Here we reformulate the problem to remove the states from the constraints. From (2) we have Fixing an integer
then we obtain
(3)
Since in each state consumption
is nonnegative, the states can only decrease over time. On the other hand, they cannot be negative. Hence, the sequence
is nonincreasing and bounded from below by zero. Therefore the limit
exists and is nonnegative. Letting
in (3) we get
The problem becomes: maximize (1) subject to
(4)
Step 2. Getting rid of square roots. Denoting we have to maximize
subject to
Step 3. Applying the result on the Cauchy-Schwarz inequality. The result from the previous post does not depend on the number of variables. For our present needs it is formulated as follows: Let be fixed numbers and let
be variables satisfying the restriction
Then the infinite weighted sum
is maximized at the point
and the maximized value is
Letting we get
and
with the maximized value
And no clumsy terminology and notation from the Infinite Horizon Dynamic Programming method!
Leave a Reply
You must be logged in to post a comment.