Raising the bar

Consumption-Savings Problem

Here we look at the solution of

Example 7.5 (mt3042 Optimization Guide by M. Baltovic)

Suppose a consumer, let's call her Ms Thrifty, has an initial wealth of $latex w$. In any period $latex t=1,...,T$, denote the wealth she has at the start of the period by $latex w_t$, so that $latex w_1=w.$ She can choose to spend $latex c_t\in[0,w_t]$, giving her a utility of $latex u(c_t)=c_t$. However, the wealth $latex w_t-c_t$ she does not consume in that period will be stored in a bank account where it earns an interest rate of $latex r>0$ by the beginning of the next period. Thus, she will enter the next time period $latex t+1$ with wealth $latex w_{t+1}=(1+r)(w_t-c_t)$.

Actions of Ms Thrifty can be identified with her spendings $latex \{c_1,...,c_T\}$ and the same spendings are her rewards. Thus, a strategy is a sequence of spendings $latex \{c_1,...,c_T\}$ and the value of that strategy is the sum of rewards $latex V(\{c_1,...,c_T\})=c_1+...+c_T.$ To obtain the solution, we use backwards induction.

Period $latex T$. The reward $latex c_T\in[0,w_T]$ is obviously maximized at $latex c_T=w_T$ so that

(1) $latex c_T=w_T=(1+r)(w_{T-1}-c_{T-1}).$

Period $latex T-1$. According to the Bellman equations, at each state $latex s$ we need to select an action $latex a$ that maximizes the sum

(2) $latex r(s,a)+V(remainder).$

I deliberately don't use the full notation because I prefer the following verbal description: $latex r(s,a)$ is the immediate reward for taking action $latex a$ at state $latex s$ (for taking one step leading us to the next state). Whatever is the next state, $latex remainder$ means an optimal strategy leading us from that state to the final destination. For period $latex T-1$ we use (1) to write (2) as

(3) $latex c_{T-1}+c_T=c_{T-1}+(1+r)(w_{T-1}-c_{T-1})=-rc_{T-1}+(1+r)w_{T-1}.$

Since $latex c_{T-1}\in[0,w_{T-1}],$ (3) is maximized at $latex c_{T-1}=0$, giving

(4) $latex V(\{0,c_{T}\})=(1+r)w_{T-1}$.

Period $latex T-2$. Here (2) is

$latex c_{T-2}+V(\{0,c_{T}\})$ (using (4))

$latex =c_{T-2}+(1+r)w_{T-1}$ (using $latex w_{T-1}=(1+r)(w_{T-2}-c_{T-2})$)

$latex =c_{T-2}+(1+r)^{2}(w_{T-2}-c_{T-2})=[ 1-(1+r)^2]c_{T-2}+(1+r)^2w_{T-2}.$

This is maximized at $latex c_{T-2}=0$ and the value function for three last actions is $latex V(\{0,0,c_T\})=(1+r)^{2}w_{T-2}$. This easily generalizes leading to the solution $latex V(\{0,...,0,c_T\})=(1+r)^{T-1}w_1=(1+r)^{T-1}w.$

Ms Thrifty fasts for $latex T-1$ periods, then gorges in the last period and dies a happy death. Math is a cruel science.