Note: The GitHub repo SolvingMicroDSOPs associated with this document contains python code that produces all results, from scratch, except for the last section on indirect inference. The numerical results have been confirmed by showing that the answers that the raw python produces correspond to the answers produced by tools available in the Econ-ARK toolkit, more specifically those in the HARK which has full documentation. The MSM results at the end have have been superseded by tools in the EstimatingMicroDSOPs repo.

¹Carroll: Department of Economics, Johns Hopkins University, Baltimore, MD, ccarroll@jhu.edu     The notes were originally written for my Advanced Topics in Macroeconomic Theory class at Johns Hopkins University; instructors elsewhere are welcome to use them for teaching purposes. Relative to earlier drafts, this version incorporates several improvements related to new results in the paper “Theoretical Foundations of Buffer Stock Saving” (especially tools for approximating the consumption and value functions). Like the last major draft, it also builds on material in “The Method of Endogenous Gridpoints for Solving Dynamic Stochastic Optimization Problems” published in Economics Letters, available at http://www.econ2.jhu.edu/people/ccarroll/EndogenousArchive.zip, and by including sample code for a method of simulated moments estimation of the life cycle model a la Gourinchas and Parker (2002) and Cagetti (2003). Background derivations, notation, and related subjects are treated in my class notes for first year macro, available at http://www.econ2.jhu.edu/people/ccarroll/public/lecturenotes/consumption. I am grateful to several generations of graduate students in helping me to refine these notes, to Marc Chan for help in updating the text and software to be consistent with Carroll (2006), to Kiichi Tokuoka for drafting the section on structural estimation, to Damiano Sandri for exceptionally insightful help in revising and updating the method of simulated moments estimation section, and to Weifeng Wu and Metin Uyanik for revising to be consistent with the ‘method of moderation’ and other improvements. All errors are my own. This document can be cited as Carroll (2023a) in the references.

1 Introduction

These lecture notes provide a gentle introduction to a particular set of solution tools for the canonical consumption-saving/portfolio allocation problem. Specifically, the notes describe and solve optimization problems for a consumer facing uninsurable idiosyncratic risk to nonfinancial income (e.g., labor or transfer income), first without and then with optimal portfolio choice,¹ with detailed intuitive discussion of various mathematical and computational techniques that, together, speed the solution by many orders of magnitude. The problem is solved with and without liquidity constraints, and the infinite horizon solution is obtained as the limit of the finite horizon solution. After the basic consumption/saving problem with a deterministic interest rate is described and solved, an extension with portfolio choice between a riskless and a risky asset is also solved. Finally, a simple example shows how to use these methods (via the statistical ‘method of simulated moments’ (MSM for short)) to estimate structural parameters like the coefficient of relative risk aversion (a la Gourinchas and Parker (2002) and Cagetti (2003)).

2 The Problem

The usual analysis of dynamic stochastic programming problems packs a great many events (intertemporal choice, stochastic shocks, intertemporal returns, income growth, the taking of expectations, time discounting, and more) into a complex decision in which the agent makes an optimal choice simultaneously taking all these elements into account. For the dissection here, we will be careful to break down everything that happens into distinct operations so that each element can be scrutinized and understood in isolation.

We are interested in the behavior of a consumer who begins period $t$ with a certain amount of ‘capital’

which immediately earns a return factor $R_t$. Simultaneously, the consumer receives noncapital income $y_t$, which is the product of ‘permanent income’ $p_t$ and a transitory shock ${𝜃}_t$:

whose expectation is 1 (that is, before realization of the transitory shock, the consumer’s expectation is that actual income will on average be equal to permanent income $p_t$).

The combination of the capital return and income defines the consumer’s ‘market resources’ (sometimes called ‘cash-on-hand,’ following Deaton (1992)):

available to be spent on consumption $c_t$ for a consumer subject to a liquidity constraint that requires $c_t\le m_t$ (though we are not imposing such a constraint yet—see subsection 6.8). Finally we define

mnemonically as ‘assets-after-all-actions-are-accomplished.’

The consumer’s goal is to maximize discounted utility from consumption over the rest of a lifetime ending at date $T$:

Income evolves according to:

Equation (6) indicates that we are allowing for a predictable average profile of income growth over the lifetime ${\{G\}}_0^T$ (to capture typical career wage paths, pension arrangements, etc).² Finally, the utility function is of the Constant Relative Risk Aversion (CRRA) form, $u(\bullet )={\bullet }^{1-\rho }∕(1-\rho )$.

It is well known that this problem can be rewritten in recursive (Bellman) form:

subject to the Dynamic Budget Constraint (DBC) defined by equation (3), and to the dynamic process for income defined in (6) and to a transition equation that defines next period’s initial capital as this period’s end-of-period assets:

3 Normalization

The single most powerful method for speeding the solution of such models is to redefine the problem in a way that reduces the number of state variables (if at all possible). In the consumption context, the obvious idea is to see whether the problem can be rewritten in terms of the ratio of various variables to permanent noncapital (‘labor’) income $p_t$ (henceforth for brevity, ‘permanent income.’)

In the last period of life $T$, there is no future value, $v_{T+1}=0$ (boldface $v$ denotes the value function in levels; the nonbold normalized counterpart $v$ is defined below), so the optimal plan is to consume everything:

Now define nonbold variables as the bold variable divided by the level of permanent income in the same period, so that, for example, $m_T=m_T∕p_T$; and define $v_T(m_T)=u(m_T)$.³ For our CRRA utility function, $u(xy)=x^{1-\rho }u(y)$, so (9) can be rewritten as

Because we are dividing $t+1$ level variables by $p_{t+1}=G_{t+1}{p}_t$, a normalized return factor emerges:

(We treat $R$ as time-invariant and drop the period subscript that appeared in (3).)

Now define a new optimization problem:

Then it is easy to see that for $t=T-1$, we can write boldface (nonnormalized) $v$ as a function of $v$ (normalized value) and permanent income:

and so on back to all earlier periods (by backward induction: if the factorization holds at $t+1$, substituting into the Bellman equation at $t$ and using the homogeneity of CRRA utility yields the same factorization at $t$). Hence, if we solve the problem (12) which has only a single state variable $m_t$, we can obtain the levels of the value function from (13), and of consumption and all other variables from the corresponding permanent-income-normalized solution objects by multiplying each by $p_t$, e.g. 

We have thus reduced the problem from two continuous state variables to one (and thereby enormously simplified its solution).

For future reference it will be useful to write (12) in the traditional way, by substituting $a_t$ and $k_{t+1}$ into $m_{t+1}$:

4 Notation

4.1 Periods, Stages, Perches

The problem so far assumes that the agent has only one decision. But agents often have multiple choices per period—for example, a consumption decision, a labor supply choice, and a choice of what proportion $\varsigma$ of capital $k$ to invest in a risky vehicle. We identify each stage type in two ways: by a short-name (a textual name, given when the stage is first introduced) and by a control-name (the stage’s control variable, if any). For example, a labor supply stage might have short-name labor and control-name $\ell$; a consumption stage has control-name $c$. A stage list that constitutes a period may therefore be written by short-name, e.g. $[\mathsf{\text{labor}},\mathsf{\text{cons-noshocks}},\mathsf{\text{disc}}]$, or by control-name, e.g. $[\ell ,c,\beta ]$. A modeler might want to explore whether the order in which the stages are solved makes any difference, either to the substantive results or to aspects of the computational solution like speed and accuracy; with this scheme they do so merely by changing the order in which the stages are listed in the specification of the period.

If, as in section 2, we hard-wire into the solution code for each stage an assumption that its successor stage will be something in particular (say, the consumption stage assumes that the portfolio choice is next), then if we want to change the order of the stages (say, labor supply after consumption, followed by portfolio choice), we will need to re-hard-wire each of the stages to know new things about its new successor (for example, the specifics of the distribution of the rate of return on the risky asset must be known by whatever stage precedes the portfolio choice stage).

The cardinal insight of Bellman (1957) is that everything that matters for the solution to the current problem is encoded in a ‘continuation-value function.’

Using that insight, we describe here a framework for isolating the stage problems within a period from each other, and the period from its successors or predecessors in any other period. The advantage of this isolation is that each stage problem becomes a self-contained module: Its internal logic—the computation it performs on value functions—is defined independently of where it sits in the sequence of stages.

Modularity is valuable because it makes exploring such alternative model structures cheap. Using control-name indexing (e.g., the consumption stage by $c$), after considering the stage-order $[\ell ,c,\varsigma ]$, the modeler can reorder the stages to consider, say, the order $[\ell ,\varsigma ,c]$ without rewriting any of the code that solves each individual stage.⁴ What must change are the transitions—the mappings that connect the end of one stage to the beginning of the next—which must be rewired to reflect the new ordering and its implied information structure. The stage-level code itself remains untouched.

4.2 Perches

The key is to distinguish, within each stage’s Bellman problem, three viewpoints or ‘perches’ (we use that word to empasize that the perch does not do anything: It is merely a collection of mathematical and computational functions and objects).

1. Arrival: Incoming state variables (e.g., $k$) are known, but any shocks associated with the stage have not been realized and decision(s) have not yet been made

2. Decision: The agent solves the decision problem for the period

3. Continuation: After all decisions have been made, their consequences are measured by evaluation of the continuing-value function at the values of the ‘outgoing’ state variables (sometimes called ‘post-state’ variables)

This framework is silent about when shocks (if any) occur. In a consumption problem, the usual assumption is that shocks have been realized before the spending decision is made so that the consumer knows their resources when they decide how much to spend. But in a portfolio choice problem, the portfolio share decision must be made before the shock that determines the risky rate of return is known.

This cons-with-shocks stage corresponds to the consumption problem defined above.

The table illustrates notation we can use when analyzing the problem from a context ‘inside’ a particular stage of a specific period. We require that no variable can have more than one meaning or interpretation inside a period, and we prohibit any reference to values of any variables or functions or other model objects from outside the stage (or period). This is why we use different letters, $k$ and $a$, to represent liquid resources before and after the consumption decision, even if ultimately this period’s continuation value of $a$ will transmute into the next period’s initial $k$. (Both $k$ and $a$ are “k-type” variables—investable capital before returns are realized; the distinction from “m-type” variables like $m$, which represent spendable resources after returns, is formalized below.)

In contrast, items like value functions $v$ or expectations operators $𝔼$ have different meanings at different perches; we capture this using a subscript like $\prec$. The fact that all functions in a perch depend on the same state variables (shown in the second column) allows us to write those functions without specifying their arguments.

4.3 Builders and Connectors

Modularity requires that objects inside a period have no direct access to objects from any other period. This means that we must endow a stage or a period, at the time of its creation, with its end-of-stage-or-period value function $v_{\succ }$.

For example, in a model in which every period contains only the single-stage consumption problem above, the continuation value function for the last (and only) stage at t will need to be ’connected’ to the arrival value function in (t+1), which of necessity requires us to use t-related notation. Concretely, if we designate the end-of-period value function as $v\succ (t)$ (which is defined as the continuation value function from the last stage in the period), we use the notation

to describe what the builder does when constructing the predecessor to period $t+1$. The use of the ‘$:=$’ signals creation: the left-hand side is brought into existence by the builder.

Expectation operators across perches. The subscript on an expectation operator $𝔼$ indicates the information set at that perch: ${𝔼}_{\prec }$ conditions on the Arrival state but not on any shocks realized between Arrival and Decision. For adjacent perches at a period boundary—$𝔼\succ (t)$ (Continuation of period $t$) and ${𝔼}_{\prec (t+1)}$ (Arrival of period $t+1$)—the information sets are identical, so the two operators are mathematically interchangeable; the notational distinction reflects viewpoint (looking backward from $t$’s exit versus forward from $(t+1)$’s entry).

Tying two adjacent stages or periods together also requires that we define a Connector, $C$, which specifies the relationship between the continuation-perch state variable(s) of the predecessor to the arrival-perch state variable(s) of the successor. Again concretely, for two successive periods each of which contains only a single consumption stage like the one described above, the Connector would look like:

State-variable types. A Connector is a pure rename: it asserts that two variables from adjacent stages or periods are different names for the same object. This is only meaningful if the two variables are of the same type. In our framework, state variables fall into two types:

• k-type (capital): investable assets before returns and income are realized. Examples: $k$ (beginning-of-period capital) and $a$ (end-of-period assets after consumption, awaiting next period’s returns).

• m-type (market resources): spendable resources after returns and income shocks have been realized—what Deaton (1992) calls “cash-on-hand.” Examples: $m$ (market resources at the Decision perch) and $\check{m}$ (post-shock market resources within a period).

The Connector $C(a\leftrightarrow k)$ is valid because both $a$ and $k$ are k-type; $C(\check{m}\leftrightarrow m)$ is valid because both are m-type. A Connector that crossed types—say, $C(m\leftrightarrow k)$—would be illegal, because market resources and investable capital are not merely different names for the same quantity: converting between them requires a substantive transformation (the realization of returns and income).

4.4 Building the Pile Backward

We call the structure in which accumulating periods are stored the pile $P$. Once backward induction is complete, the pile holds the full definition—what might colloquially be called “the full model.” (We avoid the term “model” here because it is used in too many other ways and contexts.) Since we accrete the elements of $P$ one by one as we iterate backward, it can be thought of as a pile of defined periods (and the Connectors between them).

The process of building the pile is straightforward. We start from the terminal period (section 3: $v_T(m)=u(m)$, consume everything), so initially $P=\{p_T\}$. We will denote the ‘builder’ as a computational object with notation like $B\leftarrow prd$, and we will speak of the backward-induction creation of a new period as being the result of ‘applying‘ the $B\leftarrow prd$ to the existing $P$. The backward builder augments the $p$ by creating a new Connector and then the new period’s solution, so the pile then has the form $P=\{p_{T-1},C_{T-1,T},p_T\}$. Section 6 details the construction of $p_{T-1}$; with multiple stages or control variables the structure generalizes as in section 8.

5 The Usual Theory, and a Bit More Notation

For reference and to illustrate our new notation, we will now derive the Euler equation and other standard results for the problem described above. Since we can write value as of the end of the consumption stage as a function of $a_t$:

Derivative notation convention. Throughout this document, a superscript $\partial$ on a value function means its derivative with respect to the relevant state variable at that perch: $v_{\succ }^{\partial }(a)\equiv d{v}_{\succ }∕da$, $v_{\sim }^{\partial }(m)\equiv d{v}_{\sim }∕dm$. In code this maps to the .vda attribute on each Perch object (e.g., stg[’cntn’].vda for $v_{\succ }^{\partial }$). The “consumed function” $c_{\succ }(a)\equiv {(v_{\succ }^{\partial }(a))}^{-1∕\rho }$ (the inverse-marginal-value transformation) is computed inside the EGM step of solve_one_period.

The first order condition for (14) with respect to $a_t$⁵ is

where we are using a superscript $\partial$ to denote the first derivative of a function (e.g. $v^{\partial }$ is the derivative of $v$ with respect to its argument).⁶ For functions of more than one argument, we append the variable name: $v^{\partial x}$ denotes the partial derivative of $v$ with respect to $x$. Because the Envelope theorem tells us that

we can substitute the LHS of (19) for the RHS of (18) to get

and rolling forward one period,

so that substituting the LHS in equation (18) finally gives us the Euler equation for consumption:

The derivation above used period-qualified subscripts (e.g., $v_{\succ (t)}$, $v_{\sim (t+1)}^{\partial }$) because the Euler equation relates objects across periods. Macro shorthands like $v\succ (t)$ expand to $v_{\succ (t)}$, i.e. $v_{\succ (t)}$. We can now restate the problem (14) using the simpler within-stage notation, which drops the period qualifier:

whose first order condition with respect to $c$ is

which is mathematically equivalent to the usual Euler equation for consumption.⁷

We will revert to this formulation when we reach subsection 6.9.

6 Solving the Next-to-Last Period

This section examines the second-to-last period in detail, illustrating a number of powerful techniques for speeding and improving its solution; in doing so it illustrates how the pile $P$ (subsection 4.4) is built backward. We set $G_t=1$ here to reduce clutter.

6.1 A Direct Expression for $c_{T-1}$

If $t=T$, the second-to-last-period decision-perch problem is

Using (0) $t=T$; (1) $v_{\sim (t)}(m)=u(m)$; (2) the definition of $u(m)$; and (3) the definition of the expectations operator,

where $F(𝜃)$ is the cumulative distribution function for $𝜃$, this maximization problem implicitly defines a ‘period-and-stage-local function’ $c$ that yields optimal consumption in period $t-1$ for any specific numerical level of resources like $m=1.7$. The explicit statement of the problem is

But because there is no general analytical solution, for any given $m$ we must use numerical tools to find the $c$ that maximizes the expression. This is excruciatingly slow: for every candidate $c$, a definite integral over $(0,\infty )$ must be calculated numerically, and optimization is itself a costly operation, so the combination of the two is a double-whammy for slowdown.

6.2 Discretizing the Distribution

Our first speedup trick is therefore to construct a discrete approximation to the lognormal distribution that can be used in place of numerical integration. That is, we want to approximate the expectation over $𝜃$ of a function $g(𝜃)$ by calculating its value at set of $n_{𝜃}$ points ${𝜃}_i$, each of which has an associated probability weight $w_i$:

(because adding $n$ weighted values to each other is enormously faster than general-purpose numerical integration).

Such a procedure is called a ‘quadrature’ method of integration; Tanaka and Toda (2013) survey a number of options, but for our purposes we choose the one which is easiest to understand: An ‘equiprobable’ approximation (that is, one where each of the values of ${𝜃}_i$ has an equal probability, equal to $1∕n_{𝜃}$).

We calculate such an $n$-point approximation as follows.

Define a set of points from ${♯}_0$ to ${♯}_{n_{𝜃}}$ on the $[0,1]$ interval as the elements of the set $♯=\{0,1∕n,2∕n,\dots ,1\}$.⁸ Call the inverse of the $𝜃$ distribution $F_{}^{-1}$, and define the points ${♯}_i^{-1}=F_{}^{-1}({♯}_i)$. Then the conditional mean of $𝜃$ in each of the intervals numbered 1 to $n$ is:

and when the integral is evaluated numerically for each $i$ the result is a set of values of $𝜃$ that correspond to the mean value in each of the $n$ intervals.

The method is illustrated in Figure 1. The solid continuous curve represents the “true” CDF $F(𝜃)$ for a lognormal distribution such that $𝔼[𝜃]=1$, ${\sigma }_{𝜃}=0.1$. The short vertical line segments represent the $n_{𝜃}$ equiprobable values of ${𝜃}_i$ which are used to approximate this distribution.⁹

The following notebook snippet constructs these points.

We now substitute our approximation (29) for $v\succ (t-1)(a)$ in (25) which is simply the sum of $n_{𝜃}$ numbers and is therefore easy to calculate (compared to the full-fledged numerical integration (26) that it replaces).

6.3 The Approximate Consumption and Value Functions

Given any particular value of $m$, a numerical maximization tool can now find the $c$ that solves (25) in a reasonable amount of time.

The notebook responsible for computing an estimated consumption function begins in “Solving the Model by Value Function Maximization,” where a vector of possible values of market resources $m$ is created. In these notes we use $m$ for such a vector (e.g. $m[1]$ the first entry, $m[-1]$ the last). For illustration we take the grid to be the first five nonnegative integers, $\{0,1,2,3,4\}$.

6.4 An Interpolated Consumption Function

This is accomplished in “An Interpolated Consumption Function,” which generates an interpolating function that we designate ${\stackrel{`}{c}}_{\sim (t-1)}(m)$.

Figures 2 and 3 show plots of the constructed ${\stackrel{`}{c}}_{t-1}$ and ${\stackrel{`}{v}}_{t-1}$. While the ${\stackrel{`}{c}}_{t-1}$ function looks very smooth, the fact that the ${\stackrel{`}{v}}_{t-1}$ function is a set of line segments is very evident. This figure provides the beginning of the intuition for why trying to approximate the value function directly is a bad idea (in this context).¹⁰

6.5 Interpolating Expectations

Piecewise linear ‘spline’ interpolation as described above works well for generating a good approximation to the true optimal consumption function. However, there is a clear inefficiency in the program: Since it uses equation (25), for every value of $m$ the program must calculate the utility consequences of various possible choices of $c$ (and therefore $a_{t-1}$) as it searches for the best choice.

For any given index $j$ in $m[j]$, as it searches for the corresponding optimal $a$, the algorithm will end up calculating $v_{\succ (t-1)}(\tilde{a})$ for many $\tilde{a}$ values close to the optimal $a_{t-1}$. Indeed, even when searching for the optimal $a$ for a different $m$ (say $m[k]$ for $k\ne j$) the search process might compute $v_{\succ (t-1)}(a)$ for an $a$ close to the correct optimal $a$ for $m[j]$. But if that difficult computation does not correspond to the exact solution to the $m[k]$ problem, it is discarded.

The notebook section “Interpolating Expectations,” now interpolates the expected value of ending the period with a given amount of assets.¹¹

Figure 4 compares the true value function to the approximation produced by following the interpolation procedure; the approximated and exact functions are of course identical at the gridpoints of $a$ and they appear reasonably close except in the region below $m=1$.

Nevertheless, the consumption rule obtained when the approximating ${\stackrel{`}{v}}_{\succ (t-1)}(a_{t-1})$ is used instead of $v_{\succ (t-1)}(a_{t-1})$ is surprisingly bad, as shown in figure 5. For example, when $m$ goes from 2 to 3, ${\stackrel{`}{c}}_{t-1}$ goes from about 1 to about 2, yet when $m$ goes from 3 to 4, ${\stackrel{`}{c}}_{t-1}$ goes from about 2 to about 2.05. The function fails even to be concave, which is distressing because Carroll and Kimball (1996) prove that the correct consumption function is strictly concave in a wide class of problems that includes this one.

6.6 Value Function versus First Order Condition

Loosely speaking, our difficulty reflects the fact that the consumption choice is governed by the marginal value function, not by the level of the value function (which is the object that we approximated). To understand this point, recall that a quadratic utility function exhibits risk aversion because with a stochastic $c$,

(where $\cancel{c}$ is the ‘bliss point’ which is assumed always to exceed feasible $c$). However, unlike the CRRA utility function, with quadratic utility the consumption/saving behavior of consumers is unaffected by risk since behavior is determined by the first order condition, which depends on marginal utility, and when utility is quadratic, marginal utility is unaffected by risk:

Intuitively, if one’s goal is to accurately capture choices that are governed by marginal value, numerical techniques that approximate the marginal value function will yield a more accurate approximation to optimal behavior than techniques that approximate the level of the value function.

The first order condition of the maximization problem in period $T-1$ is:

The downward-sloping curve in Figure 6 shows the value of $c^{-\rho }$ for our baseline parameter values for $0\le c\le 4$ (the horizontal axis). The solid upward-sloping curve shows the value of the RHS of (32) as a function of $c$ under the assumption that $m=3$. Optimal consumption given $m=3$ is the $c$ at which the two curves intersect—just below $c=2$. The dashed curve shows the same for $m=4$; its intersection with $u^c(c)$ is slightly below $c=2.5$, so increasing $m$ from 3 to 4 raises optimal consumption by about 0.5.

Now consider the derivative of ${\stackrel{`}{v}}_{(t-1)}$. Because the function is piecewise linear, its derivative ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }(a_{t-1})$ is a step function: constant between adjacent gridpoints, with jumps at each gridpoint.

The solid-line step function in Figure 6 depicts the actual value of ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }(3-c)$. When we attempt to find optimal values of $c$ given $m$ using ${\stackrel{`}{v}}_{\succ (t-1)}(a_{t-1})$, the numerical optimization routine will return the $c$ for which $u^c(c)={\stackrel{`}{v}}_{\succ (t-1)}^{\partial }(m-c)$. Thus, for $m=3$ the program will return the value of $c$ for which the downward-sloping $u^c(c)$ curve intersects with the ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }(3-c)$; as the diagram shows, this value is exactly equal to 2. Similarly, if we ask the routine to find the optimal $c$ for $m=4$, it finds the point of intersection of $u^c(c)$ with ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }(4-c)$; and as the diagram shows, this intersection is only slightly above 2. Hence, this figure illustrates why the numerical consumption function plotted earlier returned values very close to $c=2$ for both $m=3$ and $m=4$.

We would obviously obtain much better estimates of the point of intersection between $u^c(c)$ and $v_{\succ (t-1)}^{\partial }(m-c)$ if our estimate of ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }$ were not a step function. In fact, we already know how to construct linear interpolations to functions, so the obvious next step is to construct a linear interpolating approximation to the expected marginal value of end-of-period assets function at the points in $a$:

yielding $v_{\succ (t-1)}^{\partial }$ (the vector of expected end-of-period-$(T-1)$ marginal values of assets corresponding to $aVec$), and construct ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }(a_{t-1})$ as the linear interpolating function that fits this set of points.

The results are shown in Figure 7. The linear interpolating approximation looks roughly as good (or bad) for the marginal value function as it was for the level of the value function. However, Figure 8 shows that the new consumption function (long dashes) is a considerably better approximation of the true consumption function (solid) than was the consumption function obtained by approximating the level of the value function (short dashes).

6.7 Transformation

Even the new-and-improved consumption function diverges notably from the true solution, especially at lower values of $m$. That is because the linear interpolation does an increasingly poor job of capturing the nonlinearity of $v_{\succ (t-1)}^{\partial }$ at lower and lower levels of $a$.

This is where we unveil our next trick. To understand the logic, start by considering the case where ${ℛ}_t=\beta =G_t=1$ and there is no uncertainty (that is, we know for sure that income next period will be ${𝜃}_t=1$). The final Euler equation (recall that we are still assuming that $t=T$) is then:

In the case we are now considering with no uncertainty and no liquidity constraints, the optimizing consumer does not care whether a unit of income is scheduled to be received in the future period $t$ or the current period $t-1$; there is perfect certainty that the income will be received, so the consumer treats its PDV as equivalent to a unit of current wealth. Total resources available at the point when the consumption decision is made is therefore comprised of two types: current market resources $m$ and ‘human wealth’ (the PDV of future income) of $h_{t-1}=1$ (because it is the value of human wealth as of the end of the period, there is only one more period of income of 1 left).

Of course, this is a highly nonlinear function. However, if we raise both sides of (35) to the power $(-1∕\rho )$ the result is a linear function:

This is a specific example of a general phenomenon: A theoretical literature discussed in Carroll and Kimball (1996) establishes that under perfect certainty, if the period-by-period marginal utility function is of the form $c_t^{-\rho }$, the marginal value function will be of the form ${(\gamma m_t+\zeta )}^{-\rho }$ for some constants $\{\gamma ,\zeta \}$. This means that if we were solving the perfect foresight problem numerically, we could always calculate a numerically exact (because linear) interpolation.

The key insight is that much of the nonlinearity in $v^{\partial }$ comes from raising to the power $-\rho$. By inverting that operation (raising to $-1∕\rho$), we can ‘unwind’ it, and the remaining nonlinearity is much smaller. Specifically, applying the foregoing insights to the end-of-period value function $v_{\sim (t-1)}^{\partial }(a)$, we can define an ‘inverse marginal value’ function

which would be linear in the perfect foresight case.¹² We then construct a piecewise-linear interpolating approximation to the ${\Lambda }_t^{\partial }$ function, ${\stackrel{`}{\Lambda }}_{\succ }^{\partial }(a_t)$, and for any $a$ that falls in the range $\{a[1],a[-1]\}$ we obtain our approximation of marginal value from:

The most interesting thing about all of this, though, is that the ${\Lambda }_t^{\partial }$ function has another interpretation. Recall our point in (24) that $u^c(c_t)=v{\succ }^{\partial }(m_t-c_t)$. Since with CRRA utility $u^c(c)=c^{-\rho }$, this can be rewritten and inverted

This gives ${\Lambda }^{\partial }$ a concrete interpretation: for any ending $a$, it reveals how much the agent must have consumed to (optimally) reach that $a$. We will therefore henceforth refer to it as the ‘consumed function:’

Thus, for example, for period $t-1$ our procedure is to calculate the vector of $c$ points on the consumed function:

with the idea that we will construct an approximation of the consumed function ${\stackrel{`}{c}}_{\succ (t-1)}(a)$ as the interpolating function connecting these $\{a,c\}$ points.

6.8 The Natural Borrowing Constraint and the $a_{t-1}$ Lower Bound

This is the appropriate moment to ask an awkward question: How should an interpolated, approximated ‘consumed’ function like ${\stackrel{`}{c}}_{\succ (t-1)}(a_{t-1})$ be extrapolated to return an estimated ‘consumed’ amount when evaluated at an $a_{t-1}$ outside the range spanned by $\{a[1],...,a[n]\}$?

For most canned piecewise-linear interpolation tools like scipy.interpolate, when the ‘interpolating’ function is evaluated at a point outside the provided range, the algorithm extrapolates under the assumption that the slope of the function remains constant beyond its measured boundaries (that is, the slope is assumed to be equal to the slope of nearest piecewise segment within the interpolated range); for example, if the bottommost gridpoint is $a1=a[1]$ and the corresponding consumed level is $c1=c_{\succ (t-1)}(a_1)$ we could calculate the ‘marginal propensity to have consumed’ ${\varkappa }_1={\stackrel{`}{c}}_{\succ (t-1)}^{\partial }(a1)$ and construct the approximation as the linear extrapolation below $a[1]$ from:

To see that this will lead us into difficulties, consider what happens to the true (not approximated) $v_{\succ (t-1)}^{\partial }(a_{t-1})$ as $a_{t-1}$ approaches a quantity we will call the ‘natural borrowing constraint’: ${\underline{a}}_{t-1}=-\underline{𝜃}{ℛ}_t^{-1}$. From (33) we have

But since $\underline{𝜃}={𝜃}_1$, exactly at $a={\underline{a}}_{t-1}$ the first term in the summation would be ${(-\underline{𝜃}+{𝜃}_1)}^{-\rho }=1∕0^{\rho }$ which is infinity. The reason is simple: $-{\underline{a}}_{t-1}$ is the PDV, as of $t-1$, of the minimum possible realization of income in $t$ (${ℛ}_t{\underline{a}}_{t-1}=-{𝜃}_1$). Thus, if the consumer borrows an amount greater than or equal to $\underline{𝜃}{ℛ}_t^{-1}$ (that is, if the consumer ends $t-1$ with $a_{t-1}\le -\underline{𝜃}{ℛ}_t^{-1}$) and then draws the worst possible income shock in period $t$, they will have to consume zero in period $t$, which yields $-\infty$ utility and $+\infty$ marginal utility.

As Zeldes (1989) first noticed, this means that the consumer faces a ‘self-imposed’ (or, as above, ‘natural’) borrowing constraint (which springs from the precautionary motive): They will never borrow an amount greater than or equal to $\underline{𝜃}{ℛ}_t^{-1}$ (that is, assets will never reach the lower bound of ${\underline{a}}_{t-1}$). The constraint is ‘self-imposed’ in the precise sense that if the utility function were different (say, Constant Absolute Risk Aversion), the consumer might be willing to borrow more than $\underline{𝜃}{ℛ}_t^{-1}$ because a choice of zero or negative consumption in period $t$ would yield some finite amount of utility.¹³

This self-imposed constraint cannot be captured well when the $v_{\succ (t-1)}^{\partial }$ function is approximated by a piecewise linear function like ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }$, because it is impossible for the linear extrapolation below $\underline{a}$ to correctly predict $v_{\succ (t-1)}^{\partial }({\underline{a}}_{t-1})=\infty .$

So, the marginal value of saving approaches infinity as $a\downarrow {\underline{a}}_{t-1}=-\underline{𝜃}{ℛ}_t^{-1}$. But this implies that $\underset{a\downarrow {\underline{a}}_{t-1}}{lim}{c}_{\succ (t-1)}(a)={(v_{\succ (t-1)}^{\partial }(a))}^{-1∕\rho }=0$; that is, as $a$ approaches its ‘natural borrowing constraint’ minimum possible value, the corresponding amount of worst-case $c$ must approach its lower bound: zero.

The upshot is a realization that all we need to do to address these problems is to prepend each of the $a_{t-1}$ and $c_{t-1}$ from (41) with an extra point so that the first element in the mapping that produces our interpolation function is $\{{\underline{a}}_{t-1},0.\}$. This is done in section “The Self-Imposed ‘Natural’ Borrowing Constraint and the $a_{t-1}$ Lower Bound” of the notebook.

Figure 9 shows the result. The solid line calculates the exact numerical value of the consumed function $c_{\succ (t-1)}(a)$ while the dashed line is the linear interpolating approximation ${\stackrel{`}{c}}_{\succ (t-1)}(a).$ This figure illustrates the value of the transformation: The true function is close to linear, and so the linear approximation is almost indistinguishable from the true function except at the very lowest values of $a$.

Figure 10 similarly shows that when we generate ${\stackrel{`}{\stackrel{`}{v}}}_{\succ (t-1)}^{\partial }(a)$ using our augmented ${[{\stackrel{`}{c}}_{\succ (t-1)}(a)]}^{-\rho }$ (dashed line) we obtain a much closer approximation to the true marginal value function $v_{\succ (t-1)}^{\partial }(a)$ (solid line) than we obtained in the previous exercise which did not do the transformation (Figure 7).¹⁴

6.9 The Method of Endogenous Gridpoints (‘EGM’)

The solution procedure above for finding $c_{t-1}(m)$ still requires us, for each point in $m{}_{t-1}$, to use a numerical rootfinding algorithm to search for the value of $c$ that solves $u^c(c)=v_{\succ (t-1)}^{\partial }(m-c)$. Though sections 6.7 and 6.8 developed a highly efficient and accurate procedure to calculate ${\stackrel{`}{v}}_{\succ (t-1)}^{\partial }$, those approximations do nothing to eliminate the need for using a rootfinding operation for calculating, for an arbitrary $m$, the optimal $c$. And rootfinding is a notoriously computation-intensive (that is, slow!) operation.

Fortunately, it turns out that there is a way to completely skip this slow rootfinding step. The method can be understood by noting that we have already calculated, for a set of arbitrary values of $a=a{}_{t-1}$, the corresponding $c$ values for which this $a$ is optimal.

But with mutually consistent values of $c{}_{t-1}$ and $a{}_{t-1}$ (consistent, in the sense that they are the unique optimal values that correspond to the solution to the problem), we can obtain the $m{}_{t-1}$ vector that corresponds to both of them from

These $m$ gridpoints are “endogenous” in contrast to the usual solution method of specifying some ex-ante (exogenous) grid of values of $m$ and then using a rootfinding routine to locate the corresponding optimal consumption vector $c$.

This routine is performed in the “Endogenous Gridpoints” section of the notebook. First, the endOfPrd.cCntn_Tm1 function is called for each of the pre-specified values of end-of-period assets stored in $aVec$. These values of consumption and assets are used to produce the list of endogenous gridpoints, stored in the object mVec_egm. With the $c$ values in hand, the notebook can generate a set of $m{}_{t-1}$ and $c{}_{t-1}$ pairs that can be interpolated between in order to yield ${\stackrel{`}{c}}_{\sim (t-1)}(m)$ at virtually zero computational cost!¹⁵

One might worry about whether the $\{m,c\}$ points obtained in this way will provide a good representation of the consumption function as a whole, but in practice there are good reasons why they work well (basically, this procedure generates a set of gridpoints that is naturally dense right around the parts of the function with the greatest nonlinearity).

Figure 11 plots the actual consumption function $c_{t-1}$ and the approximated consumption function ${\stackrel{`}{c}}_{t-1}$ derived by the method of endogenous grid points. Compared to the approximate consumption functions illustrated in Figure 8, ${\stackrel{`}{c}}_{t-1}$ is quite close to the actual consumption function.

6.10 Improving the $a$ Grid

Thus far, we have arbitrarily used $a$ gridpoints of $\{0.,1.,2.,3.,4.\}$ (augmented in the last subsection by ${\underline{a}}_{t-1}$). But it has been obvious from the figures that the approximated ${\stackrel{`}{c}}_{\succ (t-1)}$ function tends to be farthest from its true value at low values of $a$. Combining this with our insight that ${\underline{a}}_{t-1}$ is a lower bound, we are now in position to define a more deliberate method for constructing gridpoints for $a$ – a method that yields values that are more densely spaced at low values of $a$ where the function is more nonlinear.

A pragmatic choice that works well is to find the values such that (1) the last value exceeds the lower bound by the same amount $\overline{a}$ as our original maximum gridpoint (in our case, 4.); (2) we have the same number of gridpoints as before; and (3) the multi-exponential growth rate (that is, $e^{e^{e^{...}}}$ for some number of exponentiations $n$ – our default is 3) from each point to the next point is constant (instead of, as previously, imposing constancy of the absolute gap between points).

Section “Improve the $agrid$” begins by defining a function which takes as arguments the specifications of an initial grid of assets and returns the new grid incorporating the multi-exponential approach outlined above.

Notice that the graphs depicted in Figures 12 and 13 are notably closer to their respective truths than the corresponding figures that used the original grid.

6.11 Program Structure

In section “Solve for $c_t(m)$ in Multiple Periods,” the natural and artificial borrowing constraints are combined with the endogenous gridpoints method to approximate the optimal consumption function for a specific period. Then, this function is used to compute the approximated consumption in the previous period, and this process is repeated for some specified number of periods.

The essential structure of the program is a loop that iteratively solves for consumption functions by working backward from an assumed final period, using the dictionary cFunc_life to store the interpolated consumption functions up to the beginning period. Consumption in a given period is utilized to determine the endogenous gridpoints for the preceding period. This is the sense in which the computation of optimal consumption is done recursively.

In the terminology of section 4.3, each iteration of this backward loop is an invocation of the backward builder $B\leftarrow prd$: it creates the Connector (inserted into $P$ between the new and existing period solutions), uses it and the already-solved next period to perform the creation of $v\succ (t)$, and solves for the optimal consumption rule. The dictionary cFunc_life is the computational embodiment of $P$—the growing structure of solved periods and Connectors between them assembled by backward induction.

For a realistic life cycle problem, it would also be necessary at a minimum to calibrate a nonconstant path of expected income growth over the lifetime that matches the empirical profile; allowing for such a calibration is the reason we have included the ${\{G\}}_t^T$ vector in our computational specification of the problem.

6.12 Results

The code creates the relevant ${\stackrel{`}{c}}_t(m)$ functions for any period in the horizon, at the given values of $m$. Figure 14 shows ${\stackrel{`}{c}}_{T-n}(m)$ for $n=\{20,15,10,5,1\}$. At least one feature of this figure is encouraging: the consumption functions converge as the horizon extends, something that Carroll (2023b) shows must be true under certain parametric conditions that are satisfied by the baseline parameter values being used here.

The construction of $B\leftarrow prd$ in this single-stage case uses the builders and connectors of subsection 4.3.

7 The Infinite Horizon

All of the solution methods presented so far have involved period-by-period iteration from an assumed last period of life, as is appropriate for life cycle problems. However, if the parameter values for the problem satisfy certain conditions (detailed in Carroll (2023b)), the consumption rules (and the rest of the problem) will converge to a fixed rule as the horizon (remaining lifetime) gets large, as illustrated in Figure 14. Furthermore, Deaton (1991), Carroll (1992; 1997) and others have argued that the ‘buffer-stock’ saving behavior that emerges under some further restrictions on parameter values is a good approximation of the behavior of typical consumers over much of the lifetime. Methods for finding the converged functions are therefore of interest, and are dealt with in this section.

Of course, the simplest such method is to solve the problem as specified above for a large number of periods. This is feasible, but there are much faster methods.

7.1 Convergence

In solving an infinite-horizon problem, it is necessary to have some metric that determines when to stop because a solution that is ‘good enough’ has been found.

A natural metric is defined by the unique ‘target’ level of wealth that Carroll (2023b) proves will exist in problems of this kind under certain conditions: The $\widehat{m}$ such that

where the accent is meant to signify that this is the value that other $m$’s ‘point to.’

Given a consumption rule $c(m)$ it is straightforward to find the corresponding $\widehat{m}$. So for our problem, a solution is declared to have converged if the following criterion is met: $\left|{\widehat{m}}_{t+1}-{\widehat{m}}_t\right|<𝜖$, where $𝜖$ is a very small number and defines our degree of convergence tolerance.

Similar criteria can obviously be specified for other problems. However, it is always wise to plot successive function differences and to experiment a bit with convergence criteria to verify that the function has converged for all practical purposes.

8 Multiple Control Variables and Modularity

We now consider problems with multiple control variables. Section 8.1 presents the joint consumption-and-portfolio optimization as a canonical example: we derive the simultaneous first-order conditions and show that direct numerical solution of the multidimensional problem is computationally expensive—but that decomposing it into a sequence of single-control problems is much faster. Section 8.2 develops the stage-based machinery for such decompositions, building on the modular notation from section 4: the discounting stage (disc), the standalone portfolio optimization (section 8.2.2), and the general-purpose portable returns stage that unifies portfolio choice and shock realization. Section 8.2.5 combines these building blocks into three canonical period types.

8.1 The Joint Optimization Problem

Our canonical example of a multi-control problem is the case where the agent has both a consumption choice and a decision about $\varsigma$ (archaic Greek stigma): the share of assets in the risky asset. Given a risky-asset return factor $R$ and a riskless return factor $R$, the realized portfolio return is

Written as a single combined decision (substituting the budget constraint):

Whether the stochastic variables $R$ and $𝜃$ are revealed at the end of the current period or the start of the next does not affect this equation. The first-order conditions for this joint problem are:

Direct simultaneous solution of these two conditions is computationally expensive; the remainder of this section develops modular stage-based machinery that decomposes the problem into a sequence of simpler single-control optimizations.

8.2 Decomposing Into Modular Stages

The single-stage-per-period formulation from sections 2–6 is equivalent to a sequence of simpler stages, each with a single job. We decompose it in that way so that adding portfolio choice later requires no change to the consumption-stage code—only the stage list and the Connectors between stages change (see section 4). The decomposition yields the same value functions, consumption function, and Euler equation.

8.2.1 The Discounting Stage (disc)

The simplest decomposition isolates discounting into a standalone stage that applies for the entire period. We call it the disc stage (control-name $\beta$). From equation (15), $v\succ (t):=\beta v\prec (t+1)$; the factor $\beta$ was implicitly inside the backward builder $B\leftarrow prd$ that created $v\succ$. Instead, henceforth at the end of every period, we put the stub disc stage which has no choice, no shocks, and a trivial decision value function $v\sim =\beta \cdot v\succ$, and we set the discount factor to 1 for every prior stage in the period (leaving all the discounting to the disc stage).¹⁶

The two-stage period has stage list $[\mathsf{\text{cons-with-shocks}},\mathsf{\text{disc}}]$ (more compactly, $[c,\beta ]$). The convention that every non-disc stage is undiscounted and every period ends with disc will mean we do not have to rearrange discounting as we rearrange stages within the period.

8.2.2 The Standalone Portfolio Problem

Consider the standalone problem of an ‘investor’ choosing the portfolio share $\varsigma$. The state variable at the start is $k$ (capital available for investment); the Decision chooses $\varsigma$; stochastic shocks ($R$, $𝜃$) are then realized, yielding the Continuation state $\check{m}=\Re (\varsigma )k+𝜃$. The portfolio share must be chosen before the return shock $R$ is realized. We write $\check{m}$ rather than $m$ for the post-shock state because a strict rule of our framework is the prohibition of multiple timings of a given variable within a period. Note that $\check{m}$ is m-type (market resources after shocks), matching $m$; the $ˇ$ decoration distinguishes the two timings while preserving the type (see section 4.3).

The first-order condition with respect to $\varsigma$ is:

where $v_{\succ }^{\partial }$ denotes the derivative of the continuation value function with respect to $\check{m}$.

The Decision equation yields the portfolio share function:

8.2.3 The portable Stage

The portfolio optimization problem above can be generalized into a single configurable returns-and-shocks stage by making the portfolio share an optional parameter rather than a control variable that must always be optimized. We call this the portable stage (“portfolio-able”): it is “able” to perform portfolio optimization or not, depending on a parameter $\overline{\varsigma }$.

The stage’s Arrival state includes the investable capital $k$ and an optional portfolio share $\overline{\varsigma }\in [0,1]\cup \{\perp \}$, where $\perp$ means “absent—optimize.” The behavior of the stage depends on $\overline{\varsigma }$:

• Optimize mode ($\overline{\varsigma }=\perp$): The stage solves the portfolio optimization problem from section 8.2.2, choosing $\varsigma$ optimally via (50) with first-order condition (49).

• Fixed-share mode ($\overline{\varsigma }\in [0,1]$): The portfolio share is prescribed; no optimization occurs. The value function is $v\prec (k,\overline{\varsigma })={𝔼}_{\prec }[v\succ (\Re (\overline{\varsigma })k+𝜃)]$.

• No-risky-investment mode ($\overline{\varsigma }=0$): A special case of fixed-share mode. All assets earn the riskless return: $\check{m}=Rk+𝜃$. The risky return $R$ is irrelevant. The value function reduces to $v\prec (k,0)={𝔼}_{\prec }[v\succ (Rk+𝜃)]$.

The perches of this stage are:

When $\overline{\varsigma }=\perp$, the Decision value function is $v\sim =\underset{\varsigma }{max}{𝔼}_{\sim }[v\succ ]$. When $\overline{\varsigma }\in [0,1]$, it is $v\sim =𝔼[v\succ ]$ with $\varsigma$ fixed at $\overline{\varsigma }$.

Argument notation. When a stage can be constructed with a fixed parameter, we write the value of that argument in parentheses. Thus $\varsigma (0)$ denotes the portable stage with $\overline{\varsigma }=0$ (the shocks-only configuration described below), and $\varsigma (\perp )$ denotes portable in optimize mode.

8.2.4 Separating Shocks from Choices

The cons-with-shocks stage in section 4.2 combines the $k\to m$ shock realization and the $m\to a$ consumption decision. We split these into two stages.