workspace-course-topics

Bellman Stage Decomposition

Prerequisites:

Objective

Take a paper with a dynamic optimization problem involving multiple choice variables and decompose its Bellman equation into the modular stage/perch architecture introduced in class.

Your starting point is Section 12 (“Multiple Control Variables”) and Section 13 (“Modular Stage Architecture”) of SolvingMicroDSOPs.ipynb. Those sections show how the consumption-portfolio problem is broken into modular stages with arrival, decision, and continuation perches. Your assignment is to produce a notebook that does the same thing for the Bellman problem in your chosen paper — using the relevant parts of Sections 12–13 as a template for how to describe and compose stages.

You do not need to replicate the earlier sections of SolvingMicroDSOPs.ipynb (the single-control solution methods, grid construction, etc.). The point is to think carefully about the mathematical structure: what are the state variables, control variables, shocks, and how does the within-period problem decompose into stages?

Choose your paper

Pick a paper whose model has a Bellman equation with at least two choice variables (or one choice variable plus a non-trivial shock structure that creates multiple stages). Good candidates:

If you are unsure whether your paper qualifies, ask: “Does the agent make more than one decision per period, or is there a natural way to split the within-period problem into sequential stages?” If yes, it qualifies.

Set up

  1. Clone SolvingMicroDSOPs (if you haven’t already): 
    cd ~/github
git clone https://github.com/llorracc/SolvingMicroDSOPs.git
cd SolvingMicroDSOPs
  2. Read Sections 12–13 of SolvingMicroDSOPs.ipynb carefully — these are your template:
    • Section 12 (“Multiple Control Variables”) introduces the portfolio share as a second control alongside consumption, showing what a multi-control problem looks like in practice
    • Section 13 (“Modular Stage Architecture”) decomposes that same problem into disc, cons-noshocks, and portable stages, each with arrival/decision/continuation perches, and shows how stages compose right-to-left into a period

    You do not need to work through Sections 1–11 (single-control solution methods). Sections 12–13 are what your notebook should resemble in scope and structure.

  3. Study the Imai-Keane worked examples. There are two companion notebooks for Imai & Keane (2004), the lifecycle consumption–labor-supply model from John Green’s REMARK (JohnRGreen/ImaiKeane_replication):

    • ImaiKeane2004_bellman_one-stage_original.ipynb — the one-stage decomposition. It shows that the original Imai-Keane model has a single decision node per period (the agent chooses consumption and hours simultaneously after observing both shocks), and carefully works through the state variable types, shock placement, and connector.

    • ImaiKeane2004_two-stage-decomposition.ipynb — the two-stage decomposition, which breaks the same problem into a labor-supply stage followed by a consumption stage. This notebook shows how a problem with one simultaneous choice of two variables can be recast as two sequential one-variable problems, and is the direct model for this assignment. Study its structure carefully: how each stage’s arrival, decision, and continuation perches are defined; how the reward is split across stages; how the stages compose right-to-left; and how the inter-period connector works.

    Your task is to do the same kind of decomposition for your chosen paper, at the same level of detail as ImaiKeane2004_two-stage-decomposition.ipynb.

What to produce

Create a notebook called {PAPER}_bellman-stages.ipynb where {PAPER} is a short identifier for your paper (e.g., the citekey: ImaiKeane2004, Carroll1997buffer, etc.).

The notebook should contain the following sections. Use markdown cells liberally — the reasoning matters more than the code.

Section 1: The paper and its model

Section 2: Why multiple stages?

Section 3: Stage decomposition

For each stage in your decomposition:

Also describe the connector between periods. The connector is not a stage — it links the continuation perch of the last stage in period $t$ to the arrival perch of the first stage in period $t+1$. It handles the mechanical state transition (returns, income) and any expectation over shocks that will be realized at next period’s arrival. Nothing happens “between periods” except connectors.

Important: Shocks are realized at the arrival perch of the stage where they are first observed — not between periods, and not at a separate “shock-realization stage.”

Show how the stages compose into a full period, analogous to how SolvingMicroDSOPs.ipynb Section 13.4 composes portable → cons-noshocks → disc right-to-left.

Example — Imai-Keane (2004): See ImaiKeane2004_two-stage-decomposition.ipynb for a complete two-stage decomposition: labor-supply → consumption → [connector]. Both shocks ($\xi$ and $\epsilon$) resolve at the arrival perch of the labor-supply stage; the agent chooses hours $h$ there, then passes resources $m$ and updated human capital $s’$ to the consumption stage where $c$ is chosen by EGM. Discounting is folded into the inter-period connector. The key insight is that labor supply affects both current resources and the law of motion for human capital — a coupling that the monolithic formulation hides inside a single max operator, but which is structurally explicit in the stage decomposition’s first-order condition. Your decomposition should achieve something analogous for your paper.

Section 4: Comparison with the monolithic formulation

Deliverable

{PAPER}_bellman-stages.ipynb — pushed to your REMARK repo (or your fork of SolvingMicroDSOPs on a branch named after your paper).

What to check before submitting:

If you get stuck