ConsIndShockModel doc update

[llorracc]

Main edit is to add Bellman equations for cyclical model example

[sbenthall]

I think the market asset update equations are inconsistent with respect to their dependence on c_t or c_{t-1}. Which is it?

But as a deeper issue, it seems to me that construing the problem as a system of multiple Bellman functions for multiple states within a single time period t breaks the abstraction.

Which period is the "postState" for t?

What optimality guarantees are there for problems construed in this way?

Why not instead model the problem in more traditional Bellman form, but with a new state variable for the quarters?

[llorracc]

I think the market asset update equations are inconsistent with respect to their dependence on c_t or c_{t-1}. Which is it?

Not sure what you mean. If you propose an edit I can look at it.

But as a deeper issue, it seems to me that construing the problem as a system of multiple Bellman functions for multiple states within a single time period t breaks the abstraction.

Not sure what you mean by that either.

Which period is the "postState" for t?

t+1

You can treat the quarter as a state. But why would you? That would be useful only if there was a chance that you could move from quarter 0 to quarter 3 without going through quarters 1 and 2.

And, you would then need to be careful to make sure that you used the period-appropriate transition equation, so there's an extra bit of baggage there.

Note that there's no reason the subperiods need to be separated in time. The Bellman equation for the portfolio problem has consumers simultaneously choosing consumption and the portfolio share, but when we actually solve the problem we do it sequentially: Given assets at the end of the period $a$, find the optimal portfolio share, then construct marginal value for each gridpoint of $a$. Then find the corresponding optimal choice of $c$ via the EGM. Look at the discussion of the portfolio problem in SolvingMicroDSOPs; a less entertaining alternative notation to my gothic v for end-of-period value would have been to call it v_{t,1}.

We do need to define a consistent notation for subperiods. I'm not wedded to the, e.g., $v_{t,n}$ version here; one could imagine, for example, something like $v^{n}{t}$ or $v{t}[n]$ or $v[t].n$ or many other choices. This is something we should discuss among ourselves and with Pablo, and codify in the NARK.

[sbenthall]

Not sure what you mean. If you propose an edit I can look at it.

Oh, I think I understand what's going better now based on your comment.

But as a deeper issue, it seems to me that construing the problem as a system of multiple Bellman functions for multiple states within a single time period t breaks the abstraction.

Not sure what you mean by that either.

Let me try a different question.
Is it possible to rewrite the 4 value functions in such a way that it is a single value function in Bellman form?

You can treat the quarter as a state. But why would you? That would be useful only if there was a chance that you could move from quarter 0 to quarter 3 without going through quarters 1 and 2.

I appreciate this argument.

I see now that in this notation, you have put the subperiod structure inside an MDP-level timestep.
I think that's a viable way to do things.
I think this way of setting up the equations is a bit at odds with the current HARK implementation.

If I am not mistaken, HARK in essence stretches out a cycle so for each what-you-call-"sub-period" is a whole discrete MDP-level time step.
What I was proposing, with a state variable c' = (c + 1) mod C, and indexing into parameters vectors, was more consistent with the representation implied by that implementation.

I think it would be best if the consistent subperiod notation we settle on was consistent with the HARK implementation.
As a design goal, I would prefer it if it were very easy to go directly from a Bellman equation to a functioning model with a working simulator.

[llorracc]

If I am not mistaken, HARK in essence stretches out a cycle so for each what-you-call-"sub-period" is a whole discrete MDP-level time step.

Right. The twist is that you are solving a different Bellman equation in the next step than in the current step. For example, in the portfolio problem, you are solving an optimal portfolio share problem given a lump of assets a. No consumption choice is involved. But once you've determined optimal portfolio choice for any given a, you are then ready to solve the optimal consumption problem (without any portfolio choice).

I think it would be best if the consistent subperiod notation we settle on was consistent with the HARK implementation.

I think it will be more important that it is consistent with the Dolo/DolARK implementation. Of course, ideally, all three of them would be consistent, but Dol- is more important than HARK because kind of the point of HARK is that you have the freedom to solve the problem however you like, while the point of Dol- is to define the problem rigorously in hopes that eventually there will be general purpose solutions.

As a design goal, I would prefer it if it were very easy to go directly from a Bellman equation to a functioning model with a working simulator.

Total agreement on this. In fact, the goal should be not just that it is easy, but that it is automatic. That is, if you have properly specified the stochastic process that people perceive when they are solving the Bellman equation, that should be sufficient to generate the stochastic simulation of the model without you having to specify anything else (except maybe the number of agents to simulate).

[llorracc]

For @sbenthall to merge

[sbenthall]

In conversation, we discussed how the equations in this PR are in "sequential" form.
This expresses a model that has the same solution as an equivalent model written in "simultaneous" form.
Chris has argued that a model in sequential form is computationally easier to solve, which is why sequential form is preferred in these HARK-related documents.

We also discussed how a problem may be reduced alternately to:

• Bellman equations that are different for each sub-period in a sequential model
• Bellman equations that are different between periods

A simultaneous form model exists when the model is infinite horizon (whether or not there is a sub-period structure).
(This infinite horizon simultaneous form model is the one that corresponds most directly to an MDP's solution.)

A simultaneous form model can sometimes but not always be decomposed into a sequential form problem.