-
-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ConsIndShockModel doc update #600
Conversation
I think the market asset update equations are inconsistent with respect to their dependence on c_t or c_{t-1}. Which is it? But as a deeper issue, it seems to me that construing the problem as a system of multiple Bellman functions for multiple states within a single time period t breaks the abstraction. Which period is the "postState" for t? What optimality guarantees are there for problems construed in this way? Why not instead model the problem in more traditional Bellman form, but with a new state variable for the quarters? |
Not sure what you mean. If you propose an edit I can look at it.
Not sure what you mean by that either.
t+1 You can treat the quarter as a state. But why would you? That would be useful only if there was a chance that you could move from quarter 0 to quarter 3 without going through quarters 1 and 2. And, you would then need to be careful to make sure that you used the period-appropriate transition equation, so there's an extra bit of baggage there. Note that there's no reason the subperiods need to be separated in time. The Bellman equation for the portfolio problem has consumers simultaneously choosing consumption and the portfolio share, but when we actually solve the problem we do it sequentially: Given assets at the end of the period We do need to define a consistent notation for subperiods. I'm not wedded to the, e.g., |
Oh, I think I understand what's going better now based on your comment.
Let me try a different question.
I appreciate this argument. I see now that in this notation, you have put the subperiod structure inside an MDP-level timestep. If I am not mistaken, HARK in essence stretches out a cycle so for each what-you-call-"sub-period" is a whole discrete MDP-level time step. I think it would be best if the consistent subperiod notation we settle on was consistent with the HARK implementation. |
Right. The twist is that you are solving a different Bellman equation in the next step than in the current step. For example, in the portfolio problem, you are solving an optimal portfolio share problem given a lump of assets a. No consumption choice is involved. But once you've determined optimal portfolio choice for any given a, you are then ready to solve the optimal consumption problem (without any portfolio choice).
I think it will be more important that it is consistent with the Dolo/DolARK implementation. Of course, ideally, all three of them would be consistent, but Dol- is more important than HARK because kind of the point of HARK is that you have the freedom to solve the problem however you like, while the point of Dol- is to define the problem rigorously in hopes that eventually there will be general purpose solutions.
Total agreement on this. In fact, the goal should be not just that it is easy, but that it is automatic. That is, if you have properly specified the stochastic process that people perceive when they are solving the Bellman equation, that should be sufficient to generate the stochastic simulation of the model without you having to specify anything else (except maybe the number of agents to simulate). |
For @sbenthall to merge |
In conversation, we discussed how the equations in this PR are in "sequential" form. We also discussed how a problem may be reduced alternately to:
A simultaneous form model exists when the model is infinite horizon (whether or not there is a sub-period structure). A simultaneous form model can sometimes but not always be decomposed into a sequential form problem. |
Main edit is to add Bellman equations for cyclical model example