-
-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor getStates to use a transition function #761
Comments
`getPostStates` is also a candidate for this kind of conversion, as it's
also a purely mechanical step based on the model definitions:
`getStates` takes post-states (from t-1) and shocks to get today's states
`getPostStates` takes states and controls to get today's post-states
The methods `getShocks`, `getControls`, `simDeath` and `simBirth` are also
decent candidates, but are a bit more complicated because they either (a)
involve distribution objects or (b) have varying representations of the
policy functions.
…On Thu, Jul 16, 2020 at 1:33 PM Sebastian Benthall ***@***.***> wrote:
One of (perhaps the only?) representation of the transition function in
HARK models is in the getStates() method, which is called in the main
simulation loop.
For example:
https://github.com/econ-ark/HARK/blob/455d09e44306e3bc24edf028c0daad3f6968b364/HARK/ConsumptionSaving/ConsRepAgentModel.py#L221
It would simply be a matter of refactoring to extract the mathematics
within this method into a single function of the same type signature as
Dolo's transition function:
https://dolo.readthedocs.io/en/latest/model_specification.html#transitions
If done in this way, it would be easier to do the following:
- start with a Dolang/YAML model document
- parse the document into a Dolo model, which would include a
transition function object
- pass that transition function object into HARK
That would make it so the standard YAML representation of the model was
not merely superficially connected to the HARK object, but also in an
internal way.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#761>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADKRAFPJ7QZHI653U54AKDTR342XPANCNFSM4O4UMJXQ>
.
|
I believe 'getShocks' corresponds to what Dolo splits off into a separate collection, The shocks values are then an input in the transition function, in the Dolo spec. This relates to #760. Sorry I missed your note about bringing it up on the call. |
I don't think Dolo does death and birth. I believe this is a substantive difference between HARK and Dolo at this point. |
Indeed, in dolo, implicitely we live and let die.
Currently the problem you can represent in dolo is a unique stationary
process. There is currently nothing in it to represent a non-stationary
process or the evolution of a population. What you have in "transition" is
the detrended/conditional version of the model.
…On Thu, Jul 16, 2020, 7:43 PM Sebastian Benthall ***@***.***> wrote:
I don't think Dolo does death and birth. I believe this is a substantive
difference between HARK and Dolo at this point.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#761 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACDSKPB4NPAJTYKUAQT36DR3434TANCNFSM4O4UMJXQ>
.
|
I'm working on this issue. One thing I'm running into: it looks like:
but there's currently no way this difference is tracked structurally in the code. It is handled by the way individual variables are coded. Example: it looks like I'm working with the PerfectForesight model, which is supposed to be a quite basic case. But it looks like the aggregate state variable tracking here is breaking the MDP abstraction. I'm going to need to write some generalized support for this, I suppose. I wonder what thoughts others have, especially @mnwhite . |
Example: it looks like pLvlNow has a value for each agent, but PlvlAggNow is
a scalar? I guess because it is an aggregate value?
Exactly: Everybody experiences the same aggregate state. Having it be a
scalar prevents potential bugs were different people were living in
different aggregate states at the same time.
On Mon, Sep 14, 2020 at 12:08 PM Sebastian Benthall < ***@***.***> wrote:
I'm working on this issue.
One thing I'm running into: it looks like:
- *some* state variables are stored with a separate value for each
agent in a simulation,
- and *some* are meant to be scalars
but there's currently no way this difference is tracked structurally in
the code. It is handled by the way individual variables are coded.
Example: it looks like pLvlNow has a value for each agent, but PlvlAggNow
is a scalar? I guess because it is an aggregate value?
I'm working with the PerfectForesight model, which is supposed to be a
quite basic case. But it looks like the aggregate state variable tracking
here is breaking the MDP abstraction.
"breaking the MDP abstraction.'
Not sure what you mean by that. There's nothing about being a scalar that
prevents the aggregate state from being a state...
… I'm going to need to write some generalized support for this, I suppose. I
wonder what thoughts others have, especially @mnwhite
<https://github.com/mnwhite> .
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#761 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK75TON55ST3SIUGSPXTSFY5YHANCNFSM4O4UMJXQ>
.
--
- Chris Carroll
|
I was under the impression that an agent simulated within an AgentType was an isolated run of an MDP. The MDP formalism has no information exchanged between different runs. Now it sounds like you are saying that it is typical for the simulation of multiple agents within an AgentType to share information over the course of their simulated run. Technically, I believe that makes it a Multi-Agent Markov Decision process (MMDP) of some kind, because each agent is making decisions based on an individual state, and there's also aggregated state. I thought this extra level of complexity was introduced only in the Market class. I wonder how dolo deals with aggregated state like this. @albop ? |
I"m honestly not sure I follow the discussion. Here is the model in dolark:
I'm still not sure this casuistic distinctions contribute to the ongoing discussion. Can you precise the question a bit more? |
Now it sounds like you are saying that it is typical for the simulation
of multiple agents within an AgentType to share information over the course
of their simulated run. Technically, I believe that makes it a Multi-Agent
Markov Decision process (MMDP) of some kind, because each agent is making
decisions based on an individual state, and there's also aggregated state.
I thought this extra level of complexity was introduced only in the Market
class.
Well, yes, versions where you don't need to use the Market class are
versions where there is no meaningful equilibrium between the agents. But
for research purposes, almost all uses of the toolkit are likely to need to
be aggregate/macro models (which is to say, they will use the Market
class). Many of our DemARKs and most examples do not use the whole Market
class because they are designed to be focused explorations of specific
points, and if the point in question is not meaningfully affected by
general equilibrium (="Market") considerations, there's no point including
the extra complexity of building them in a GE framework.
But I'm guessing that your question is motivated somehow by some practical
concern about implementation, but I think neither Pablo nor I has a clear
sense of what that is.
"to share information over the course of their simulated run" ...
Well, kind of.
I'd say that the chief distinction between what economists typically do in
these kinds of simulations, and what "ABM" people do, is that economists
assume that people interact ONLY with the market, whose equilibrium is the
result of everybody's collective actions. No single individual has any
meaningful effect on any other individual. So, if you were worried about
needing to build some kind of direct-interaction framework - no. (Though
it is an ambition that eventually it would be nice to have simulation tools
that allow for this).
…On Mon, Sep 14, 2020 at 4:14 PM Sebastian Benthall ***@***.***> wrote:
"breaking the MDP abstraction.'
I was under the impression that an agent simulated within an AgentType was
an isolated run of an MDP. The MDP formalism has no information exchanged
between different runs.
Now it sounds like you are saying that it is typical for the simulation of
multiple agents within an AgentType to share information over the course of
their simulated run.
Technically, I believe that makes it a Multi-Agent Markov Decision process
(MMDP) of some kind, because each agent is making decisions based on an
individual state, and there's also aggregated state. I thought this extra
level of complexity was introduced only in the Market class.
I wonder how dolo deals with aggregated state like this. @albop
<https://github.com/albop> ?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#761 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK76M26JROAT44KG2PVDSFZ2QZANCNFSM4O4UMJXQ>
.
--
- Chris Carroll
|
Thank you. This is helpful. It is certainly helpful for me if I understand the underlying mathematics of the models. So it's useful to know that these are not-exactly-MDPs. I see what you mean about it being different if the aggregate variables are exogenous or endogenous. Maybe there's different formal model classes going on: My impression was that type (3) simulations in dolo required the dolark extensions to allow for the definition of a Is there an example of a dolo simulation that does a type (2) model? How do you specify the aggregated exogenous state in the YAML? |
Hmmm. The use of I can see why given that it's not stochastic, and because it was model that was written early on, it made sense to make the 'shortcut' of writing it in as a state variable of the agent. But thinking about the cleanest future architecture, it seems like an important point whether this is exogenous state that is shared across all agents in a simulation, or exogenous state that can vary per agent. I have a small hack around this, so it's not blocking me in the short run. |
The aggregate state is always endogenous for interesting models. But it can be time-invariant, like the constant interest rate of the Ayiagari model. This one is also projected as a constant parameter (to be detemined) into each agents' problem (cf: https://github.com/EconForge/dolark/blob/master/examples/ayiagari.yaml) |
There appear to be some models in HARK that are of type (2). It sounds like Dolo does not support this. I will leave it to the economists to decide whether or not they are interesting in their terms. |
You are not getting the distinction between "the aggregate state" -- which
is the configuration of ALL the aggregate state variables -- and "an
aggregate state variable" -- which is the value of a single state
variable. Like, ALL the aggregate states would include some measure of the
distribution of wealth -- which is endogenous to all the past shocks. It
would ALSO include what could be an exogenous state "boom" or "bust" which
is determined by (exogenous) solar radiation ("sunspots"). "The aggregate
state" encompasses the collection of all of them, exogenous and
endogenous. And any collection in which ANY variables are endogenous is an
endogenous collection (an "endogenous aggregate state").
…On Mon, Sep 14, 2020 at 5:32 PM Sebastian Benthall ***@***.***> wrote:
The aggregate state is always endogenous for interesting models.
There appear to be some models in HARK that are of type (2).
It sounds like Dolo does not support this.
I will leave it to the economists to decide whether or not they are
interesting in their terms.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#761 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK76JXRTMGJ7NSWFRLV3SF2DXXANCNFSM4O4UMJXQ>
.
--
- Chris Carroll
|
Ah, ok. Thank you for that clarification. As I understand it:
What surprised me about Hence, it was aggregation without a Market class involved. I am confused now whether aggregate variables always come from the market or if they may also come from, e.g., solar radiation. I also want to clarify that I am only including in this variables that have some downstream effect on the endogenous variables or controls. There are some cases where HARK computes and tracks a variable for monitoring or analyis purposes. I would call these "epiphenomenal variables" myself, due to some perhaps esoteric training in other fields. |
I think you meant to include "idiosyncratic" here?
This is what we tend to call a "partial equilibrium" macro model.
Again, I think your first phrase meant to be "there are endogenous and exogenous idiosyncratic variables"
e.g. aggregate wealth
e.g., whether the economy is in a "boom" or "bust" state (or, the value of some aggregate productivity shock)
It's not an edge case -- it's a very common setup. "Partial equilibrium" models are increasingly popular, because the profession has finally realized that
Right -- that's what "partial equilibrium" basically means -- you haven't imposed a Market equilibrium.
There's not any sense in which exogenous aggregate variables need to "come from" the Market class -- as illustrated by our various partial equilibrium models with aggregate shocks. If you are solving a GE model, it might be tidy to bundle together the treatment all of the aggregate variables, exogenous and endogenous, in the part of the code that deals with the Market mechanism, but there is no necessity for doing so with respect to exogenous aggregate variables.
Except possibly for debugging/diagnostic purposes, it's hard to see why "epiphenominal" variables would ever be interesting. Your definition of them basically is that these are variables that have no economic consequence. |
I think I see your point! But when it's distinguishing from aggregates, I see: yes, there should be an "idiosyncratic" there.
Ah, thank you. This is hugely clarifying.
And yet, they are sometimes being tracked in the HARK code. In this and the other active thread, I'm trying to get at terminology that can help clarify what's going on in the software. It may make it into documentation or more scholarly writeups. But maybe there are interesting cases for this. Maybe there's a model of an economy where, say, inequality measured by the Gini coefficient has to endogenous effect, but is nevertheless of policy of research interest. |
But maybe there are interesting cases for this. Maybe there's a model of
an economy where, say, inequality measured by the Gini coefficient has to
endogenous effect, but is nevertheless of policy of research interest.
Good example.
The distribution of wealth itself matters, but a single summary statistic
of it does not have any independent effect.
…On Wed, Sep 16, 2020 at 2:48 PM Sebastian Benthall ***@***.***> wrote:
I think you meant to include "idiosyncratic" here?
I think I see your point!
For a single MDP, "idiosyncratic" adds no new information.
But when it's distinguishing from aggregates, I see: yes, there should be
an "idiosyncratic" there.
It's not an edge case -- it's a very common setup. "Partial equilibrium"
models are increasingly popular
Ah, thank you. This is hugely clarifying.
Except possibly for debugging/diagnostic purposes, it's hard to see why
"epiphenominal" variables would ever be interesting.
And yet, they are sometimes being tracked in the HARK code.
Presumably for debugging or diagnostic purposes.
This is why I brought it up.
In this and the other active thread, I'm trying to get at terminology that
can help clarify what's going on in the software. It may make it into
documentation or more scholarly writeups.
But maybe there are interesting cases for this. Maybe there's a model of
an economy where, say, inequality measured by the Gini coefficient has to
endogenous effect, but is nevertheless of policy of research interest.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#761 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK7236WZOVW6L4QLJOGLSGEB7PANCNFSM4O4UMJXQ>
.
--
- Chris Carroll
|
namespace for states; transition function for #761
Fixed with #836 |
One of (perhaps the only?) representation of the transition function in HARK models is in the getStates() method, which is called in the main simulation loop.
For example:
HARK/HARK/ConsumptionSaving/ConsRepAgentModel.py
Line 221 in 455d09e
It would simply be a matter of refactoring to extract the mathematics within this method into a single function of the same type signature as Dolo's
transition
function: https://dolo.readthedocs.io/en/latest/model_specification.html#transitionsIf done in this way, it would be easier to do the following:
That would make it so the standard YAML representation of the model was not merely superficially connected to the HARK object, but also in an internal way.
The text was updated successfully, but these errors were encountered: