Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Framing and periodicity #798

Closed
sbenthall opened this issue Aug 13, 2020 · 52 comments
Closed

Framing and periodicity #798

sbenthall opened this issue Aug 13, 2020 · 52 comments
Assignees
@alanlujan91
Labels
Abstract Stages Design
Milestone
1.0.0

Comments

@sbenthall
Copy link
Contributor

The PerfForesightConsumerType has two poststate_vars: aNrmNow and pLvlNow
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1450

However pLvlNow is set in getStates():
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1665

...and not in getPostStates(). Rather, aLvlNow is updated as a poststate in the code:
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/ConsIndShockModel.py#L1706

Correcting the poststate_var field to list aLvlNow instead of pLvlNow raises an AttributeError.
I guess this is because the class is depending on the automatic initialization of the poststates, to create the attribute.
@sbenthall sbenthall added this to the 0.x.y milestone Aug 13, 2020
@mnwhite
Copy link
Contributor

mnwhite commented Aug 14, 2020 via email

@sbenthall
Copy link
Contributor Author

Hmmm.

For context, I bumped into this while working on #761.
I'm trying to functionalize out some of the behavior and that requires consistency as to how the functions used in simulations and variable designations are used.

If I understand correctly, you are saying:

  • pLvlNow is a state variable because it is a function of past state and stochastic shocks and is an input to the determination of a period's control values.
  • It is also an intertemporal transition, so is in that respect a 'poststate'

I find this confusing. If the variable is being used intertemporally, wouldn't its value in the previous period be pLvlPrev, or $p_{t-1}$, and the new value be pLvlNow, or p? It seems like some notational slippage was introduced into the code here.

I.e., why not:

  • in getStates, assign a value to state variable pLvlNow based on pLvlPrev and the stochastic shocks
  • in getPostStates, assign to poststate variable pLvlPrev that value of pLvlNow.

(I expect that once the state variables are all properly listed out, it should be possible to automatically store the previous value for all of them. But that would be a separate issue.)

Also, what about aLvlNow? Can you explain why that is being set in getPostStates(), but not listed as a poststate_var?

@mnwhite
Copy link
Contributor

mnwhite commented Aug 14, 2020 via email

@sbenthall
Copy link
Contributor Author

Ok. As I said, the main thing that's needed for refactoring is consistency.

That's interesting about the tracking issue.
I suppose my ideal implementation would look like this:

  • At the beginning of a period, move all the state variables into storage in a state_prev dictionary
  • Use those 't - 1' values as an input to the transition function, which would
    • evaluate each state and control variable in the order determined by their equations
  • track all the state and control variables in every run by default

In the short term, I'd like to go ahead with having the previous state stored in pLvlPrev consistently across the library, since I don't think I'm going to be able to convince you to drop the 'poststate' construct without a very compelling implementation in hand.

@llorracc
Copy link
Collaborator

This is related to the discussion Seb and I had with @albop last week.

I believe we agreed to move to a system in which there was a clearly defined point at which each exogenous shock was realized and at which each endogenous decision was made.

The transition equations would define the state variables. But we would need to have a name for every state variable that exists at any point -- including some that might not have names in the mathematical formulation of the problem.

For example, in the portfolio problem:

Subperiod State Shock Operation
0 b_{t} = a_{t-1} \RPort \Risky Transition
1 p_{t}=p_{t-1}\PermShk_{t} \PermShk_{t} Transition
2 y_{t} = p_{t}\TranShk_{t} TranShk Transition
3 m_{t} = b_{t} + y_{t} Transition
4 a_{t} = m_{t} - c_{t} c_{t} is chosen
5 \PortShare \PortShare chosen optimally

There are several virtues of disarticulating things in this way:

  1. There are no more "post-state" and "pre-state" variables. There are just the states at each subperiod.
  2. The state variables are all defined by the transition equations -- there's no need to have any separate articulation of what is a state and what is a control
  3. All the variables have a unique value within the period. So we don't need to have extra notation to decide whether the value in question is a "pre" or a "post" value
  4. In the case of models where we are really thinking of shocks as being realized simultaneously (like the risky return, the transitory shock, and the permanent shock, the order in which we lay out the draws of those shocks in sequence does not matter for the mathematical solution, but might matter sometimes for computational purposes.

@mnwhite
Copy link
Contributor

mnwhite commented Aug 17, 2020 via email

@sbenthall
Copy link
Contributor Author

Ok cool. I think this is something we are getting consensus around then.

I'm working towards making this happen.

I'm trying to start with some more basic refactoring: #760 #761 #660
These all have to do with make sure each model variable (exogenous shock, control variable, state variable) is getting accounted explicitly in each model an attached unambiguously to a bit of functionality for how it's getting assigned.

When everything has been standardized around that, it should be possible to write a "magic simulator" that uses them, along with a decision rule for the control variables.

Then "magic solvers" can be written that algorithmically find good decision rules.

@llorracc
Copy link
Collaborator

@mnwhite makes a good point about multivariate statistical draws, and that for computational efficiency we might want to have several LHS variables rather than one.

Let me expound a point that I forgot to make earlier, then elaborate on it in light of Matt's response.

The point was that we should refer to subperiods by the name of the product(s) returned therein, rather than by a numerical order (0, 1, etc). So the earlier figure should have been

Subperiod State Shock Operation
b b_{t} = a_{t-1} \RPort \Risky Transition
p p_{t}=p_{t-1}\PermShk_{t} \PermShk_{t} Transition
y y_{t} = p_{t}\TranShk_{t} TranShk Transition
m m_{t} = b_{t} + y_{t} Transition
a a_{t} = m_{t} - c_{t} c_{t} is chosen
portshare \PortShare \PortShare chosen optimally

If (counterfactually) the most efficient way to solve this problem were a simultaneous method that produced both c and portshare, the last line could be modified as below. And if it were necessary to draw the transitory and permanent shocks jointly (because they are mutually correlated, we could modify the representation as below

Subperiod State Shock Operation
b b_{t} = a_{t-1} \RPort \Risky Transition
PermShk_TranShk $\psi,\theta$ \PermShkTranShkMatrix Draw
p p_{t}=p_{t-1}\PermShk_{t} \PermShk_{t} Transition
y y_{t} = p_{t}\TranShk_{t} TranShk Transition
m m_{t} = b_{t} + y_{t} Transition
c_portshare \PortShare c and \PortShare chosen optimally
a a_{t} = m_{t} - c_{t} Transition

Then we would be able to write sentences like "In the period labeled c_portshare, both the level of consumption and the share of the portfolio invested in risky assets are jointly determined."

@mnwhite
Copy link
Contributor

mnwhite commented Aug 17, 2020 via email

@sbenthall
Copy link
Contributor Author

The equations/rules for assignments will specify a partial order, but not necessarily a total order, on the variables.

In my view, this means we should not consider them to be numbered.

It also means that the exogenous shocks can be both "simultaneous" and at differentiated. What's important is that they can be evaluated prior to state or control variables.

This was referenced Aug 18, 2020
Closed
Closed
@sbenthall
Copy link
Contributor Author

I might not get to the full refactor of post-post-state glory in the next PR.

For the transitional work, @mnwhite could you tell me if aLvlNow is a state or a post-state in AggShockConsumerType ?

@albop
Copy link
Collaborator

albop commented Sep 14, 2020 via email

@sbenthall
Copy link
Contributor Author

In my view, there is not a linear series of subperiods.

Rather, over a period at time t, there is a directed acyclic graph of variables.

In simulation, a decision rule induces the choice variables into a normal random variable with conditional dependencies on its inputs. State variables are like this as well. Exogenous shocks are the roots (no parents) of the DAG structure. Simulation proceeds down the DAG. In principle, the evaluation of two nodes at the same "depth" can be parallelized at the computational level. Nothing makes it sequential, necessarily.

When doing the solution, I expect you can work in reverse in a similar way. Start with the "deepest" nodes in the DAG, solve them, then go by backwards induction to "back" of the graph.

So I think I agree with what @albop is saying, in that what matters about a "period" is which variable's states are being sampled/solved. But calling them "subperiods" imposes a linearity on this process needlessly.

@sbenthall sbenthall changed the title In PerfForesightConsumerType, pLvlNow is not of poststate Functional periodicity Sep 15, 2020
@sbenthall
Copy link
Contributor Author

I've renamed this issue to make it clear that it is now about the new proposed architecture.
I will slate it for HARK 1.0 though that may be ambitious.

@sbenthall sbenthall modified the milestones: 0.x.y, 1.0.0 Sep 15, 2020
@albop
Copy link
Collaborator

albop commented Sep 15, 2020

Actually I find the term subperiod confusing. I prefer post-state which is less ambiguous. How to generalize it ? Post-post-state or post-k-state or just k-state ?

@sbenthall
Copy link
Contributor Author

If I understand correctly from @mnwhite , "post state" refers to any variables (state, control, shocks) that need to have values at the start of period t in order to resolve the downstream variables.

One major function of the "post state" designation has been to mark variables that need to be initialized with some value when an agent is "born". [This gets in HARK's multiple agent simulation paradigm, which includes mortality. I believe this paradigm is another extension beyond the MDP formalism].

Because these variables are always evaluated/initialized before the period t, I propse that we call them "antecedent variables" instead of "post states"--as they are neither always "post" nor always "states".

I think that @albop you may be using "post-state" slightly differently--I mean, with something different in mind--in your most recent comment.

Maybe it would be better to decouple the terminology around the model (which contains variables, which may be state, shock, or control variables) and the terminology around the solution/simulation algorithms (which may approach the problem of solving or simulation a model in sequential stages).

"period" I tend to use unambiguously to mean that which is denoted by the discrete time parameter t which has special meaning within the MDP formalism.

@albop
Copy link
Collaborator

albop commented Sep 16, 2020

@sbenthall : I've been trying to formalize a bit the representation I had in mind, in order to clarify the terminology (sorry it's a bit long)

  1. indexing of time: according to stochastic process convention (in which mdp theory is formulated), time t indexes the filtration of the process 𝒥ₜ, aka as the information set available at each time. With this convention time is defined as the flow of information. There can be states, choices as a function of these states, choices as a function of the first choices. All of them should be indexed by t if, knowing the solution, they can be deduced, from information available at date t . time and date are exact synonyms here.

    a. when the flow of information has a special structure, there can be special notations: for instance t+ denotes the information available at any date strictly greater than t. Or if there is a certain periodicity, one could find useful to use t+1/2 or t+1/3 or something like t_spring, t_summer...
    b. a period, is the interval between two dates t1 and t2 with t2>t1, something like [t1, t2[. allowing for empty periods t1=t2 is imho a bad idea, which will ultimately create a time-paradox or foster the the rise of populism. When it is not ambiguous, a period can be indexed by its starting date so we can refer to period t as a shortcut for [t,t+1[. But in principle the idea that a period lasts for one unit of time is purely conventional
    c. a subperiod, is a subinterval of a given period: [t,t+1[ is a subperiod of [t,t+4[ or [t,t+1/2[ is a subperiod of [t,t+1[
    d. in discrete time, one can define natural ("atomic") periods as intervals [t1, t2[ such that F_t = F_t1 for any t in [t1,t2[ and F_t1 ⊄F_t2 . With these notations, t+1 is "one unit of time later", but does not necessarily denote the "next" period, as in "the next time information will be released.

  2. notional periods
    a. within a given atomic period, [t1, t2[, one introduce new notional dates for instance sin ]t1,t2[ and use them use them to specify a specific succession of calculations. For instance if a[t1] and c[t1] are known, compute m[s] = a[t1] - m[t1] then a[t2] = m[s]*r
    b. this is not about notation: I like to denote notional periods explicitly like m[t$1] or with labels like m[t$consumption], but for additional flexibility I wouldn't have a problem with denoting by t$m a notional time which is defined by the minimal computations needed to compute m. This could be extended to something like t$(m,c) as the notional state, defined by m and c.
    c. in the basic example from a., there is no gain w.r.t. specifying that calculations must be done one after another (which a smart computer could determine on its own.. But metadata can be attached to s for instance a domain for all controls defined in s (like D(s) = m[s]\in[0,\infty[). Thanks to these metadata there is additional to perform, for instance a time-iteration algorithm.
    d. a good way to illustrate the difference between notional periods and basic indexes, is a maximization over two variables: c[t],psi[t] = argmax (some stuff). It is convenient to take the max over psi[t] then over c[t] rather than both at the same time. This can be done by defining: psi[s] = argmax( ... stuff at t+1 ) then c[t] = argmax(some stuff at s) . It is essentially a hint for the solver to solve for the complex max_{c,psi}, with some control over the intermediary function and domain which is defined at s . A clear difference with actual periods though is that you can swap c and psi without changing the model (solve for c than for psi).
    e. sometimes economists make an actual interpretation of what happens within a notional period. For instance, they say that capital is accumulated during [t,t+1[ and give a name to the end of period capital (the denote it k[t+1] but what it would be better to write it k[(t+1)-] because it is chosen based on the information known during [t,t+1[
    f. this is not the same as playing with the model timing. The "timing" is the description of how decisions are taken and information revealed, not how the economist solves it

In the above I've been using specific vocabulary which partially conflicts with the terms we've been using so far. In my mind the post-state is essentially what I have described as a notional state. And I gather "sub-period" has been used either as a synonym of a notional-date or as a slightly more general notion that I don't really grasp. I kind of feel strongly that whichever naming scheme we choose should at least be compatible with the ones in 1., that I think are quite universal. For this reason, I would avoid talking about sub-periods.

The spirit of my proposal was that after you define the exogenous process, you have defined the actual time structures (atomic periods) and that suitable notation shortcuts (namely filtering on exogenous values) where enough to define the "model".
Then notional dates, or sub-states, or ..., are used to describe a specific solution method.

In the table above with the list of operations, (transition, choice, random), I perceive a desire to unify everything in one description of what happens between t1 and t2, whatever t1 and t2 represent. I'm a bit skeptic about it for now, though I can't say for sure it won't work. It is enough for a RL algorithm after all. I suspect that operationally you would end up reconstructing a structure set similar to what I described. In this case, the definition of time is a result of the model (for instance, some transitions do t->t+1 some others don't).

@sbenthall
Copy link
Contributor Author

You have introduced both mathematical concepts and notation that I have been unfamiliar with.
Please forgive me if I am slow to understand and so push back in ways that may seem obvious to you.

You bring up "filtration". I'm afraid I'm unfamiliar with this idea. Looking it up, and looking up stochastic processes, I believe I see t as the index into the index set, which are the natural numbers in most cases we consider here.

When discussing an MDP, we typically formalize it in terms of a set S of states, A of actions, R of rewards, and P, the transition probabilities of states and actions. The decision rule pi specifies a probability distribution over actions given states. This induces the MDP into a markov process over the state space S. All of this is over the index set of the natural numbers, the index is denoted with t.

This is the view from "1,000 feet (or meters) up". As is quite typical with MDPs, the state space S is the Cartesian product of many substates; the action space A is a Cartesian product over many sub-actions. The structural relationships between the substates and subactions are determined by the joint probability distribution P. Note that in this formalism, the entirety of S_{t-1} is the "post state" with respect to state S_t.

The terminology gets muddled at this point because, I must presume, Economics training is focused at a smaller resolution than this. Let us, consistently with our earlier discussion, call these 'sub-states' and 'sub-actions' "variables"; we have been calling the latter "control variables". Because the joint probability distribution P involves many independent relations between the variables, it is possible to represent their structure graphically as a Bayesian network. When this is done, it makes it clear which variables are endogenous and which are exogenous: exogenous variables are those with no control variables as ancestors.

If you are following me to this point, then I think I can precisely state my disagreement with your formulation above. If t cannot be a filtration over the structured distribution over the sub-state variables because a filtration must be totally ordered, whereas the sub-state variables are only guaranteed to be partially ordered. So the premise of your formulation is incorrect.

@albop
Copy link
Collaborator

albop commented Sep 16, 2020

which premise ?

@sbenthall
Copy link
Contributor Author

  1. indexing of time: according to stochastic process convention (in which mdp theory is formulated), time t indexes the filtration of the process 𝒥ₜ, aka as the information set available at each time. With this convention time is defined as the flow of information. There can be states, choices as a function of these states, choices as a function of the first choices. All of them should be indexed by t if, knowing the solution, they can be deduced, from information available at date t . time and date are exact synonyms here.

I'm saying time t indexes a filtration on the states S and actions A.
But that the variables (sub-states, sub-actions) are not totally ordered, and so do not support a filtration.

One could attempt to impose a filtration on the sub-state/sub-action process, but that would be cumbersome and I violate the use of t in the MDP formalism.

To put it another way: it is better to think of t as a full day (September 16th), rather than a particular point in time (September 16, 12:00 AM). Many things can happen during the day (the sub-actions, the sub-state variables). But, at the lower level of resolution, I can take two actions on September 16th with different information available at each time.

Your use of "information" here confuses me a bit, to be honest. At the high level, yes, given a decision rule (S_t, A_t) is determined based on the information from (S_{t-1}, A_{t-1}). But this is not the case as the sub-state level. For example, there is no mathematical reason that an exogenous shock, which would contribute information, needs to be resolved "before" a control variable that does not depend on it.

@sbenthall
Copy link
Contributor Author

So, rather than a period being understood as a span between t and t+1, a period would be, simply, t.

Then, the question is what to name and whether and how to order what is happening within t.

@albop
Copy link
Collaborator

albop commented Sep 16, 2020

I really don't see the incompatibility. When you define an MDP, using SARP, you get a discrete random process $X_t$ indexed by t over the state-space S, where the information set at date date t is the cumulative history of all the past events (the filtration is just the set of all possible information sets). This random process has the markov property, that is the distribution of $X_{t+1}$ is characterized by looking at $X_t$. All of this is perfectly consistent and standard. t is time and it is totally ordered. I think whatever we say should be consistent with that.

One issue it that time defined by each markov chain update (each epoch) does not directly match physical time so one might want to replace $t$ by another index that is still totally ordered: $0_spring, 0_summer, 1_spring, ....$. That one is easy.

The other problem is that some of our model reformulations are implicitly changing the definition of markov chain. Take the updating equation $m' = (m - c) r$ and assume it represents the true model. I could reformulate it with a additional transition: $a' = m-c$ then $m' = a r$. I can see the result as a markov chain with additional state a, where I need to update the MC twice to arrive to a new value of m starting from an initial m. And yet, I would still like to go from t to t+1 to match the original formulation we need to break the equivalence between a markov chain update and a unit time increment. This is why we need to add another kind of date, and find another name for the "intermediary" state. The set of possible intermediary states is not totally ordered, I agree with that. Suppose there are two choice variables $x_1$ and $x_2$. And I start defining other variables as c$x_1 and b$x_2 which are determined respectively by x_1 and x_2. Which one occurs first ? We don't know and I don't think it's necessarily a bad feature. But that is the very reason while I dislike the term "subperiod" in this context.

Incidentely, I've started to use here the term "epoch" to denote the update, of the markov chain from the flow of time. It is not 100% clarifying though as it is not immune to the kind of reformulation I just mentioned.

@albop
Copy link
Collaborator

albop commented Sep 16, 2020

  1. indexing of time: according to stochastic process convention (in which mdp theory is formulated), time t indexes the filtration of the process 𝒥ₜ, aka as the information set available at each time. With this convention time is defined as the flow of information. There can be states, choices as a function of these states, choices as a function of the first choices. All of them should be indexed by t if, knowing the solution, they can be deduced, from information available at date t . time and date are exact synonyms here.

I'm saying time t indexes a filtration on the states S and actions A.
But that the variables (sub-states, sub-actions) are not totally ordered, and so do not support a filtration.

One could attempt to impose a filtration on the sub-state/sub-action process, but that would be cumbersome and I violate the use of t in the MDP formalism.

To put it another way: it is better to think of t as a full day (September 16th), rather than a particular point in time (September 16, 12:00 AM). Many things can happen during the day (the sub-actions, the sub-state variables). But, at the lower level of resolution, I can take two actions on September 16th with different information available at each time.

Your use of "information" here confuses me a bit, to be honest. At the high level, yes, given a decision rule (S_t, A_t) is determined based on the information from (S_{t-1}, A_{t-1}). But this is not the case as the sub-state level. For example, there is no mathematical reason that an exogenous shock, which would contribute information, needs to be resolved "before" a control variable that does not depend on it.

Yes, I see there is some misunderstanding about the definition of the information set. In your case, assume the transition of your markov chains rely on some i.i.d. distribution $e_t$, then the information set omega at date $t$ is all the past realizations of this distribution: $(e_0, e_1, .... e_t)$.

I have also referred in the past to the concept of indexing variables by the state of the (exogenous) system but it is not relevant to the discussion here.

@sbenthall
Copy link
Contributor Author

sbenthall commented Sep 18, 2020 •
edited
Loading

  1. d. in discrete time, one can define natural ("atomic") periods as intervals [t1, t2[ such that F_t = F_t1 for any t in [t1,t2[ and F_t1 ⊄F_t2 . With these notations, t+1 is "one unit of time later", but does not necessarily denote the "next" period, as in "the next time information will be released.

What is "F" in this notation?

There is an alternative formulation of the Markov property where the
future distribution of of a process can be forecast using (X_t, X_{t-1}, ... X_{t-k}) with some finite k.

True.

seasonality for instance. As a modeller you might want to refer to t+1 as
the next quarter, while t+4 would be the next year.

I see. Yes, seasonality. This case has been quite tricky.

My impression of this discussion so far is that there are many equivalent ways to model the seasonality problem, and that the sticking point is computational tractability.

My own preference would be, in terms of setting up the model:

  • reserve t (time, time step) as the natural numbers, scoped such that the k=1 Markov property holds
  • a different variable, such as c, be used to represent the number of time steps within a cyclical superperiod (i.e, c = 4 for four seasons, where one time step (t) is a season.)
  • a different variable, such as s, advances deterministically as t mod c.

But then have the solver and simulator know how to handle these cases intelligently and not waste time on, say, computing what happens if s_(t+1) = s_(t) + 2 because that never happens.

I.e., separate the model definition and computational levels of abstraction cleverly.

This is a quite useless comment.

I will wait at least a decade before trying to debate this point with you further.
But at that point, interplanetary macroeconomics may be viable and I would like to revisit the topic.

Problem is not whether frames are clearly and uniquely defined by
transitions but which set of frames are useful.

Ah. This gets to the heart of the matter.

Is there a way to formulate what "usefulness" is?

If so, maybe there's a way to programmatically discover the best framings of a particular model.

@sbenthall
Copy link
Contributor Author

And I wanted to have an example in mind where this is not the case. I can see why it would be the case for some limited information problem though. In the basic consumption-savings model it is not the case. In fact there another condition is met that a frame does not loose information w.r.t. its parent frame. Do you have one easy characterization of one if the models you are working on ?

Trying to get at this more directly... suppose you have a problem with several control variables:

  • consumption, informed by current market assets
  • endogenous labor quantity, informed by an exogenous physical energy level
  • risky asset allocation share, informed by an exogenous and intermittent selective attention to stock prices.

I wonder if it matters whether:

  • (a) these variables are determined in sequential order, each made with knowledge of the preceding choices, or
  • (b) if the variables are determined only in light of the state at t-1.

Maybe the consumer can only investigate their asset allocations on days that they don't labor.

@albop
Copy link
Collaborator

albop commented Sep 18, 2020 via email

@albop
Copy link
Collaborator

albop commented Sep 18, 2020 via email

@sbenthall
Copy link
Contributor Author

It is getting to the point where it will be possible to implement a general framework for this. I expect it will be viable to work on it soon after #836 is pulled.

I'm wondering, from a software design standpoint, how we would like to support this in HARK.

My preference would be for this architecture to ultimately be in HARK.core.

My understanding is that the status quo is to support the overwriting of the core functionality for custom models in cases like this. See the work of @Mv77 on the new multi-stage portfolio model, this comment from @mnwhite, and the work in PR #832

Recall that the goal for this work is twofold:

  • to support the frame-based organization of a model
  • to allow the user to write a model configuration that is not "hard coded" as a model class.

I'm somewhat ambivalent between trying to modify the core first and then refactoring all the existing models to fit, vs. the perhaps more conservative strategy of making minimal changes to the core while subclassing AgentType to create a class with the desired API. This latter approach was what I was aiming for when working on #696.

Speaking of #696, I wonder how its experimental approach to dictionary-based model configuration looks in light of the more recent discussions in this thread. While in that experimental PR each state has its own individual transition function--because the dictionary of transition functions uses single variable names as keys--it would also be possible for it to use tuples of variables names as keys, allowing for transition functions that determine multiple variables 'at once'.

This was referenced Sep 30, 2020
Merged
Open
@sbenthall sbenthall mentioned this issue Oct 22, 2020
Closed
This was referenced Nov 6, 2020
Open
Merged
@sbenthall
Copy link
Contributor Author

I'm now trying to figure out a generic data structure for the frame architecture (this is informed by work on #862). @Mv77 wondering if you could weigh in on this especially.

I think we have consensus about what a frame is. The question is how to design the implementation so that it elegantly does all the things we want it to do.

This is a list of desiderata for what a Frame implementation might have:
(1) A Frame should know what model variables are updated when it is simulated. Name suggestion: target
(2) A Frame should know what model variables it depends on when it is simulated.
(3) A Frame should know the transition function. This is used in simulations.
(4) A Frame should ideally know how to differentiate its own transition function. This is used in solutions. (I could be wrong here ... could somebody please confirm? @llorracc @mnwhite )

I have been playing with implementations in terms of the basic data structures that come with Python. But now I'm wondering if we should have a dedicated Frame class that can capture all this information.

The other question that's coming up for me is that when listing the variables that a transition function depends on, it needs to distinguish between current time values and previous time values for any referenced model variables.

This can be done structurally (i.e., separate lists of variable names for 'now' and previous values) or notationally (i.e., adding 'prev' to indicate a previous value). I can see pros or cons to either approach.

I have a prototype implementation ready for your review at PR #865 but it sidesteps these issues, which I do think will matter moving forward.

@sbenthall sbenthall mentioned this issue Nov 9, 2020
Closed
3 tasks
@Mv77
Copy link
Contributor

Mv77 commented Nov 9, 2020

On the data-structure question, my own uninformed prior is that the most future-proof way to do things would be for frames to have their own class. They seem like a complex enough object that at some point in the future you might want to add properties or attributes to it that might not be accommodated by primitive structures without substantial changes. It might also make it more user friendly in the sense of, e.g., knowing that the 'target' is at frame.target instead of having to remember/look up that it is on the third entry of some tuple named frame.

@sbenthall
Copy link
Contributor Author

Thanks @Mv77 that does sound right.

The implementations in #865 and #866 don't yet use a dedicated class, but I will move them in that direction.

I think we can get the best of both worlds with a sensible constructor. I.e. initialize the Frame with something like

Frame(target, scope, transition)

@Mv77 Mv77 mentioned this issue Nov 16, 2020
Open
@sbenthall
Copy link
Contributor Author

Just a broader example of what the Frame initialization fragment may look like

Frame(
    target=('pLvl', 'PlvlAgg', 'bNrm', 'mNrm'),
    scope=('aNrm','Risky',"Adjust'),
    transition=transitionEquation,
    default=(0,1),
    # anything else that's needed for the solver? such as...
    arbitrage=arbitrageEquation
)

@llorracc
Copy link
Collaborator

One point to make here is that "arbitrage equation" is a legacy from the Representative Agent world in which one of the chief cheats is that all optimization solutions are interior solutions for which a first order condition holds. That often will not be true. (Like, in our dcegm tool). So I think I'd advocate replacing "arbitrage" with "optimality" in the above, which allows for the possibility of dcegm-like tools.

@sbenthall
Copy link
Contributor Author

Roger that.
What does 'optimality' refer to in general? A function to be optimized?

@llorracc
Copy link
Collaborator

llorracc commented Nov 18, 2020 via email

@sbenthall sbenthall modified the milestones: 1.0.0, 1.x.y Dec 10, 2020
@sbenthall
Copy link
Contributor Author

I have another question about the design of frames.

Control variables may be subject to bounds that functions of other state variables.

https://dolo.readthedocs.io/en/latest/model_specification.html#boundaries

In @llorracc 's examples in this thread, which I'm working from, it's not clear where the bounds are specified.

My intuition is:

  • A frame with a control variable may also have bounds for that variable
  • Any state variables that are inputs to the functions computing the bounds of the control variable should be included in the 'scope' of the frame

What I'm unclear about is how a frame-specific constraint would impact the complexity of a frame's solution problem, and whether these constraints are any impediment to the composability of frame solutions in backwards induction.

@sbenthall sbenthall modified the milestones: 1.x.y, 1.1.0 Jan 23, 2021
@sbenthall sbenthall mentioned this issue Apr 1, 2021
Closed
@alanlujan91 alanlujan91 self-assigned this Aug 19, 2022
@econ-ark econ-ark locked and limited conversation to collaborators Jun 3, 2024
@sbenthall sbenthall converted this issue into discussion #1440 Jun 3, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Assignees

@alanlujan91 alanlujan91

Labels
Abstract Stages Design
Projects
None yet
Milestone
1.0.0
Development

No branches or pull requests
6 participants
@sbenthall @albop @llorracc @alanlujan91 @mnwhite @Mv77