Labeled models #1177

alanlujan91 · 2022-10-05T19:31:00Z

Please ensure your pull request adheres to the following guidelines:

Tests for new functionality/models or Tests to reproduce the bug-fix in code.
Updated documentation of features that add new functionality.
Update CHANGELOG.md with major/minor changes.

sbenthall · 2022-10-11T17:06:37Z

It looks like this PR includes a lot of changes that work in your new syntax for utility functions.

Can these changes be merged without the additional changes that are about labelled models?

It would be easier to review the material in this PR if it was broken up into multiple PRs that focus an particular functions.

sbenthall · 2022-10-11T17:09:38Z

examples/LabeledModels/LabeledModels.py

+    dims=("mNrm"),
+    attrs={"long_name": "Normalized Market Resources"},
+)
+state = Dataset({"mNrm": mNrm})  # only one state var in this model


I see you are using a DataArray for an individual variable (?) and a DataSet for an object that bundles all states, actions, etc. (?)

I'm curious why you made this choice. My understanding is that a DataSet is analogous to a tabular database, containing many DataArrays each much like an individual table. The main purpose of a DataSet is to allow joins across its DataArrays. But there will be no just joins across an Agent's states.

Did I get this right? What am I missing?

It seemed appropriate to me that the state variable is a collection of DataArray's each with it's own attributes, names, etc, which would correspond to a Dataset. In this way, the state variable can be generalized to be an n-dimensional Dataset, each dimension being a DataArray.

sbenthall · 2022-10-11T17:12:31Z

examples/LabeledModels/LabeledModels.py

+    def __init__(self, dataset: xr.Dataset, CRRA: float):
+
+        self.dataset = dataset
+        self.CRRA = CRRA


It seems like what you are doing here is making a callable wrapper around an xr.DataSet.

I wonder if this can be implemented in a more general way.

There might be a more general way. This is to use the trick of using the inverse value function to approximate the value function.

sbenthall · 2022-10-11T17:13:56Z

examples/LabeledModels/LabeledModels.py

+#
+
+# %%
+def state_transition(s=None, a=None, params=None):


Ah, yes. I've found it useful to write transition functions in similar way.

In my code, I've been using x for states and k for discretely realized values of shocks.

Here, s=state, a=action(s). Elsewhere I might have used shocks explicitly, so I should probably be more consistent with this nomenclature.

sbenthall · 2022-10-11T17:17:34Z

HARK/distribution.py

+        attrs=None,
+        var_names=None,
+        var_attrs=None,
+    ):


Docstring for this please

sbenthall · 2022-10-11T17:17:51Z

HARK/distribution.py

+        return ldd
+
+    @classmethod
+    def from_dataset(cls, x_obj, pmf):


Docstring for this please

HARK/ConsumptionSaving/ConsLabeledModel.py

sbenthall · 2022-10-11T17:24:28Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+
+        # egm requires swap dimensions; make actions and state functions of state
+        action = xr.Dataset(acted).swap_dims({"aNrm": "mNrm"})
+        state = xr.Dataset(state).swap_dims({"aNrm": "mNrm"})


You seem to be imputing some semantics on the other order of the dimensions in these DataSets.

What is that semantics?

This was a suggestion by @llorracc. An action is a consequence of the state, but given the end of period state, the endogenous grid method tells you how the agent must have acted in order to be optimal. This should be made more clear in documentation.

This is cleverly implemented!

sbenthall · 2022-10-11T17:27:33Z

Documentation/CHANGELOG.md

@@ -28,6 +28,7 @@ Release Date: TBD
 * Adds keyword argument `labels` to `expected()` when using `DiscreteDistributionXRA` to allow for expressive functions that use labeled xarrays. [#1156](https://github.com/econ-ark/HARK/pull/1156)
 * Adds a wrapper for [`interpolation.py`](https://github.com/EconForge/interpolation.py) for fast multilinear interpolation. [#1151](https://github.com/econ-ark/HARK/pull/1151)
 * Adds support for the calculation of dreivatives in the `interpolation.py` wrappers. [#1157](https://github.com/econ-ark/HARK/pull/1157)
+* Creates `UtilityFuncCRRA` which is an object oriented utility function with a coefficient of constant relative risk aversion and includes derivatives and inverses. [#1168](https://github.com/econ-ark/HARK/pull/1168)


We should merge in this PR #1168 first and then revisit this material around Labeled Models.

alanlujan91 · 2022-10-11T17:36:14Z

It looks like this PR includes a lot of changes that work in your new syntax for utility functions.

Can these changes be merged without the additional changes that are about labelled models?

It would be easier to review the material in this PR if it was broken up into multiple PRs that focus an particular functions.

Yes, there are changes on this branch that are part of other PRs, but I should probably remove those.

The most important files for this PR are HARK/ConsumptionSaving/ConsLabeledModel.py and the corresponding notebook examples/LabeledModels/LabeledModels.ipynb.

alanlujan91 · 2022-10-11T19:51:13Z

I have removed the changes in #1168 but this still depends on the changes in #1170

codecov-commenter · 2022-10-12T00:29:58Z

Codecov Report

Base: 73.32% // Head: 73.26% // Decreases project coverage by -0.07% ⚠️

Coverage data is based on head (cddfcf7) compared to base (69e0aa3).
Patch coverage: 68.20% of modified lines in pull request are covered.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1177      +/-   ##
==========================================
- Coverage   73.32%   73.26%   -0.07%     
==========================================
  Files          74       76       +2     
  Lines       12122    12529     +407     
==========================================
+ Hits         8889     9179     +290     
- Misses       3233     3350     +117

Impacted Files	Coverage Δ
HARK/rewards.py	`52.26% <39.47%> (-2.19%)`	⬇️
HARK/ConsumptionSaving/ConsLabeledModel.py	`68.36% <68.36%> (ø)`
...K/ConsumptionSaving/tests/test_ConsLabeledModel.py	`91.17% <91.17%> (ø)`
HARK/numba_tools.py	`18.97% <100.00%> (ø)`
HARK/distribution.py	`83.06% <0.00%> (+2.19%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

add estimagic

HARK/ConsumptionSaving/ConsLabeledModel.py

sbenthall · 2022-10-19T15:30:39Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+                state_dict[coords] = state[coords]
+
+        result = CRRAutility(
+            self.dataset["v_inv"].interp(


I am confused about what mathematical object "v_inv" refers to.

It does not appear to be the mathematical inverse of the value function, which would be my first guess.

In the last period, $v(m) = u(c(m)) = u(m)$, so we define the "inverse" as $v^{-1}(m) = u^{-1}(v(m)) \equiv m$.

In this case $v$ is curved but $v^{-1}$ is linear, so if we want to approximate $v$ off the grid we get better results linearly approximating $v^{-1}$ then recurving, than linearly interpolating $v$, especially at low values of $m$.

We can do the same at earlier periods, where $v^{-1}_t(m) = u^{-1}(v_t(m))$. In For $t \ne T$, $v^{-1}_t$ is not linear, but it's much less curved than $v_t$ (does not go to neg infty) so a linear approximation approach is still better.

sbenthall · 2022-10-19T15:32:47Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+        self,
+        value: ValueFuncCRRALabeled,
+        policy: xr.Dataset,
+        continuation: ValueFuncCRRALabeled,


To the extent that value, policy, and continuation all are wrappers around an underlying Dataset with shared coordinates, maybe this can be designed more flexibly.

sbenthall · 2022-10-19T15:50:56Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+
+        dataset = xr.Dataset(
+            {
+                "cNrm": cNrm,


I think it's an interesting choice that this is being labeled with cNrm, which names a variable, when it refers to a policy or decision rule that chooses the value for this variable based on preceding state variables.

I believe in HARK this was called something like cFunc in order to denote that it's a decision rule.

I wonder if there's a good reason to pick the more concise notation in this version of the model

I assumed that was an artifact of differentiating between an array and a callable (function). Since xarray does both, I didn't think i needed to differentiate

sbenthall · 2022-10-19T15:54:02Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+            value=vfunc,
+            policy=dataset[["cNrm"]],
+            continuation=None,
+            attrs={"m_nrm_min": 0.0},


whereas value, policy, and continuation are all objects at the level of abstraction of the Bellman problem, attrs is as the level of abstraction of the underlying xarray-powered data structure.

That's somewhat mismatched, and I wonder about the meaning of the `"m_nrm_min" field here. Is this representing a constraint on the policy, or a description of the state grid?

(Also, we should probably use either cNrm or c_nrm but not switch between them?)

m_nrm_min is the minimum allowable market resources, or natural borrowing constraint. I think this can be considered a Bellman property, but not sure how to design it since it's just a lower bound.

sbenthall · 2022-10-19T15:58:19Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+        return ns
+
+    def reverse_transition(self, ps=None, a=None, params=None):
+        """state from post_state and actions"""


'reverse' or 'inverse' ?

I think reverse is more appropriate. State transition is forward (state to post-state, given action), reverse transition is backward (post-state to state, given action).

sbenthall · 2022-10-19T16:06:00Z

HARK/ConsumptionSaving/ConsLabeledModel.py

+        variables.update(ps)
+
+        variables["reward"] = self.u(action["cNrm"])
+        variables["v"] = variables["reward"] + params.Discount * continuation(ps)


It may be my own ignorance, but the question of whether the application of params.Discount should be here or in the continuation (and whether other discounting bits like the PermGroFac term should be here or in the continuation) is confusing to me.

For this piece, I wanted to consider Discount as a coefficient (float) parameter. Stochastic discounts go into the calculation of the continuation function.

implement UtilityFuncCRRA

alanlujan91 · 2023-01-30T00:35:06Z

@llorracc Further changes can be made to these models, but I think they are in good enough shape for release 0.13, pending passing checks

sbenthall · 2023-01-31T17:52:17Z

Hi @alanlujan91

I know you are stoked to get this feature in before the 0.13 release.
But for a substantial addition to the code base like this, I think we really do need to insist on docs and tests, which are still missing.

We use the numpy standard for docstrings so API documentation is autogenerated into ReadTheDocs:
https://numpydoc.readthedocs.io/en/latest/format.htmls.

For the tests, they can probably be copied and edited from the existing consindshock tests, and used to verify that the new implementation is bug free.

Given that the 0.13 release is already way overdue, I would recommend that we let you give this PR the attention it deserves and hold off on merging it until later. I would like to get on a semiannual release cycle for HARK, which would mean that we would have another release this coming summer.

llorracc · 2023-01-31T18:55:10Z

@sbenthall, while I am in agreement with your desire to have a regular release schedule and twice a year sounds right, I'm really keen to get this into the 0.13 release -- so much so that I'd propose that we wait until this is ready before making the 0.13 release.

sbenthall · 2023-01-31T19:13:29Z

Very well. We can hold off on the 0.13 release. But we need tests and docs for this work.

llorracc · 2023-01-31T19:24:17Z

I agree, we need to wait for tests and docs.

minimal testing for labeled models

sbenthall · 2023-02-07T10:07:13Z

Merged. Hooray! Great work @alanlujan91

alanlujan91 added 3 commits September 7, 2022 11:09

Merge branch 'DiscDistLabeled' into labeled_models

ee79828

Create ConsLabeledModel.py

8f0ee53

Create example notebook

cd401aa

sbenthall reviewed Oct 11, 2022

View reviewed changes

HARK/distribution.py Outdated

attrs=None,

var_names=None,

var_attrs=None,

):

Copy link

Contributor

sbenthall Oct 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstring for this please

sbenthall reviewed Oct 11, 2022

View reviewed changes

HARK/distribution.py Outdated

return ldd

@classmethod

def from_dataset(cls, x_obj, pmf):

Copy link

Contributor

sbenthall Oct 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docstring for this please

sbenthall reviewed Oct 11, 2022

View reviewed changes

HARK/ConsumptionSaving/ConsLabeledModel.py Show resolved Hide resolved

sbenthall reviewed Oct 11, 2022

View reviewed changes

sbenthall mentioned this pull request Oct 11, 2022

UtilityFuncCRRA #1168

Merged

3 tasks

alanlujan91 added 2 commits October 11, 2022 15:42

remove changes to utilities

ea7565b

Merge remote-tracking branch 'upstream/master' into labeled_models

e10bc04

alanlujan91 mentioned this pull request Oct 12, 2022

Changes to DiscreteDistributionXRA #1170

Merged

3 tasks

Merge remote-tracking branch 'upstream/master' into labeled_models

83a5a5a

alanlujan91 added 2 commits October 12, 2022 10:04

update labeled_models after master merge

a9f294c

Update setup.cfg

428ae83

add estimagic

sbenthall reviewed Oct 19, 2022

View reviewed changes

HARK/ConsumptionSaving/ConsLabeledModel.py Outdated Show resolved Hide resolved

sbenthall reviewed Oct 19, 2022

View reviewed changes

Merge remote-tracking branch 'upstream/master' into labeled_models

fd9813d

alanlujan91 added 4 commits January 19, 2023 08:14

fix bug in numba tools

f255261

Merge remote-tracking branch 'upstream/master' into labeled_models

5e3614d

Merge remote-tracking branch 'upstream/master' into labeled_models

eb0e326

final update

23b4da6

implement UtilityFuncCRRA

alanlujan91 changed the title ~~[WIP] Labeled models~~ Labeled models Jan 30, 2023

alanlujan91 added the Ready-To-Merge Has been reviewed and when branch is updated and checks pass it should be merged label Jan 30, 2023

alanlujan91 requested a review from llorracc January 30, 2023 00:30

Update CHANGELOG.md

1de84a3

sbenthall added Needs: Testing Needs: Documentation and removed Ready-To-Merge Has been reviewed and when branch is updated and checks pass it should be merged labels Jan 31, 2023

alanlujan91 added 3 commits February 5, 2023 16:26

update documentation

5fbd8d5

update notebook

10a947e

Create test_ConsLabeledModel.py

cddfcf7

minimal testing for labeled models

alanlujan91 added Status: Review Needed Ready-To-Merge Has been reviewed and when branch is updated and checks pass it should be merged and removed Needs: Testing Needs: Documentation labels Feb 6, 2023

alanlujan91 assigned sbenthall and unassigned alanlujan91 Feb 6, 2023

alanlujan91 added 2 commits February 6, 2023 12:03

Merge remote-tracking branch 'upstream/master' into labeled_models

0c512c0

Merge remote-tracking branch 'upstream/master' into labeled_models

f97125a

alanlujan91 mentioned this pull request Feb 7, 2023

Pre-commit pre-0.13 #1225

Merged

sbenthall merged commit 78d8e18 into econ-ark:master Feb 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Labeled models #1177

Labeled models #1177

alanlujan91 commented Oct 5, 2022 •

edited

Loading

sbenthall commented Oct 11, 2022

sbenthall Oct 11, 2022

alanlujan91 Oct 11, 2022

sbenthall Oct 11, 2022

alanlujan91 Oct 11, 2022

sbenthall Oct 11, 2022

alanlujan91 Oct 11, 2022

sbenthall Oct 11, 2022

sbenthall Oct 11, 2022

sbenthall Oct 11, 2022

alanlujan91 Oct 11, 2022

sbenthall Oct 19, 2022

sbenthall Oct 11, 2022

alanlujan91 commented Oct 11, 2022

alanlujan91 commented Oct 11, 2022

codecov-commenter commented Oct 12, 2022 •

edited

Loading

sbenthall Oct 19, 2022

alanlujan91 Nov 16, 2022

sbenthall Oct 19, 2022

sbenthall Oct 19, 2022

alanlujan91 Nov 16, 2022

sbenthall Oct 19, 2022

alanlujan91 Nov 16, 2022

sbenthall Oct 19, 2022

alanlujan91 Nov 16, 2022

sbenthall Oct 19, 2022

alanlujan91 Nov 16, 2022

alanlujan91 commented Jan 30, 2023

sbenthall commented Jan 31, 2023

llorracc commented Jan 31, 2023

sbenthall commented Jan 31, 2023

llorracc commented Jan 31, 2023

sbenthall commented Feb 7, 2023

Labeled models #1177

Labeled models #1177

Conversation

alanlujan91 commented Oct 5, 2022 • edited Loading

sbenthall commented Oct 11, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alanlujan91 commented Oct 11, 2022

alanlujan91 commented Oct 11, 2022

codecov-commenter commented Oct 12, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alanlujan91 commented Jan 30, 2023

sbenthall commented Jan 31, 2023

llorracc commented Jan 31, 2023

sbenthall commented Jan 31, 2023

llorracc commented Jan 31, 2023

sbenthall commented Feb 7, 2023

alanlujan91 commented Oct 5, 2022 •

edited

Loading

codecov-commenter commented Oct 12, 2022 •

edited

Loading