-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Separate core and dense state space. #310
Conversation
…urceEconomics/respy into one-step-ahead-simulation
Codecov Report
@@ Coverage Diff @@
## master #310 +/- ##
=======================================
Coverage 84.22% 84.22%
=======================================
Files 42 42
Lines 2732 2732
=======================================
Hits 2301 2301
Misses 431 431 Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice PR. All comments are minor and it would be ok to merge as is.
Closes #237.
Current behavior
Currently, a part of the state space is simply duplicated by all values of another dimension. The first part comprises experiences and similar variables which are mutually exclusive. We call this the core state space. Types and observables duplicate the core state space which is why we call it the dense state space.
The duplication causes a lot of problems because it unnecessarily requires a lot of memory.
Desired behavior
Remove the duplication to save memory and try to exploit the division for better parallelization.
Solution / Implementation
Core changes
_SingleDimStateSpace
is similar to the state space of KW94 and has no dense dimension._MultiDimStateSpace
comprises many of the former state spaces for each of the product of dense dimensions. The attributestate_space.sub_state_spaces
is a dictionary where the key are tuples of the values of dense state space dimensions. For a model with four types, the keys are[(0,), (1,), (2,), (3,)]
. The values of the keys are dictionaries which contain information on the specific covariates for this part of the state space. Because the dense dimensions are constant per sub state space, the covariates are also constant. The keys are the names and values are values.get_attribute
,set_attribute
and the accessor for data in one period.parallelize_across_dense_dimensions
which can be applied to functions whose calculations have no side-effects to other dense dimensions. For now, this is a simple for-loop, but it is easy to replace with joblib. The decorator recognizes, if arguments for the wrapped functions are dictionaries with dense state space dimensions as keys and automatically parallelizes over them.Additional changes
options
is irrelevant ascompute_covariates
iterates over the covariates until no additional covariate can be computed.rp.get_solve_func(params, options)
, then solve withstate_space = solve(params)
.