Fix shocks in simulated data (#414)

.conda/meta.yaml

@@ -18,9 +18,8 @@ requirements:
     - python >=3.6
   run:
     - python >=3.6
-    - chaospy
     - click
-    - estimagic >=0.0.30
+    - estimagic >=0.1.2
     - hypothesis
     - joblib
     - fastparquet

.pre-commit-config.yaml

@@ -10,16 +10,16 @@ repos:
     -   id: debug-statements
     -   id: end-of-file-fixer
 -   repo: https://github.com/asottile/pyupgrade
-    rev: v2.19.0
+    rev: v2.24.0
     hooks:
     -   id: pyupgrade
         args: [--py36-plus]
 -   repo: https://github.com/asottile/reorder_python_imports
-    rev: v2.5.0
+    rev: v2.6.0
     hooks:
     -   id: reorder-python-imports
 -   repo: https://github.com/psf/black
-    rev: 21.5b2
+    rev: 21.7b0
     hooks:
     -   id: black
 -   repo: https://github.com/asottile/blacken-docs
@@ -47,7 +47,7 @@ repos:
             Pygments,
         ]
 -   repo: https://github.com/PyCQA/doc8
-    rev: 0.9.0a1
+    rev: 0.9.0
     hooks:
     -   id: doc8
 # -   repo: https://github.com/codespell-project/codespell

docs/about_us.rst

@@ -68,7 +68,7 @@ an issue on Github, writing an `email`_ to or join our `zulipchat group
 
       |OSE| |space| |TRA| |space| |UniBonn| |space| |DIW|
 
-      .. |OSE| image:: https://raw.githubusercontent.com/OpenSourceEconomics/ose-corporate-design/master/logos/OSE_logo_RGB.svg
+      .. |OSE| image:: https://raw.githubusercontent.com/OpenSourceEconomics/ose-logos/main/OSE_logo_RGB.svg
         :width: 20%
         :target: https://open-econ.org
 

docs/conf.py

@@ -18,7 +18,7 @@
 author = "The respy Development Team"
 
 # The full version, including alpha/beta/rc tags.
-release = "2.1.0"
+release = "2.1.1"
 version = ".".join(release.split(".")[:2])
 
 # -- General configuration ------------------------------------------------
@@ -68,6 +68,8 @@
     "python": ("https://docs.python.org/3.8", None),
 }
 
+bibtex_bibfiles = ["explanations/refs.bib"]
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ["_templates"]
 html_static_path = ["_static"]
@@ -138,6 +140,6 @@
 
 
 html_sidebars = {
-    "index": ["sidebar-search-bs.html", "custom-intro.html"],
-    "about_us": ["sidebar-search-bs.html", "custom-about-us.html"],
+    "index": ["search-field", "custom-intro"],
+    "about_us": ["search-field", "custom-about-us"],
 }

docs/how_to_guides/how_to_numerical_integration.ipynb

@@ -2,124 +2,135 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "metadata": {},
    "source": [
     "# Numerical Integration\n",
     "\n",
     "One important component of the solution to the DCDP problem in **respy** models is numerical integration. A bottleneck in solving and estimating the model is the solution of the expected value function, the so-called $EMax(\\cdot)$. Solving the $EMax(\\cdot)$ requires us to solve a multi-dimensional integral at every point in the state space. The integrated value function does not have an analytical solution and thus requires the application of numerical methods.\n",
     "\n",
     "As the models become more complex, the computational burden increases as adding new features to the model increases the required number of function evaluations, which are the costly operation in numerical integration. Numerical integration usually uses monte carlo simulation. Results from applied mathematics, however, suggest methods that are more efficient and thus enable a performance increase. For the same number of function evaluations (and hence computational cost) quasi-Monte Carlo methods achieve a significantly higher accuracy. **respy** thus enables users to select between various methods for the numerical approximation of the $EMax(\\cdot)$. The numerical integration is controlled in the `options` of a specified model.\n"
-   ]
+   ],
+   "metadata": {}
   },
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {},
-   "outputs": [],
    "source": [
-    "import respy as rp\n",
+    "import respy as rp\r\n",
     "_, options = rp.get_example_model(\"kw_94_one\", with_data=False)"
-   ]
+   ],
+   "outputs": [],
+   "metadata": {}
   },
   {
    "cell_type": "markdown",
-   "metadata": {},
    "source": [
     "## Numerical integration method\n",
     "\n",
     "The option `monte_carlo_sequence` controls how points are drawn.\n",
     "\n",
     "- `random`: Points are drawn randomly (crude Monte Carlo).\n",
     "- `sobol` or `halton`: Points are drawn from low-discrepancy sequences (superiority in coverage). This means a given approximation error can be achieved with less points."
-   ]
+   ],
+   "metadata": {}
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "<div class=\"alert alert-block alert-warning\">\r\n",
+    "**Note**: **respy** relies on [chaospy](https://chaospy.readthedocs.io/en/master) for the `sobol` and `halton` sequence. You need to install it in addition to **respy**.\r\n",
+    "</div>"
+   ],
+   "metadata": {}
   },
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {},
+   "source": [
+    "options[\"monte_carlo_sequence\"]"
+   ],
    "outputs": [
     {
+     "output_type": "execute_result",
      "data": {
       "text/plain": [
        "'random'"
       ]
      },
-     "execution_count": 3,
      "metadata": {},
-     "output_type": "execute_result"
+     "execution_count": 3
     }
    ],
-   "source": [
-    "options[\"monte_carlo_sequence\"]"
-   ]
+   "metadata": {}
   },
   {
    "cell_type": "markdown",
-   "metadata": {},
    "source": [
-    "## Solution draws\n",
-    "\n",
+    "## Solution draws\r\n",
+    "\r\n",
     "The number of solution draws controls how many points are used to evaluate an integral. You can specify them using the option `solution_draws`."
-   ]
+   ],
+   "metadata": {}
   },
   {
    "cell_type": "code",
    "execution_count": 4,
-   "metadata": {},
+   "source": [
+    "options[\"solution_draws\"]"
+   ],
    "outputs": [
     {
+     "output_type": "execute_result",
      "data": {
       "text/plain": [
        "500"
       ]
      },
-     "execution_count": 4,
      "metadata": {},
-     "output_type": "execute_result"
+     "execution_count": 4
     }
    ],
-   "source": [
-    "options[\"solution_draws\"]"
-   ]
+   "metadata": {}
   },
   {
    "cell_type": "markdown",
-   "metadata": {},
    "source": [
     "Increasing the number of solution draws increases the accuracy of the solution at the cost of the computational burden."
-   ]
+   ],
+   "metadata": {}
   },
   {
    "cell_type": "raw",
-   "metadata": {},
    "source": [
-    "<div class=\"d-flex flex-row gs-torefguide\">\n",
-    "    <span class=\"badge badge-info\">Project</span>\n",
-    "\n",
-    "    Find an exploration of numerical integration methods in   \n",
-    "    EKW models in <a\n",
-    "    href=\"../projects/numerical_integration.html\">Improving the Numerical Integration</a>.\n",
+    "<div class=\"d-flex flex-row gs-torefguide\">\r\n",
+    "    <span class=\"badge badge-info\">Project</span>\r\n",
+    "\r\n",
+    "    Find an exploration of numerical integration methods in   \r\n",
+    "    EKW models in <a\r\n",
+    "    href=\"../projects/numerical_integration.html\">Improving the Numerical Integration</a>.\r\n",
     "</div>"
-   ]
+   ],
+   "metadata": {}
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
+   "name": "python3",
+   "display_name": "Python 3.9.5 64-bit ('respy': conda)"
   },
   "language_info": {
+   "name": "python",
+   "version": "3.9.6",
+   "mimetype": "text/x-python",
    "codemirror_mode": {
     "name": "ipython",
     "version": 3
    },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.9"
+   "nbconvert_exporter": "python",
+   "file_extension": ".py"
+  },
+  "interpreter": {
+   "hash": "b549dadab0c2edb1c58f223f7584f57e60f2cc8e65ef8392efb9b23cb30dad20"
   }
  },
  "nbformat": 4,

docs/release_notes.rst

@@ -21,6 +21,8 @@ releases are available on `Anaconda.org
   (:ghuser:`MaxBlesch`, :ghuser:`amageh`).
 - :gh:`406` More information in example models guide (:ghuser:`carolinalvarez`, 
   :ghuser:`amageh`).
+- :gh:`414` Fix bug in simulate that added untransformed shocks to df. Removes
+  chaospy from conda dependencies(:ghuser:`amageh`).
 
 
 2.0.0 - 2019-2020

docs/rtd_environment.yml

@@ -9,8 +9,8 @@ dependencies:
   - sphinx
   - sphinx-autobuild
   - sphinx-autoapi
-  - sphinxcontrib-bibtex<2.0.0
-  - pydata-sphinx-theme<0.6.0
+  - sphinxcontrib-bibtex>=2.0.0
+  - pydata-sphinx-theme>=0.6.0
   - parso>=0.8.1
   - pip:
     - sphinx-tabs

docs/tutorials/index.rst

@@ -112,7 +112,7 @@ you will find several cross-references to the theoretical concepts behind the mo
                         <div class="card-header">Exogenous Processes</div>
                         <div class="card-body flex-fill">
                             <p class="card-text">
-                                We introduce exogenous processes such as childrearing 
+                                We introduce exogenous processes such as child rearing 
                                     or health conditions to our model.
                             </p>
                         </div>

docs/tutorials/installation.rst

@@ -47,6 +47,17 @@ path, installing **respy** is as simple as typing
 in a command shell. The whole package repository can be found under
 https://anaconda.org/OpenSourceEconomics/respy.
 
+
+If you want to use different numerical integration methods implemented in **respy** you
+also need to additionally install the package
+`chaospy <https://chaospy.readthedocs.io>`_ as it is not added automatically as a
+package dependency.
+
+.. code-block:: bash
+
+    $ pip install chaospy
+
+
 As **respy** relies heavily on ``pandas``, you might also want to install their
 `recommended dependencies <https://pandas.pydata.org/pandas-docs/stable/getting_started/
 install.html#recommended-dependencies>`_ to speed up internal calculations done with

environment.yml

@@ -7,13 +7,12 @@ dependencies:
   - pip
   - anaconda-client
   - bottleneck
-  - chaospy
   - click
   - codecov
   - conda-build
   - conda-verify
   - doc8
-  - estimagic>=0.0.30
+  - estimagic>=0.1.2
   - fastparquet
   - hypothesis
   - joblib
@@ -26,7 +25,7 @@ dependencies:
   - numexpr
   - numpydoc
   - pandas>=0.24
-  - numpy>=1.2
+  - numpy>=1.21.0
   - pdbpp
   - pre-commit
   - pyarrow
@@ -41,12 +40,13 @@ dependencies:
   - snakeviz
   - sphinx
   - sphinx-autobuild
-  - sphinxcontrib-bibtex<2.0.0
+  - sphinxcontrib-bibtex>=2.0.0
   - sphinx-autoapi
-  - pydata-sphinx-theme>=0.3.0
+  - pydata-sphinx-theme>=0.6.0
   - tox-conda
   - pip:
     - apprise
     - bump2version
     - pytest-randomly
     - sphinx-tabs
+    - chaospy>=4.2.3

respy/__init__.py

@@ -35,7 +35,7 @@
     "add_noise_to_params",
 ]
 
-__version__ = "2.1.0"
+__version__ = "2.1.1"
 
 
 def test(*args, **kwargs):

respy/config.py

@@ -3,6 +3,14 @@
 
 import numpy as np
 
+# Check if chaospy is installed.
+try:
+    import chaospy  # noqa
+except ImportError:
+    CHAOSPY_INSTALLED = False
+else:
+    CHAOSPY_INSTALLED = True
+
 # Obtain the root directory of the package. Do not import respy which creates a circular
 # import.
 ROOT_DIR = Path(__file__).parent
@@ -58,7 +66,7 @@
     "solution_seed": 3,
     "core_state_space_filters": [],
     "negative_choice_set": {},
-    "monte_carlo_sequence": "sobol",
+    "monte_carlo_sequence": "random",
     "cache_compression": "snappy",
 }
 

respy/method_of_simulated_moments.py

@@ -292,6 +292,8 @@ def get_diag_weighting_matrix(empirical_moments, weights=None):
         }
 
         flat_weights = _flatten_index(weights)
+        flat_empirical_moments = _flatten_index(empirical_moments)
+        flat_weights = flat_weights.reindex_like(flat_empirical_moments)
 
     return np.diag(flat_weights)
 
@@ -369,10 +371,10 @@ def _flatten_index(moments):
     pandas.DataFrame
 
     """
-    moments = copy.deepcopy(moments)
     data_flat = []
 
     for name, series_or_df in moments.items():
+        series_or_df = series_or_df.copy(deep=True)
         series_or_df.index = series_or_df.index.map(str)
         # Unstack pandas.DataFrames and pandas.Series to add
         # columns/name to index.