Update master to Pastas 1.4 (#686)

.github/workflows/auto-author-assign.yml

@@ -12,4 +12,4 @@ jobs:
   assign-author:
     runs-on: ubuntu-latest
     steps:
-      - uses: toshimaru/auto-author-assign@v1.6.2
+      - uses: toshimaru/auto-author-assign@v2.1.0

.github/workflows/ci.yml

@@ -38,7 +38,7 @@ jobs:
           - name: Test suite with py312-ubuntu
             python: "3.12"
             toxenv: py312
-            experimental: true
+            experimental: false
           - name: Formatting with black + isort
             python: "3.9"
             toxenv: format
@@ -60,10 +60,10 @@ jobs:
       # Pytest
       PYTEST_ADDOPTS: "--color=yes"
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python }}
           check-latest: true

.github/workflows/python-publish.yml

@@ -13,9 +13,9 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: '3.x'
     - name: Install dependencies

.github/workflows/test_benchmark_notebooks.yml

@@ -25,10 +25,10 @@ jobs:
       # Pytest
       PYTEST_ADDOPTS: "--color=yes"
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
 
       - name: Set up Python ${{ matrix.python }}
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python }}
           check-latest: true

doc/examples/fix_parameters.ipynb

@@ -5,7 +5,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Fixating parameters while fitting\n",
+    "# Fixing parameters while fitting\n",
     "    \n",
     "*Developed by Mark Bakker, TU Delft*"
    ]
@@ -203,6 +203,15 @@
     "ml.solve(tmin=\"1985\", tmax=\"2010\", solver=ps.LeastSquares())"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Fixing a parameter\n",
+    "\n",
+    "Now let's fix the parameter recharge$_n$ and solve the model again. We can do this by using the `Model.set_parameter` method and setting `vary=False`. For the sake of this example, we also fix the parameter to a value of 1.0, such that the Gamma response function becomes an Exponential function."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -212,10 +221,17 @@
     "ml = ps.Model(ho)\n",
     "sm1 = ps.StressModel(recharge, ps.Gamma(), name=\"recharge\", settings=\"prec\")\n",
     "ml.add_stressmodel(sm1)\n",
-    "ml.set_parameter(\"recharge_n\", vary=False)\n",
+    "ml.set_parameter(\"recharge_n\", vary=False, initial=1)\n",
     "ml.solve(tmin=\"1985\", tmax=\"2010\", solver=ps.LeastSquares())"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In the fit report, you will see that the parameter recharge$_n$ is now not varied anymore and kept to the value of one."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -224,12 +240,19 @@
    "source": [
     "ml.plot(figsize=(10, 4));"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
   "anaconda-cloud": {},
   "kernelspec": {
-   "display_name": "pastas_dev",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -243,7 +266,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:41:22) [MSC v.1929 64 bit (AMD64)]"
+   "version": "3.11.3"
   },
   "vscode": {
    "interpreter": {

doc/examples/index.rst

@@ -47,7 +47,7 @@ Basics
 
 .. _Preprocessing user-provided time series: prepare_timeseries.html
 .. _A basic model: basic_model.html
-.. _Fixating parameters while fitting: fix_parameters.html
+.. _Fixing parameters while fitting: fix_parameters.html
 .. _Calibration: calibration_options.html
 .. _Modeling with different timesteps: modeling_timestep.html
 

doc/examples/recharge_estimation.ipynb

@@ -103,6 +103,7 @@
     "dt = 10\n",
     "freq = \"10D\"\n",
     "h = head.iloc[0::dt]\n",
+    "h = h.loc[~h.index.duplicated(keep='first')]\n",
     "\n",
     "mls = {\n",
     "    \"Linear\": [ps.FourParam(), ps.rch.Linear()],\n",

doc/examples/signatures.ipynb

@@ -9,17 +9,17 @@
    "source": [
     "# Groundwater signatures\n",
     "\n",
-    "*R.A. Collenteur, University of Graz, 2022*\n",
+    "*R.A. Collenteur, Eawag, 2023*\n",
     "\n",
-    "In this notebook we introduce the **groundwater signatures** module available in Pastas. The signatures methods can be accessed through the `signatures` module in the `pastas.stats` sub-package. This module is partly based on the work from [Heudorfer et al. (2019)](#References) and others (see [References](#References)).\n",
+    "In this notebook we introduce the **groundwater signatures** module available in Pastas. The signatures methods can be accessed through the `signatures` module in the `pastas.stats` sub-package.\n",
     "\n",
-    "Groundwater signatures characterize different aspects of a groundwater time series, and can be subdivided in different categories: shape, distribution, and structure. Groundwater signatures are also referred to as 'indices' or 'quantitative' metrics. In Pastas, 'signatures' is adopted to avoid any confusion with time indices and goodness-of-fit metrics. \n",
+    "Groundwater signatures are quantitative metrics that characterize different aspects of a groundwater time series. They are commonly subdivided in different categories: shape, distribution, and structure. Groundwater signatures are also referred to as 'indices' or 'quantitative' metrics. In Pastas, 'signatures' is adopted to avoid any confusion with time indices and goodness-of-fit metrics. \n",
     "\n",
-    "The signatures can be used to *objectively* characterize different groundwater systems, for example distinquishing between fast and slow groundwater systems. The use of signatures is common in other parts of hydrology (e.g., rainfall-runoff modeling) and can be applied in all phases of modeling (see, for example [McMillan, 2020](#References) for an overview). \n",
+    "The signatures can be used to *objectively* characterize different groundwater systems, for example, distinguishing between fast and slow groundwater systems. The use of signatures is common in other parts of hydrology (e.g., rainfall-runoff modeling) and can be applied in all phases of modeling (see, for example, [McMillan, 2021](#References) for an overview). \n",
     "\n",
     "<div class=\"alert alert-warning\">\n",
     "<b>Note:</b>\n",
-    "The `signatures` module is still work in progress and needs further verification. Please report any issues and bugs on the Pastas GitHub repository!\n",
+    "The `signatures` module is under active development and any help is welcome. Please report any issues and bugs on the Pastas GitHub repository!\n",
     "</div>"
    ]
   },
@@ -103,34 +103,21 @@
    "source": [
     "## 3. Plot the results\n",
     "\n",
-    "Depending on the signature, different ranges of parameters can be expected. We therefore split the signatures in two range for visualization purposes."
+    "Depending on the signature, different ranges of parameters can be expected. We therefore normalize the signatures values by the mean value of each signature. This way we can easily compare the two groundwater systems."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "63e5dfaa",
+   "id": "e8fe7d8d-5ae1-4cae-a267-080f5cdcbf75",
    "metadata": {},
    "outputs": [],
    "source": [
-    "fig = plt.figure(constrained_layout=True, figsize=(10, 4))\n",
-    "gs = GridSpec(1, 5, figure=fig)\n",
-    "ax1 = fig.add_subplot(gs[0, :4])\n",
-    "ax2 = fig.add_subplot(gs[0, 4])\n",
-    "\n",
-    "low_values = df.where(df < 2).dropna()\n",
-    "low_values.plot(ax=ax1)\n",
-    "ax1.set_xticks(np.arange(len(low_values.index)))\n",
-    "ax1.set_xticklabels(low_values.index, rotation=90)\n",
-    "ax1.grid()\n",
-    "high_values = df.where(df > 2).dropna()\n",
-    "high_values.plot(ax=ax2)\n",
-    "ax2.set_xticks(np.arange(len(high_values)))\n",
-    "ax2.set_xticklabels(high_values.index, rotation=90)\n",
-    "ax2.grid()\n",
-    "\n",
-    "ax1.set_title(\"Low value signatures\")\n",
-    "ax2.set_title(\"High value signatures\");"
+    "_, ax = plt.subplots(1,1, figsize=(10, 4))\n",
+    "df.div(df.mean(axis=1), axis=0).mul(100).plot(ax=ax)\n",
+    "ax.set_xticks(np.arange(len(df.index)), df.index, rotation=90, )\n",
+    "ax.set_ylabel(\"% Change from the mean\\n signature  value\")\n",
+    "ax.grid()"
    ]
   },
   {
@@ -141,14 +128,16 @@
    },
    "source": [
     "## 4. Interpretation of signatures\n",
-    "The different signatures can be used to compare the different systems or characterize a single system. For example, the first head time series has a high bimodality coefficient (>0.7), indicating a bimodel distribution of the data. This makes sense, as this time series is used as an example for the non-linear threshold model (see notebook). Rather than (naively) testing all model structures, this is an example where we can potentially use a groundwater signature to identify a 'best' model structure beforehand.\n",
     "\n",
-    "Another example. The second time series is observed in a much slower groundwater system than the first. This is for example clearly visible and quantified by the different values for the 'pulse_duration', the 'recession and recovery constants', and the 'slope of the duration curves'. We could use this type of information to determine whether we should use a 'fast' or 'slow' response function (e.g., an Exponential or Gamma function). These are just some examples of how groundwater signatures can be used to improve groundwater modeling, more research on this topic is required. Please contact us if interested!\n",
+    "The different signatures can be used to compare the different systems or characterize a single system. For example, the first head time series has a high bimodality coefficient (>0.7), indicating a bimodal distribution of the data. This makes sense, as this time series is used as an example for the non-linear threshold model (see notebook). Rather than (naively) testing all model structures, this is an example where we can potentially use a groundwater signature to identify a 'best' model structure beforehand.\n",
     "\n",
-    "A little disclaimer: from the plot above it is actually not that straightforward to compare the indices, because the range in values is large. For example, the rise and fall rate visually show no differences, but their numbers vary by over 200%. Thus, interpretation requires some more work.  \n",
+    "Another example. The second time series is observed in a much slower groundwater system than the first. This is, for example, clearly visible and quantified by the different values for the 'pulse_duration', the 'recession and recovery constants', and the 'slope of the duration curves'. We could use this type of information to determine whether we should use a 'fast' or 'slow' response function (e.g., an Exponential or Gamma function). These are just some examples of how groundwater signatures can be used to improve groundwater modeling, more research on this topic is required. Please contact us if interested!\n",
+    "\n",
+    "A little disclaimer: from the data above, it is actually not that straightforward to compare the signature values because the range in values is large. For example, the rise and fall rate show small differences in absolute values, but their numbers vary by over 200%. Thus, interpretation requires some more work.  \n",
     "\n",
     "### References\n",
-    "The following references are helpfull in learning about the groundwater signatures:\n",
+    "\n",
+    "The following references are helpful in learning about the groundwater signatures:\n",
     "\n",
     "- Heudorfer, B., Haaf, E., Stahl, K., Barthel, R., 2019. [Index-based characterization and quantification of groundwater dynamics.](https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2018WR024418) Water Resources Research.\n",
     "- Haaf, E., Giese, M., Heudorfer, B., Stahl, K., Barthel, R., 2020. [Physiographic and Climatic Controls on Regional Groundwater Dynamics.](https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019WR026545) Water Resources Research.\n",
@@ -173,7 +162,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.5"
+   "version": "3.11.3"
   },
   "vscode": {
    "interpreter": {

doc/examples/uncertainty.ipynb

@@ -59,6 +59,8 @@
     "    .loc[\"2006\":]\n",
     "    .iloc[0::10]\n",
     ")\n",
+    "gwl = gwl.loc[~gwl.index.duplicated(keep='first')]\n",
+    "\n",
     "evap = pd.read_csv(\n",
     "    \"data_wagna/evap_wagna.csv\", index_col=0, parse_dates=True, skiprows=2\n",
     ").squeeze()\n",

pastas/io/base.py

@@ -138,7 +138,9 @@ def _load_model(data: dict) -> Model:
     # Add parameters, use update to maintain correct order
     ml.parameters = ml.get_init_parameters(noise=ml.settings["noise"])
     ml.parameters.update(data["parameters"])
-    ml.parameters = ml.parameters.apply(to_numeric, errors="ignore")
+
+    # Convert parameters to numeric
+    ml.parameters = ml.parameters.infer_objects()
 
     # When initial values changed
     for param, value in ml.parameters.loc[:, "initial"].items():

pastas/io/pas.py

@@ -4,9 +4,11 @@
 
 R.A. Collenteur - August 2017
 """
+
 import datetime
 import json
 from collections import OrderedDict
+from io import StringIO as stringIO
 from logging import getLogger
 
 from pandas import (
@@ -37,7 +39,7 @@ def pastas_hook(obj: dict):
             obj[key] = val
         elif key == "series":
             try:
-                obj[key] = read_json(value, typ="series", orient="split")
+                obj[key] = read_json(stringIO(value), typ="series", orient="split")
             except:
                 obj[key] = value
             if isinstance(obj[key], Series):
@@ -51,7 +53,7 @@ def pastas_hook(obj: dict):
             # Necessary to maintain order when using the JSON format!
             value = json.loads(value, object_pairs_hook=OrderedDict)
             param = DataFrame(data=value, columns=value.keys()).T
-            obj[key] = param.apply(to_numeric, errors="ignore")
+            obj[key] = param.infer_objects()
         else:
             try:
                 obj[key] = json.loads(value, object_hook=pastas_hook)