Squashed commit of the following:

commit c2a058d Merge: d94014c c206fc4 Author: Arthur van Dam <arthurvd@gmail.com> Date: Wed Jul 6 15:15:43 2022 +0200 Merge branch 'feat/242_lateraldischarge_forcingmodel' commit c206fc4 Author: Arthur van Dam <arthurvd@gmail.com> Date: Wed Jul 6 15:13:09 2022 +0200 Fix lateral tests were failing due to improved checking for realtime/float/filename commit d94014c Author: Arthur van Dam <arthurvd@gmail.com> Date: Wed Jul 6 14:56:30 2022 +0200 feat: support .bc files and more in lateral discharge (#244) * feat: support .bc files and more in lateral discharge Truly distinguish between a scalar float constant, keyword "realtime" and a .bc file. * fix invalid filename in unit test (now failed due to improved value parsing of previous commit) * Move ForcingData and RealTime to .bc such that they can be reused later in structure commit 959007e Author: Maarten Pronk <git@evetion.nl> Date: Wed Jul 6 14:53:21 2022 +0200 Removed sonarcloud and reset caches for docs. commit b9b1c06 Author: Maarten Pronk <evetion@users.noreply.github.com> Date: Wed Jul 6 14:03:53 2022 +0200 Upgrade poetry and cache versions to reset caches. (#267) commit ed896f9 Author: Prisca van der Sluis <36264671+priscavdsluis@users.noreply.github.com> Date: Tue Jul 5 08:30:23 2022 +0200 fix: allow empty file paths in the .fnm file issue: #82 #260 commit 087e0e0 Author: Maarten Pronk <git@evetion.nl> Date: Mon Jul 4 15:08:10 2022 +0200 Updated package description. commit 240def0 Author: Arthur van Dam <arthurvd@gmail.com> Date: Tue Jun 28 18:07:37 2022 +0200 Move ForcingData and RealTime to .bc such that they can be reused later in structure commit 46f2955 Author: Arthur van Dam <arthurvd@gmail.com> Date: Tue Jun 28 17:43:59 2022 +0200 fix: Add modeltype 21 to RR node for open water precipitation/evaporation (#261) * Add modeltype 21 to RR node for open water preciptation/evaporation * Document RR mt nodetypes inline Co-authored-by: Prisca van der Sluis <36264671+priscavdsluis@users.noreply.github.com> commit 5134129 Merge: 8ac6bae 6798df9 Author: Arthur van Dam <arthurvd@gmail.com> Date: Tue Jun 28 15:10:05 2022 +0200 Merge branch 'main' into feat/242_lateraldischarge_forcingmodel commit 6798df9 Author: Arthur van Dam <arthurvd@gmail.com> Date: Tue Jun 28 10:00:22 2022 +0200 168: fix the not-yet-fixed rr docs commit 8957b9f Author: Arthur van Dam <arthurvd@gmail.com> Date: Tue Jun 28 08:56:43 2022 +0200 168: Fix rr reference docs for new package structure commit 3586d6a Author: Arthur van Dam <arthurvd@gmail.com> Date: Mon Jun 27 21:55:32 2022 +0200 feat: add support for observation point ini file (#236) * feat: add support for observation point ini file code + tests + docs * #160: add forgotten test data file for obspoints * Generalize Lateral's location validator into util. Such that Observation can also use it (and later others as well) * Fix minor typo in TestLateral commit 024f58c Author: Maarten Tegelaers <BeardedPlatypus@users.noreply.github.com> Date: Mon Jun 27 16:34:47 2022 +0200 fix(filemodel): ResolveRelativeMode is incorrectly set when reading a model with 'pathsRelativeToParent' set to false (#259) * Add separate relative mode deduction when reading a model * Add additional documentation to FileModel init * Fix typo in comment * Add additional information detailing the data param * autoformat: isort & black * Docstring link in documentation only. Co-authored-by: Arthur van Dam <arthurvd@gmail.com> commit cd83c7c Author: Prisca van der Sluis <36264671+priscavdsluis@users.noreply.github.com> Date: Mon Jun 27 15:03:18 2022 +0200 #229: Add modeldata and geometry subdirectories (#257) chore: prepare hydrolib.core.modeldata and hydrolib.core.geometry subpackages (no functionality yet). commit cb1e53b Author: Arthur van Dam <arthurvd@gmail.com> Date: Mon Jun 27 10:47:38 2022 +0200 refactor(rr)!: place all RR-related code+tests in own rr subpackage (#235) * renamed most "bui" dirs/package names to "rr.meteo" * fnm/ toplevels files moved to rr/ (no separate subdir for fnm, open for discussion) commit 4e005bd Author: Maarten Tegelaers <BeardedPlatypus@users.noreply.github.com> Date: Mon Jun 20 18:13:36 2022 +0200 chore(devcontainer): fix docker devcontainer behaviour (#231) * add --no-interaction and --no-ansi flags to poetry install * replace extension test adapter with default adapter * Update the poetry logic in the docker container * Update the documentation of the devcontainer commit aa4dd9a Author: rhutten <55204099+rhutten@users.noreply.github.com> Date: Mon Jun 20 14:00:07 2022 +0200 Docs/246 update first steps tutorial notebook (#254) * #246 Markdown file 'first steps' is converted to Jupyter notebook * #246 Python packages are added to generate documentation from Jupyter Notebooks in developer mode. * #246 Markdown updated to notebook and introduction to tutorials is added. * #246 Warning regarding compound structures is solved when running pytest * #246 update of documentation lay-out * Update tutorials.md * #246 review implemented Co-authored-by: Arthur van Dam <arthurvd@gmail.com> commit 7c00946 Author: Maarten Pronk <git@evetion.nl> Date: Fri Jun 3 11:17:35 2022 +0200 Forbid fault mkdocs-autorefs version. commit 7e211ec Author: Maarten Pronk <git@evetion.nl> Date: Fri Jun 3 11:03:46 2022 +0200 Downgrade markupsafe. commit 3211f37 Author: Maarten Pronk <git@evetion.nl> Date: Fri Jun 3 10:53:52 2022 +0200 Downgrade jinja2. commit b5e9da3 Author: Maarten Pronk <evetion@users.noreply.github.com> Date: Fri Jun 3 10:46:16 2022 +0200 Updated hydrolib-core dependencies to easy combined installation with other packages. (#245) commit 8ac6bae Author: Arthur van Dam <arthurvd@gmail.com> Date: Fri Jun 3 08:09:50 2022 +0200 fix invalid filename in unit test (now failed due to improved value parsing of previous commit) commit d9d3fb1 Author: Arthur van Dam <arthurvd@gmail.com> Date: Fri Jun 3 08:05:05 2022 +0200 feat: support .bc files and more in lateral discharge Truly distinguish between a scalar float constant, keyword "realtime" and a .bc file. commit 2197a1e Author: Arthur van Dam <arthurvd@gmail.com> Date: Wed Jun 1 17:38:52 2022 +0200 doc: extend docstrings of almost all .bc classes commit 0e156c7 Author: Arthur van Dam <arthurvd@gmail.com> Date: Wed Jun 1 17:34:49 2022 +0200 docs: fix some docstring typos commit a5ece8b Author: Arthur van Dam <arthurvd@gmail.com> Date: Thu May 19 14:22:18 2022 +0200 refactor: remove dead code of _process_edges_for_branch (which contained a bug as well) commit 24dc275 Author: Arthur van Dam <arthurvd@gmail.com> Date: Thu May 19 13:57:31 2022 +0200 Fix copyright year to 2022. commit 1657e9d Author: grongen <guusrongen@gmail.com> Date: Thu May 19 11:26:34 2022 +0200 Bug/233 error short connecting branch (#234) * Add extra node coordinate to branch Add extra node coordinate to branch as well, when the initial branch only contains already present coordinates. This solved the indexing error later on. * Added test to check if coordinate is added as well Added test to check if coordinate is added as well, when adding small connecting branch commit 2f35795 Author: Arthur van Dam <arthurvd@gmail.com> Date: Thu Apr 21 15:39:23 2022 +0200 Add issue template "Task" commit f47e822 Author: Arthur van Dam <arthurvd@gmail.com> Date: Thu Apr 21 15:29:06 2022 +0200 Add issue template "Feature request"
Deltares · Jul 6, 2022 · 43db2a3 · 43db2a3
1 parent 5a3b123
commit 43db2a3
Show file tree

Hide file tree

Showing 60 changed files with 3,464 additions and 781 deletions.
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -13,12 +13,14 @@
 	// Additional extensions can be added here according to personal taste.
 	"extensions": [
 		"ms-python.python",
-		"littlefoxteam.vscode-python-test-adapter",
+		"ms-vscode.test-adapter-converter",
 		"oderwat.indent-rainbow",
 		"njpwerner.autodocstring",
 		"njqdev.vscode-python-typehint"
 	],
 	"mounts": [
 		"source=/var/run/docker.sock,target=/var/run/docker.sock,type=bind"
 	],
+	// This ensures the HYDROLIB environment is properly installed. 
+	"postCreateCommand": "poetry install --no-root --no-interaction --no-ansi"
 }
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem or missing functionality is.
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered (if applicable).
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
diff --git a/.github/ISSUE_TEMPLATE/task.md b/.github/ISSUE_TEMPLATE/task.md
@@ -0,0 +1,17 @@
+---
+name: Task
+about: Describe a piece of work (typically non-coding) that should be done.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**What is the need for this task.**
+Why should this work be done?
+
+**What is the task?**
+A clear and concise description of what you want to happen.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -23,9 +23,9 @@ jobs:
       - name: Run image
         uses: abatilo/actions-poetry@v2.0.0
         with:
-          poetry-version: 1.1.8
+          poetry-version: 1.1.11
       - name: Cache Poetry virtualenv
-        uses: actions/cache@v1
+        uses: actions/cache@v3
         id: cache
         with:
           path: ~/.virtualenvs
@@ -53,10 +53,3 @@ jobs:
           git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/$GITHUB_REPOSITORY
           git checkout $GITHUB_HEAD_REF
           git commit -am "autoformat: isort & black" && git push || true
-
-      - name: SonarCloud Scan
-        uses: SonarSource/sonarcloud-github-action@master
-        if: ${{ (matrix.os == 'ubuntu-18.04') && (matrix.python-version == 3.8) }}
-        env:
-          GITHUB_TOKEN: ${{ secrets.SONAR_GITHUB_TOKEN }} # Needed to get PR information, if any
-          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
@@ -17,9 +17,9 @@ jobs:
       - name: Run image
         uses: abatilo/actions-poetry@v2.0.0
         with:
-          poetry-version: 1.1.6
+          poetry-version: 1.1.11
       - name: Cache Poetry virtualenv
-        uses: actions/cache@v1
+        uses: actions/cache@v3
         id: cache
         with:
           path: ~/.virtualenvs

diff --git a/.github/workflows/docs_release.yml b/.github/workflows/docs_release.yml
@@ -18,9 +18,9 @@ jobs:
       - name: Run image
         uses: abatilo/actions-poetry@v2.0.0
         with:
-          poetry-version: 1.1.6
+          poetry-version: 1.1.11
       - name: Cache Poetry virtualenv
-        uses: actions/cache@v1
+        uses: actions/cache@v3
         id: cache
         with:
           path: ~/.virtualenvs

diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml
@@ -29,10 +29,10 @@ jobs:
       - name: Run image
         uses: abatilo/actions-poetry@v2.0.0
         with:
-          poetry-version: 1.1.8
+          poetry-version: 1.1.11
 
       - name: Cache Poetry virtualenv
-        uses: actions/cache@v1
+        uses: actions/cache@v3
         id: cache
         with:
           path: ~/.virtualenvs

diff --git a/Dockerfile b/Dockerfile
@@ -25,6 +25,7 @@ ENV PYTHONUNBUFFERED=1 \
     POETRY_VIRTUALENVS_IN_PROJECT=true \
     # do not ask any interactive question
     POETRY_NO_INTERACTION=1 \
+    POETRY_NO_ANSI=1\
     \
     # paths
     # this is where our requirements + virtual environment will live
@@ -34,8 +35,6 @@ ENV PYTHONUNBUFFERED=1 \
 # prepend poetry and venv to path
 ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"
 
-# `builder-base` stage is used to build deps + create our virtual environment
-FROM python-base as builder-base
 RUN apt-get update \
     && apt-get install --no-install-recommends -y \
     # deps for installing poetry
@@ -46,32 +45,19 @@ RUN apt-get update \
 # install poetry - respects $POETRY_VERSION & $POETRY_HOME
 RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python
 
-# copy project requirement files here to ensure they will be cached.
-WORKDIR $PYSETUP_PATH
-COPY poetry.lock pyproject.toml ./
-
-# install runtime deps - uses $POETRY_VIRTUALENVS_IN_PROJECT internally
-RUN poetry install --no-dev --no-root
-
-
 # `development` image is used during development / testing
 FROM python-base as development
 ENV POETRY_VIRTUALENVS_IN_PROJECT=true
-WORKDIR $PYSETUP_PATH
 
 # Install git, to ensure the git tools work within vs-code
 RUN apt-get update \
     && apt-get install git-all --no-install-recommends -y
 
+COPY --from=python-base $POETRY_HOME $POETRY_HOME
 
-# copy in our built poetry + venv
-COPY --from=builder-base $POETRY_HOME $POETRY_HOME
-COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH
-
-# quicker install as runtime deps are already installed
-RUN poetry install --no-root
-
-# will become mountpoint of our code
 WORKDIR /app
 
+# The actual 'poetry install' is called after the code is mounted. 
+# See '.devcontainer/devcontainer.json'
+
 EXPOSE 8000
diff --git a/docs/guides/devcontainers.md b/docs/guides/devcontainers.md
@@ -40,15 +40,24 @@ or we can select the  "Open a Remote Window" button, the green button in the bot
 code window, after which we can select "Reopen in container". Both will then spin up a new container in which we can work.
 Note that if it is the first time starting our repository, or if we have made changes to the Docker Images we might need to build the container, which could take a few moments.
 
-Once opened in a separate container, we can start a terminal to verify everything is working correctly. 
-When we start a new terminal, we should see the terminal of our container, e.g. something a long the lines of
-
-```bash
-(.venv) root@7573572275f1:/workspaces/HYDROLIB-core# 
-```
-
-It should now be possible to run the HYDROLIB-Core tests within our container. You might get prompted to configure either
-the python interpreter, or the python test framework. Once this is done all tests should pass as they would normally.
+Once opened in a separate container, we can start a terminal to verify everything is 
+working correctly. First, we need to ensure the correct python interpreter is used. 
+We want to use the virtual environment created by poetry, which default resides in 
+`./.venv/`. Visual Studio Code might prompt you to select an interpreter. If it does
+press the 'Select Python Interpreter', and select the virtual environment residing in
+`./.venv/`. If it does not explicitly prompt you (or if you want to change it later on),
+press `ctrl + shift + p`, and select 'Python: Select Interpreter'. If it currently is 
+not set to the virtual environment, change it to the correct interpreter.
+
+It should now be possible to run the HYDROLIB-Core tests within our container by opening 
+the 'Testing' tab. If the 'Testing' is not already configured in the 
+`./.vscode/settings.json`, we will need to configure the Python tests:
+
+1. click the 'Configure Python Tests' button in the 'Testing' tab
+2. Select 'pytest' in the pop-up window
+3. Select 'tests', this will ensure tests are located in our 'tests' directory
+
+Once this is done all tests should pass as they would normally.
 
 ##  Dockerfile configures python, poetry and other dependencies
 

diff --git a/docs/reference/bui.md b/docs/reference/bui.md
diff --git a/docs/reference/obs.md b/docs/reference/obs.md
@@ -0,0 +1,11 @@
+# Observation point .ini files
+The observation point module provides the specific logic for accessing observation point files
+for a [D-Flow FM](glossary.md#d-flow-fm) model.
+
+Generic parsing and serializing functionality comes from the generic hydrolib.core.io.ini modules.
+
+An observation point .ini file is described by the classes below.
+
+## Model
+
+::: hydrolib.core.io.obs.models
diff --git a/docs/reference/rr/meteo.md b/docs/reference/rr/meteo.md
@@ -0,0 +1,13 @@
+# The Rainfall Runoff meteo layer
+The meteo layer currently contains only support for the BUI file.
+The "bui" file contains the precipitation input data for [Rainfall Runoff](glossary.md#rainfall-runoff).
+It is represented by the classes below.
+
+## Model
+::: hydrolib.core.io.rr.meteo.models
+
+## Parser
+::: hydrolib.core.io.rr.meteo.parser
+
+## Serializer
+::: hydrolib.core.io.rr.meteo.serializer
diff --git a/docs/reference/rr.md → docs/reference/rr/rr.md b/docs/reference/rr.md → docs/reference/rr/rr.md
@@ -6,11 +6,11 @@ Support is still basic, but growing. The input 'layers' listed below correspond
 in the {{sobek_um("section.D.19")}}.
 
 ## Main input (sobek_3b.fnm)
-::: hydrolib.core.io.fnm.models
+::: hydrolib.core.io.rr.models
 
 ## General layer
 ### rainfall file
-::: hydrolib.core.io.bui.models
+::: hydrolib.core.io.rr.meteo.models
 
 
 ## Topology layer ("Topography" in SOBEK UM)

diff --git a/docs/tutorials/build_basic_model.ipynb b/docs/tutorials/build_basic_model.ipynb
@@ -0,0 +1,115 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Build a basic model\n",
+    "\n",
+    "This _Build a basic model_ tutorial illustrates how to build a simple D-Flow FM model from scratch using [HYDROLIB-core](https://github.com/Deltares/HYDROLIB-core). "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Import HYDROLIB-core functionalities and setup D-Flow FM model\n",
+    "from hydrolib.core.io.structure.models import FlowDirection, StructureModel, Weir\n",
+    "from hydrolib.core.io.mdu.models import FMModel\n",
+    "\n",
+    "fm = FMModel()\n",
+    "fm.filepath = \"test.mdu\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Add some structures, note this is invalid because it doesn't \n",
+    "have a branch or coordinates yet, but it will work for demo purposes"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Add weir to the model and assign structure to D-Flow FM model \n",
+    "struc = Weir(branchId='someBranch', chainage = 123.0, allowedflowdir=FlowDirection.none, crestlevel=0.0)\n",
+    "struc.comments.crestlevel = \"This is a comment\"\n",
+    "fm.geometry.structurefile = [StructureModel(structure=[struc])]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note that the creation of a `Weir` and other model objects requires several input arguments.\n",
+    "A `ValidationError` will be raised when the model is invalid or incomplete.\n",
+    "For instance, in the above example, if the `StructureModel` had been assigned directly to `structurefile` instead of as a list, that would have triggered a `ValidationError`.\n",
+    "\n",
+    "Now let's add this model to a DIMR config and save it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from hydrolib.core.io.dimr.models import DIMR, FMComponent\n",
+    "from pathlib import Path\n",
+    "\n",
+    "dimr = DIMR()\n",
+    "dimr.component.append(\n",
+    "    FMComponent(name=\"test\", workingDir=\".\", inputfile=fm.filepath, model=fm)\n",
+    ")\n",
+    "\n",
+    "dimr.save(recurse=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The save call on the highest-level DIMR object will result in recursive saves of all child models in the model hierarchy, so this results in four files (`dimr_config.xml`, `network.nc`, `structures.ini`,`test.mdu`) in the working directory.\n",
+    "Some more in-depth background about recursive saving of a model tree is given in another tutorial: [Loading and saving](loading_and_saving_a_model.md)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.9.13 ('.venv': poetry)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.13"
+  },
+  "orig_nbformat": 4,
+  "vscode": {
+   "interpreter": {
+    "hash": "82e465d5440b391da8d43d2f583b852925fd7bed0ed0d752c9688c1fce589220"
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}