Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement from_namelist method #43

Merged
merged 28 commits into from
Jul 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
3791e8f
Introduce namelists in the accessor
malmans2 Jun 24, 2021
5375197
Merge branch 'namelists' of https://github.com/malmans2/pyDOMCFG into…
malmans2 Jun 24, 2021
76b1722
Untested, but the main implementation should be in good shape
malmans2 Jun 24, 2021
71d11e5
use class variable introduced
malmans2 Jun 24, 2021
e074650
add tests
malmans2 Jun 24, 2021
749c16a
accessor now handles 999999
malmans2 Jun 25, 2021
b1a3c0e
Update accessor.py
jdha Jun 25, 2021
0da64b5
Merge branch 'pyNEMO:main' into namelists
malmans2 Jun 28, 2021
8ac4c72
introduce _check_namelist_entries
malmans2 Jun 28, 2021
72f0434
typo
malmans2 Jun 28, 2021
4dc3417
tidy up namelist check
malmans2 Jun 28, 2021
82ca98d
add type hints
malmans2 Jun 28, 2021
2a1d06b
minor semplification
malmans2 Jun 29, 2021
c0dfd5c
add more checks
malmans2 Jul 1, 2021
52ac3fb
let Zco handle checks
malmans2 Jul 1, 2021
066adce
deprecate ldbletanh
malmans2 Jul 1, 2021
c68bdf7
fix comment
malmans2 Jul 1, 2021
a9ecfe5
better error print
malmans2 Jul 1, 2021
9f7e188
decorate __call__
malmans2 Jul 2, 2021
54e1b24
clean utils
malmans2 Jul 2, 2021
4b7578d
add tests
malmans2 Jul 2, 2021
2dc3e23
use None
malmans2 Jul 2, 2021
271d7cd
ready for review
malmans2 Jul 5, 2021
501c120
Merge branch 'pyNEMO:main' into namelists
malmans2 Jul 5, 2021
abdb9f2
fix doc and ci
malmans2 Jul 5, 2021
c485512
qqMerge branch 'namelists' of https://github.com/malmans2/pyDOMCFG in…
malmans2 Jul 5, 2021
18b82fa
better docs
malmans2 Jul 5, 2021
2165ab7
avoid numpy with type hint bug
malmans2 Jul 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,26 @@ jobs:
- name: Run Tests
shell: bash -l {0}
run: pytest --no-cov

bare-environment:
name: bare-environment
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: conda-incubator/setup-miniconda@v2
with:
environment-file: ci/bare-environment.yml
activate-environment: pydomcfg_test_bare
auto-update-conda: false
miniforge-variant: Mambaforge
use-mamba: true

- name: Set up conda environment
shell: bash -l {0}
run: |
python -m pip install -e .
conda list

- name: Run Tests
shell: bash -l {0}
run: pytest --no-cov
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ repos:
hooks:
- id: mypy
exclude: docs
additional_dependencies: [xarray, types-pkg_resources]
additional_dependencies: [xarray, types-pkg_resources, numpy!=1.21.0]

- repo: https://github.com/PyCQA/doc8
rev: 0.9.0a1
Expand Down
9 changes: 9 additions & 0 deletions ci/bare-environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name: pydomcfg_test_bare
channels:
- conda-forge
dependencies:
- pooch
- pytest-cov
- pytest
- pytest-xdist
- xarray
1 change: 1 addition & 0 deletions ci/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ dependencies:
- pytest
- pytest-xdist
- xarray
- f90nml
1 change: 1 addition & 0 deletions ci/upstream-dev-env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ dependencies:
- pip
- pip:
- git+https://github.com/pydata/xarray.git
- git+https://github.com/marshallward/f90nml.git
4 changes: 3 additions & 1 deletion docs/developers/whats-new.rst
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
.. currentmodule:: pydomcfg
.. currentmodule:: xarray

What's New
----------

Unreleased
==========

- Introduced :py:meth:`Dataset.domcfg.from_namelist` to use
``NEMO DOMAINcfg`` namelists (:pr:`43`)
- Introduced the ``domcfg`` accessor (:pr:`36`)
- Added :py:meth:`~pydomcfg.tests.bathymetry.Bathymetry.sea_mount`
useful to generate classic sea mount test case. (:pr:`17`)
Expand Down
1 change: 1 addition & 0 deletions docs/users/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Methods
:toctree: generated/
:template: autosummary/accessor_method.rst

Dataset.domcfg.from_namelist
Dataset.domcfg.zco

Domzgr
Expand Down
7 changes: 6 additions & 1 deletion docs/users/installing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ Required dependencies
- `xarray <http://xarray.pydata.org/>`_
- `numpy <http://www.numpy.org/>`_

Optional dependencies
---------------------

- `f90nml <https://f90nml.readthedocs.io/>`_

Instructions
------------

Expand All @@ -18,5 +23,5 @@ The best way to install all dependencies is to use `conda <http://conda.io/>`_.

.. code-block:: sh

conda install -c conda-forge xarray pip
conda install -c conda-forge xarray f90nml pip
pip install git+https://github.com/pyNEMO/pyDOMCFG.git
128 changes: 124 additions & 4 deletions pydomcfg/accessor.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,36 @@
from typing import Any, Callable, TypeVar, cast
import inspect
import warnings
from collections import ChainMap
from functools import wraps
from pathlib import Path
from typing import IO, Any, Callable, TypeVar, Union, cast

import xarray as xr
from xarray import Dataset

from .domzgr.zco import Zco

try:
import f90nml

HAS_F90NML = True
except ImportError:
HAS_F90NML = False

F = TypeVar("F", bound=Callable[..., Any])
ZGR_MAPPER = {
"ln_zco": Zco,
# "ln_zps": TODO,
# "ln_sco": TODO
}


def _jpk_check(func: F) -> F:
"""
Decorator to raise an error if jpk was not set
"""

@wraps(func)
def wrapper(self, *args, **kwargs):
if not self.jpk:
raise ValueError(
Expand All @@ -29,7 +47,9 @@ class Accessor:
def __init__(self, xarray_obj: Dataset):
self._obj = xarray_obj
self._jpk = 0
self._nml_ref_path = ""

# Set attributes
@property
def jpk(self) -> int:
"""
Expand All @@ -43,12 +63,112 @@ def jpk(self) -> int:

@jpk.setter
def jpk(self, value: int):
if value <= 0:
raise ValueError("`jpk` MUST be > 0")
if value < 0:
raise ValueError("`jpk` MUST be >= 0 (use 0 to unset jpk)")
self._jpk = value

@property
def nml_ref_path(self) -> str:
"""
Path to reference namelist.

Returns
-------
str
"""
return self._nml_ref_path

@nml_ref_path.setter
def nml_ref_path(self, value: str):
self._nml_ref_path = value

# domzgr methods
# TODO:
# I think the process of creating the public API and doc
# can be further automatized, but let's not put too much effort into it
# until we settle on the back-end structure:
# See: https://github.com/pyNEMO/pyDOMCFG/issues/45
@_jpk_check
def zco(self, *args: Any, **kwargs: Any) -> Dataset:
return Zco(self._obj, self._jpk)(*args, **kwargs)
name = inspect.stack()[0][3]
return ZGR_MAPPER["ln_" + name](self._obj, self._jpk)(*args, **kwargs)

zco.__doc__ = Zco.__call__.__doc__

# Emulate NEMO DOMAINcfg tools
def from_namelist(self, nml_cfg_path_or_io: Union[str, Path, IO[str]]) -> Dataset:
"""
Auto-populate pydomcfg parameters using NEMO DOMAINcfg namelists.

Parameters
----------
nml_cfg_path_or_io: str, Path, IO
Path pointing to a namelist_cfg,
or namelist_cfg previously opened with open()

Returns
-------
Dataset
"""

nml_chained = self._namelist_parser(nml_cfg_path_or_io)
zgr_initialized, kwargs = self._get_zgr_initialized_and_kwargs(nml_chained)
return zgr_initialized(**kwargs)

def _namelist_parser(
self, nml_cfg_path_or_io: Union[str, Path, IO[str]]
) -> ChainMap:
"""Parse namelists using f90nml, chaining all namblocks"""

if not HAS_F90NML:
raise ImportError(
"`f90nml` MUST be installed to use `obj.domcfg.from_namelist()`"
)

if not self.nml_ref_path:
raise ValueError(
"Set `nml_ref_path` before calling `obj.domcfg.from_namelist()`"
" For example: obj.domcfg.nml_ref_path = 'path/to/nml_ref'"
)

if self.jpk:
warnings.warn(
"`obj.domcfg.jpk` is ignored. `jpk` is inferred from the namelists."
)

# Read namelists: cfg overrides ref
nml_cfg = f90nml.read(nml_cfg_path_or_io)
nml = f90nml.patch(self.nml_ref_path, nml_cfg)

return ChainMap(*nml.todict().values())

def _get_zgr_initialized_and_kwargs(self, nml_chained: ChainMap):

# TODO: Add return type hint when abstraction in base class is implemented

# Pick the appropriate class
zgr_classes = [
value for key, value in ZGR_MAPPER.items() if nml_chained.get(key)
]
if len(zgr_classes) != 1:
raise ValueError(
"One and only one of the following variables MUST be `.true.`:"
f" {tuple(ZGR_MAPPER)}"
)
zgr_class = zgr_classes[0]

# Compatibility with NEMO DOMAINcfg
if nml_chained.get("ldbletanh") is False:
for pp in ["ppa2", "ppkth2", "ppacr2"]:
nml_chained[pp] = None

# Get kwargs, converting 999_999 to None
parameters = list(inspect.signature(zgr_class.__call__).parameters)
parameters.remove("self")
kwargs = {
key: None if nml_chained[key] == 999_999 else nml_chained[key]
for key in parameters
if key in nml_chained
}

return zgr_class(self._obj, nml_chained["jpkdta"]), kwargs