Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify conda environments and move configs to pyproject.toml #512

Merged
merged 20 commits into from
Jun 22, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ repos:
- id: check-yaml

- repo: https://github.com/psf/black
rev: 23.1.0
rev: 23.3.0
hooks:
- id: black

Expand All @@ -30,15 +30,15 @@ repos:
additional_dependencies: [flake8-isort]

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.1.1
rev: v1.4.0
hooks:
- id: mypy
args: ["--config=setup.cfg"]
args: ["--config=pyproject.toml"]
additional_dependencies:
[
dask==2023.3.2,
numpy==1.23.5,
pandas==1.5.3,
xarray==2023.3.0,
types-python-dateutil==2.8.19,
dask,
numpy>=1.23.0,
pandas,
xarray>=2023.2.0,
types-python-dateutil,
]
8 changes: 4 additions & 4 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
version: 2

build:
os: "ubuntu-20.04"
tools:
python: "mambaforge-4.10"
os: "ubuntu-20.04"
tools:
python: "mambaforge-4.10"

# Build documentation in the docs/ directory with Sphinx
sphinx:
Expand All @@ -19,4 +19,4 @@ formats:
- pdf

conda:
environment: conda-env/readthedocs.yml
environment: conda-env/dev.yml
21 changes: 9 additions & 12 deletions conda-env/ci.yml
Original file line number Diff line number Diff line change
@@ -1,28 +1,25 @@
# Conda xcdat CI/CD environment (used in GH Actions)
# Conda xcdat CI/CD environment (used in GH Actions).
name: xcdat_ci
channels:
- conda-forge
- defaults
dependencies:
# ==================
# Base
# ==================
- python >=3.8
- pip
# Base - required for building the package.
# =========================================
- python >=3.9
- cf_xarray
- cftime
- dask
- esmpy
- lxml
- netcdf4
- numpy
- numpy >=1.23.0 # This version of numpy includes support for Python 3.11.
- pandas
- python-dateutil
- xarray
# Constrained because 0.6.3 breaks with import ESMF
# Source: https://github.com/pangeo-data/xESMF/issues/212
- xesmf >0.6.3
- xarray >=2022.02.0 # This version of Xarray drops support for Python 3.9.
- xgcm
# Optional - enables additional features.
# =========================================
- xesmf >=0.7.0 # Constrained because https://github.com/pangeo-data/xESMF/issues/212.
# Quality Assurance
# ==================
- types-python-dateutil
Expand Down
72 changes: 32 additions & 40 deletions conda-env/dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,58 +4,50 @@ channels:
- conda-forge
- defaults
dependencies:
# ==================
# Base
# ==================
# NOTE: If versions are updated, also `additional_dependencies` list for mypy in `.pre-commit-config.yaml`
- python=3.10.10
- pip=23.0.1
- cf_xarray=0.8.0
- cftime=1.6.2
- dask=2023.3.2
- lxml=4.9.2
- netcdf4=1.6.3
- numpy=1.23.5
- pandas=1.5.3
- python-dateutil=2.8.2
- xarray=2023.4.2
- xgcm=0.8.0
# ==================
# Optional
# ==================
- xesmf=0.7.0
- matplotlib-base=3.7.1
# Base - required for building the package.
# =========================================
- python >=3.9
- cf_xarray
- cftime
- dask
- lxml
- netcdf4
- numpy >=1.23.0 # This version of numpy includes support for Python 3.11.
- pandas
- python-dateutil
- xarray >=2022.02.0 # This version of Xarray drops support for Python 3.9.
- xgcm
# Optional - enables additional features.
# =========================================
- xesmf >=0.7.0 # Constrained because https://github.com/pangeo-data/xESMF/issues/212.
- matplotlib-base >=3.7.0
- nc-time-axis=1.4.1
# ==================
# Documentation
# ==================
- sphinx=5.3.0
- sphinx-autosummary-accessors=2022.4.0
- sphinx-book-theme=1.0.1
- sphinx-copybutton=0.5.1
- nbsphinx=0.9.1
- pandoc=3.1.1
- ipython=8.11.0 # Required for nbsphinx syntax highlighting
- sphinx >=5.0.0
- sphinx-autosummary-accessors
- sphinx-book-theme
- sphinx-copybutton
- nbsphinx
- pandoc
- ipython >=8.0.0 # Required for nbsphinx syntax highlighting
- gsw-xarray=0.3.0 # Required for vertical regridding example
# ==================
# Quality Assurance
# ==================
# NOTE: If versions are updated, also update 'rev' in `.pre-commit.config.yaml`
- black=23.1.0
- types-python-dateutil
# NOTE: If the tools below are updated, also update their 'rev' in `.pre-commit.config.yaml`
- black=23.3.0
- flake8=6.0.0
- flake8-isort=6.0.0
- isort=5.12.0
- mypy=1.1.1
- mypy=1.4.0
- pre-commit=3.2.0
- types-python-dateutil=2.8.19
# ==================
# Testing
# ==================
- pytest=7.2.2
- pytest-cov=4.0.0
# ==================
- pytest
- pytest-cov
# Developer Tools
# ==================
- jupyterlab=3.6.2
- tbump=6.9.0
- jupyterlab
- tbump
prefix: /opt/miniconda3/envs/xcdat_dev
41 changes: 0 additions & 41 deletions conda-env/readthedocs.yml

This file was deleted.

57 changes: 28 additions & 29 deletions docs/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,63 +12,62 @@ Prerequisites
- We highly recommend visiting the `xarray tutorial`_ and `xarray documentation`_
pages if you aren't familiar with ``xarray``.

2. xCDAT is distributed through conda, which is available through Anaconda and Miniconda.
2. xCDAT is distributed available on Anaconda. We recommend installing `mamba`_, which
is a drop-in replacement of ``conda``.

We recommend following these steps to install Miniconda:
We recommend following these steps to install Mambaforge (Linux example):

.. code-block:: console

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh

Then follow the instructions for installation. To have conda added to
your path you will need to type ``yes`` in response to ``"Do you wish the
installer to initialize Miniconda3 by running conda init?"`` (we recommend
that you do this). Note that this will modify your shell profile (e.g.,
``~/.bashrc``) to add ``conda`` to your path.
bash ./Mambaforge-Linux-x86_64.sh

Note: After installation completes you may need to type ``bash`` to
restart your shell (if you use bash). Alternatively, you can log out and
log back in.
When you see: ::

by running conda init? [yes|no]
[no] >>> yes

respond with ``yes`` so ``conda`` and ``mamba`` commands are available on
initializing a new bash terminal.

.. _mamba: https://mamba.readthedocs.io/en/latest/index.html
.. _xarray tutorial: https://tutorial.xarray.dev/intro.html
.. _xarray documentation: https://docs.xarray.dev/en/stable/getting-started-guide/index.html

Installation
------------

1. Create a conda environment from scratch with ``xcdat`` (`conda create`_)
1. Create a mamba environment from scratch with ``xcdat`` (`mamba create`_)

We recommend using the Conda environment creation procedure to install ``xcdat``.
The advantage with following this approach is that Conda will attempt to resolve
dependencies (e.g. ``python >= 3.8``) for compatibility.
We recommend using the mamba environment creation procedure to install ``xcdat``.
The advantage with following this approach is that mamba will attempt to resolve
dependencies (e.g. ``python >= 3.9``) for compatibility.

To create an ``xcdat`` conda environment with ``xesmf`` (a recommended dependency),
run:
To create an ``xcdat`` environment with ``xesmf`` (a recommended dependency), run:

.. code-block:: console

>>> conda create -n <ENV_NAME> -c conda-forge xcdat xesmf
>>> conda activate <ENV_NAME>
>>> mamba create -n <ENV_NAME> -c conda-forge xcdat xesmf
>>> mamba activate <ENV_NAME>

Note that ``xesmf`` is an optional dependency, which is required for using ``xesmf``
based horizontal regridding APIs in ``xcdat``. ``xesmf`` is not currently supported
on `osx-arm64`_ or `windows`_ because ``esmpy`` is not yet available on these
platforms. Windows users can try `WSL2`_ as a workaround.
on `windows`_ because ``esmpy`` is not yet available on these platforms. Windows
tomvothecoder marked this conversation as resolved.
Show resolved Hide resolved
users can try `WSL2`_ as a workaround.

.. _windows: https://github.com/conda-forge/esmf-feedstock/issues/64
.. _osx-arm64: https://github.com/conda-forge/esmf-feedstock/issues/74
.. _WSL2: https://docs.microsoft.com/en-us/windows/wsl/install

2. Install ``xcdat`` in an existing conda environment (`conda install`_)
2. Install ``xcdat`` in an existing mamba environment (`mamba install`_)

You can also install ``xcdat`` in an existing Conda environment, granted that Conda
You can also install ``xcdat`` in an existing mamba environment, granted that mamba
is able to resolve the compatible dependencies.

.. code-block:: console

>>> conda activate <ENV_NAME>
>>> conda install -c conda-forge xcdat xesmf
>>> mamba activate <ENV_NAME>
>>> mamba install -c conda-forge xcdat xesmf

Note: As above, ``xesmf`` is an optional dependency.

Expand All @@ -80,5 +79,5 @@ Installation
- ``matplotlib``: a library for creating visualizations in Python.
- ``cartopy``: an add-on package for ``matplotlib`` and specialized for geospatial data processing.

.. _conda create: https://docs.conda.io/projects/conda/en/latest/commands/create.html?highlight=create
.. _conda install: https://docs.conda.io/projects/conda/en/latest/commands/install.html?highlight=install
.. _mamba create: https://fig.io/manual/mamba/create
.. _mamba install: https://fig.io/manual/mamba/install
25 changes: 25 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,28 @@ exclude = '''
)/
)
'''

[tool.isort]
# Docs: https://pycqa.github.io/isort/docs/configuration/options.html#example-pyprojecttoml_4
multi_line_output = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
line_length = 88

[tool.pytest.ini_options]
# Docs: https://docs.pytest.org/en/7.2.x/reference/customize.html#configuration
junit_family = "xunit2"
addopts = "--cov=xcdat --cov-report term --cov-report html:tests_coverage_reports/htmlcov --cov-report xml:tests_coverage_reports/coverage.xml -s"
python_files = ["tests.py", "test_*.py"]
# These markers are defined in `xarray.tests` and must be included to avoid warnings when importing from this module.
markers = ["flaky", "network"]

[tool.mypy]
# Docs: https://mypy.readthedocs.io/en/stable/config_file.html
python_version = 3.10
check_untyped_defs = true
ignore_missing_imports = true
warn_unused_ignores = true
warn_redundant_casts = true
warn_unused_configs = true
Loading
Loading