v1 release (#107)

* Scripts for table 1 * Fixed typos in table1.py * Temp fix * iter module working with new refactor * Checkpoint Tried to get ShardDataset working. Switching to IterDataset. * Update table 1 code * Adding table experiments for other architectures * Update ViT architecture * Add notebook * Checkpoint * Checkpointing * Updated log dir for checkpoints * Checkpoint * Fix mse, rmse, and acc metrics * Update acc code * Reduce precision, change early stopping metric * Fixes * add lr monitor, modify checkpointing, use ddp instead of ddp_spawn * modify logging keys to separate train/val/test * use normalized lat_mse as a validation metric * change training setting * Checkpoint * Downscaling * DL models for downscaling * Pearson now does not rely on scipy * ViT experiment and PRISM support * Multiple transformation bug fix * fix duplicate denormalization bug * add ReduceLROnPlateau scheduler * update gitignore * fix resnet to match with weatherbench implementation * current best setting * update projection and prediction layers for unet * update run scripts for resnet and unet * avoid spiking loss * update vit models and scripts * Downscaling exp changes * Downscaling changes * rollback resnet and unet architectures * Merge Tung's changes * Downscaling updates * use 16 precision * CMIP6 running with iterdatamodule * add continuous data modules * update nc2npz to accumulate precipitation data * merge tung exps (#105) * avoid spiking loss * update vit models and scripts * rollback resnet and unet architectures * use 16 precision * add continuous data modules * update nc2npz to accumulate precipitation data --------- Co-authored-by: Tung Nguyen <ductungnguyen1997@gmail.com> * ERA5 downscaling updates * use 1 worker, somehone increasing # workers affects performance * update scripts to be consistent * prepare iterative exps * Downscaling checkpoint: PRISM implemented * evaluate iterative models * summarize results * update json results and add plot scripts * Add probabilistic metrics * Add class metrics * Add rank histogram visualization * update plot script * add climatebench experiments * Disable all data tests * Formatting * Fix trainer * Cleanup files * Moved processing scripts, added PRISM processing * Cleanup data downloading * Add CLI utility for downloads, processing scripts * Combine iter modules * Add data modules * Reorganize files * Remove unused notebooks * Cleaning utilities * Add visualizations for bias and mean bias * Add rank histogram visualization * Organize experiments, add ERA5 crop script * Clean up downscaling and climate projection * Add forecasting baselines * Cleanup forecasting experiments * Removed unneeded tables * Add climate projection experiments * Iterative forecasting experiments * Iterative forecasting experiments * Add extreme ERA5 * Changes for quickstart notebook * Update notebooks * Formatting * Update era5_constants.py Add units for variables * Update cmip6_constants.py Add units for variables. * Add units for visualizations * Updates for notebook * Update docs and notebook * Update docs * Finish tasks and datasets * Finish docs * Update configs * Fix pyproject.toml * Remove rasterio version for rtd * Change rtd python version * Add OS * Add images * v1 release --------- Co-authored-by: Prakhar Sharma <prakhar6sharma@gmail.com> Co-authored-by: tung-nd <ductungnguyen1997@gmail.com> Co-authored-by: Prakhar Sharma <37648724+prakhar6sharma@users.noreply.github.com>
aditya-grover · Jul 6, 2023 · 9afe308 · 9afe308
1 parent 95dc85c
commit 9afe308
Show file tree

Hide file tree

Showing 101 changed files with 9,005 additions and 2,919 deletions.
diff --git a/.gitignore b/.gitignore
@@ -161,3 +161,4 @@ cython_debug/
 
 # Ignore checkpoint files
 *.ckpt
+checkpoints/
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -5,6 +5,12 @@
 # Required
 version: 2
 
+# Set the version of Python
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.8"
+
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
   configuration: docs/source/conf.py

diff --git a/CITATION.cff b/CITATION.cff
@@ -2,39 +2,27 @@
 # Visit https://bit.ly/cffinit to generate yours today!
 
 cff-version: 1.2.0
-title: "ClimateLearn: Benchmarking Machine Learning for Data-driven Climate Science"
+title: "ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling"
 message: >-
   If you use this software, please cite it using the
   metadata from this file.
 type: software
 authors:
-  - given-names: Hritik
-    family-names: Bansal
-    email: hbansal@g.ucla.edu
-    affiliation: 'University of California, Los Angeles'
-  - given-names: Shashank
-    family-names: Goel
-    email: shashankgoel@g.ucla.edu
+  - given-names: Tung
+    family-names: Nguyen
+    email: tungnd@cs.ucla.edu
     affiliation: 'University of California, Los Angeles'
   - given-names: Jason
     family-names: Jewik
-    email: jason.jewik@cs.ucla.edu
-    affiliation: 'University of California, Los Angeles'
-  - given-names: Siddharth
-    family-names: Nandy
-    email: sidd.nandy@gmail.com
+    email: jason.jewik@ucla.edu
     affiliation: 'University of California, Los Angeles'
-  - given-names: Tung
-    family-names: Nguyen
-    email: tungnd@g.ucla.edu
-    affiliation: 'University of California, Los Angeles'
-  - given-names: Seongbin
-    family-names: Park
-    email: shannonsbpark@gmail.com
+  - given-names: Hritik
+    family-names: Bansal
+    email: hbansal@ucla.edu
     affiliation: 'University of California, Los Angeles'
-  - given-names: Jingchen
-    family-names: Tang
-    email: tangtang1228@ucla.edu
+  - given-names: Prakhar
+    family-names: Sharma
+    email: prakhar6sharma@gmail.com
     affiliation: 'University of California, Los Angeles'
   - given-names: Aditya
     family-names: Grover

diff --git a/LICENSE b/LICENSE
@@ -1,21 +1,21 @@
-# MIT License
-#
-#@title Copyright (c) 2021 CCAI Community Authors { display-mode: "form" }
-#
-# Permission is hereby granted, free of charge, to any person obtaining a
-# copy of this software and associated documentation files (the "Software"),
-# to deal in the Software without restriction, including without limitation
-# the rights to use, copy, modify, merge, publish, distribute, sublicense,
-# and/or sell copies of the Software, and to permit persons to whom the
-# Software is furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-# DEALINGS IN THE SOFTWARE.
+MIT License
+
+Copyright (c) 2021-present Machine Intelligence Group at UCLA
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/README.md b/README.md
@@ -3,33 +3,26 @@
 [![Documentation Status](https://readthedocs.org/projects/climatelearn/badge/?version=latest)](https://climatelearn.readthedocs.io/en/latest/?badge=latest)
 [![CI Build Status](https://github.com/aditya-grover/climate-learn/actions/workflows/ci.yaml/badge.svg)](https://github.com/aditya-grover/climate-learn/actions/workflows/ci.yaml)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
-[![Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WiNEK1BHsiGzo_bT9Fcm8lea2H_ghNfa)
+[![Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1LcecQLgLtwaHOwbvJAxw9UjCxfM0RMrX?usp=sharing)
 
 **ClimateLearn** is a Python library for accessing state-of-the-art climate data and machine learning models in a standardized, straightforward way. This library provides access to multiple datasets, a zoo of baseline approaches, and a suite of metrics and visualizations for large-scale benchmarking of statistical downscaling and temporal forecasting methods. For further context on our past motivation and future plans, check out our announcement [blog post](https://aditya-grover.github.io/blog/2023/climate-learn/).
 
 ## Usage
 
-[**Python3**](https://www.python.org/) is required.
+[**Python 3.8+**](https://www.python.org/) is required. The xESMF package has to be installed separately since one of its dependencies, ESMpy, is available only through Conda.
 ```
+conda install -c conda-forge xesmf
 pip install climate-learn
 ```
 
 ### Quickstart
-We have a series of tutorial Jupyter notebooks in the `notebooks` folder. We recommend reading them in the following order to see a typical ClimateLearn workflow.
-1. Data Processing
-2. Model Training & Evaluation
-3. Visualization
-
-To run the notebooks, please upload them to [Google Colab](https://colab.research.google.com/). 
+We have a quickstart notebook in the `notebooks` folder titled `Quickstart.ipynb`. It is intended for use in Google Colab and can be launched by clicking the Google Colab badge above or this link: https://colab.research.google.com/drive/1LcecQLgLtwaHOwbvJAxw9UjCxfM0RMrX?usp=sharing.
 
 We also previewed some key features of ClimateLearn at a spotlight tutorial in the "Tackling Climate Change with Machine Learning" Workshop at the Neural Information Processing Systems 2022 Conference. The slides and recorded talk can be found on [Climate Change AI's website](https://www.climatechange.ai/papers/neurips2022/114).
 
 ### Documentation
 Find us on [ReadTheDocs](https://climatelearn.readthedocs.io/).
 
-### Integrations
-- [Weights & Biases](https://wandb.ai/site)
-
 ## About Us
 ClimateLearn is managed by the Machine Intelligence Group at UCLA, headed by [Professor Aditya Grover](https://aditya-grover.github.io).
 

diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -10,20 +10,17 @@
 # -- Project information
 
 project = "ClimateLearn"
-copyright = "2022; Bansal, Goel, Jewik, Nandy, Nguyen, Park, Tang, Grover"
-author = """
-    Hritik Bansal,
-    Shashank Goel,
-    Jason Jewik,
-    Siddharth Nandy,
+copyright = "2023; Nguyen, Jewik, Bansal, Sharma, Grover"
+author = """   
     Tung Nguyen,
-    Seongbin Park,
-    Jingchen Tang,
+    Jason Jewik,
+    Hritik Bansal,
+    Prakhar Sharma,
     Aditya Grover
 """
 
-release = "0.1"
-version = "0.1.0"
+release = "1.0.0"
+version = "1.0.0"
 
 # -- General configuration
 

diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -12,36 +12,56 @@ statistical downscaling and temporal forecasting methods.
 
 .. note::
 
-   This project is under active development.
-
-About Us
---------
-ClimateLearn is managed by the Machine Intelligence Group at UCLA, headed by
-`Professor Aditya Grover <https://aditya-grover.github.io/>`_.
+   This project is under active development. The API might undergo extensive
+   changes in the near future.
 
 Getting Started
 ---------------
-Please see the `quickstart section <https://github.com/aditya-grover/climate-learn#quickstart>`_ of our GitHub repository.
+`Python 3.8+ <https://www.python.org/>`_ is required. The xESMF package has
+to be installed separately since one of its dependencies, ESMpy, is available
+only through Conda.
+
+.. code-block:: shell
+
+   conda install -c conda-forge xesmf
+   pip install climate-learn
+
+We have a quickstart notebook in the ``notebooks`` folder titled
+``Quickstart.ipynb`` that walks through an example usage of ClimateLearn for
+weather forecasting from downloading the data through visualizing the
+predictions of a trained model. It is intended for use in Google Colab and can
+be launched by clicking
+`this link <https://colab.research.google.com/drive/1LcecQLgLtwaHOwbvJAxw9UjCxfM0RMrX?usp=sharing>`_.
 
 .. toctree::
    :caption: User Guide
    :maxdepth: 2
 
-   user-guide/datasets
+   user-guide/tasks_and_datasets
    user-guide/models
    user-guide/metrics
    user-guide/visualizations
 
-.. toctree::
-   :caption: API Reference
-   :maxdepth: 1
-   :glob:
-
-   reference/*
-
 .. toctree::
    :caption: Development Guide
    :maxdepth: 1
 
    development-guide/for-developers
-   development-guide/for-maintainers
+   development-guide/for-maintainers
+
+Why did we build ClimateLearn?
+------------------------------
+
+In recent years, there has been a growing interest in the application of
+ML-based methods for weather and climate modeling. While there are some
+leaderboard benchmarks, such as WeatherBench, ClimateBench, and FloodNet, that
+propose datasets and baselines for specific tasks in climate science, a
+holistic software ecosystem that encompasses the entire data, modeling, and
+evaluation pipeline across several tasks is lacking. Hence, we built
+ClimateLearn to standardize datasets, model implementations, and evaluation
+protocols for rigorous and reproducible data-driven climate science.
+
+About Us
+--------
+ClimateLearn is built and maintained by the Machine Intelligence Group at UCLA,
+headed by `Professor Aditya Grover <https://aditya-grover.github.io/>`_.
diff --git a/docs/source/reference/data.rst b/docs/source/reference/data.rst
diff --git a/docs/source/reference/models.rst b/docs/source/reference/models.rst
diff --git a/docs/source/reference/training.rst b/docs/source/reference/training.rst
diff --git a/docs/source/reference/utils.rst b/docs/source/reference/utils.rst
diff --git a/docs/source/user-guide/datasets.rst b/docs/source/user-guide/datasets.rst
diff --git a/docs/source/user-guide/images/animated_input.png b/docs/source/user-guide/images/animated_input.png
diff --git a/docs/source/user-guide/images/animated_input.png:Zone.Identifier b/docs/source/user-guide/images/animated_input.png:Zone.Identifier
@@ -0,0 +1,3 @@
+[ZoneTransfer]
+ZoneId=3
+HostUrl=about:internet
diff --git a/docs/source/user-guide/images/bias_at_index.png b/docs/source/user-guide/images/bias_at_index.png
diff --git a/docs/source/user-guide/images/bias_at_index.png:Zone.Identifier b/docs/source/user-guide/images/bias_at_index.png:Zone.Identifier
@@ -0,0 +1,3 @@
+[ZoneTransfer]
+ZoneId=3
+HostUrl=about:internet
diff --git a/docs/source/user-guide/images/gt_at_index.png b/docs/source/user-guide/images/gt_at_index.png
diff --git a/docs/source/user-guide/images/gt_at_index.png:Zone.Identifier b/docs/source/user-guide/images/gt_at_index.png:Zone.Identifier
@@ -0,0 +1,3 @@
+[ZoneTransfer]
+ZoneId=3
+HostUrl=about:internet
diff --git a/docs/source/user-guide/images/mean_bias.png b/docs/source/user-guide/images/mean_bias.png
diff --git a/docs/source/user-guide/images/mean_bias.png:Zone.Identifier b/docs/source/user-guide/images/mean_bias.png:Zone.Identifier
@@ -0,0 +1,3 @@
+[ZoneTransfer]
+ZoneId=3
+HostUrl=about:internet
diff --git a/docs/source/user-guide/images/prediction_at_index.png b/docs/source/user-guide/images/prediction_at_index.png
diff --git a/docs/source/user-guide/images/visualize.png b/docs/source/user-guide/images/visualize.png
diff --git a/docs/source/user-guide/images/visualize_at_index.png:Zone.Identifier b/docs/source/user-guide/images/visualize_at_index.png:Zone.Identifier
@@ -0,0 +1,3 @@
+[ZoneTransfer]
+ZoneId=3
+HostUrl=about:internet
diff --git a/docs/source/user-guide/images/visualize_mean_bias.png b/docs/source/user-guide/images/visualize_mean_bias.png