Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
cff-version: 1.2.0
message: "If you use this cookbook, please cite it as below."
authors:
# add additional entries for each author -- see https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md
- family-names: Sobhani
given-names: Negin
website: https://github.com/negin513
- family-names: Brian
given-names: Vanderwende
- family-names: Cherian
given-names: Deepak
website: https://github.com/dcherian
- family-names: Kirk
given-names: Ben
- name: "Dask Cookbook contributors" # use the 'name' field to acknowledge organizations
website: "https://github.com/ProjectPythia/dask-cookbook/graphs/contributors"
title: "Dask Cookbook"
abstract: "A cookbook for Dask workflows."
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
<img src="notebooks/images/NCAR_CISL_NSF_banner.jpeg" alt="NCAR CISL logo" />

# NCAR Dask Tutorial
# NCAR Dask Tutorial

[![nightly-build](https://github.com/ProjectPythia/cookbook-template/actions/workflows/nightly-build.yaml/badge.svg)](https://github.com/ProjectPythia/cookbook-template/actions/workflows/nightly-build.yaml)
[![Binder](https://mybinder.org/badge_logo.svg)](http://binder.projectpythia.org/v2/gh/ProjectPythia/dask-cookbook/main?labpath=notebooks)
[![DOI](https://zenodo.org/badge/610934658.svg)](https://zenodo.org/badge/latestdoi/610934658)

This Project Pythia Cookbook provides a comprehensive guide to understanding the basic concepts and collections of Dask as well as its integration with Xarray.
Dask is a parallel computing library that allows you to scale your computations to multiple cores or even clusters, while Xarray is a library that enables working with labelled multi-dimensional arrays, with a focus on working with netCDF datasets.
Expand All @@ -26,6 +27,7 @@ The motivation behind this repository is to provide a clear and concise resource
</a>

## Note on Content Origin

This cookbook is part of the extensive material used in our NCAR tutorial, ["Using Dask on HPC systems"](https://github.com/NCAR/dask-tutorial.git), which was held in February 2023. The complete tutorial series also includes an in-depth exploration and practical use cases of Dask on HPC systems and best practices for Dask on HPC . For the complete set of materials, including these additional insights on Dask on HPC, please refer to the main tutorial content available [here](https://ncar.github.io/dask-tutorial/README.html).

## Structure
Expand Down Expand Up @@ -87,5 +89,6 @@ If you are interested in running this material locally on your computer, you wil
```

### Acknowledgments
* NCAR CISL/CSG Team
* ESDS Initiative

- NCAR CISL/CSG Team
- ESDS Initiative
11 changes: 1 addition & 10 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,8 @@
title: Dask Cookbook
author: Negin Sobhani, Brian Vanderwende, Deepak Cherian, and Ben Kirk
logo: notebooks/images/logos/pythia_logo-white-rtext.svg
email: negins@ucar.edu
copyright: "2023"

description: A cookbook for Dask workflows.
thumbnail: thumbnail.png
tags:
packages:
- xarray
- dask

execute:
# To execute notebooks via a binder instead, replace 'cache' with 'binder'
execute_notebooks: cache
Expand All @@ -35,9 +27,8 @@ parse:

sphinx:
config:

html_favicon: notebooks/images/icons/favicon.ico
html_last_updated_fmt: '%-d %B %Y'
html_last_updated_fmt: "%-d %B %Y"
html_theme: sphinx_pythia_theme
html_permalinks_icon: '<i class="fas fa-link"></i>'
html_theme_options:
Expand Down
5 changes: 5 additions & 0 deletions _gallery_info.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
thumbnail: thumbnail.png
tags:
packages:
- xarray
- dask
21 changes: 12 additions & 9 deletions _toc.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,17 @@
format: jb-book
root: README
parts:
- caption: Introductions to Dask
chapters:
- file: notebooks/00-dask-overview.ipynb
- file: notebooks/01-dask-array.ipynb
- file: notebooks/02-dask-dataframe.ipynb
- file: notebooks/04-dask-cluster.ipynb
- caption: Preamble
chapters:
- file: notebooks/how-to-cite

- caption: Dask and Xarray
chapters:
- file: notebooks/03-dask-xarray.ipynb
- caption: Introductions to Dask
chapters:
- file: notebooks/00-dask-overview.ipynb
- file: notebooks/01-dask-array.ipynb
- file: notebooks/02-dask-dataframe.ipynb
- file: notebooks/04-dask-cluster.ipynb

- caption: Dask and Xarray
chapters:
- file: notebooks/03-dask-xarray.ipynb
7 changes: 7 additions & 0 deletions notebooks/how-to-cite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# How to Cite This Cookbook

The material in Project Pythia's Dask Cookbook is licensed for free and open consumption and reuse. All code is served under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), while all non-code content is licensed under [Creative Commons BY 4.0 (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/). Effectively, this means you are free to share and adapt this material so long as you give appropriate credit to the Cookbook authors and the Project Pythia community.

The source code for the book is [released on GitHub](https://github.com/ProjectPythia/dask-cookbook) and archived on Zenodo. This DOI will always resolve to the latest release of the book source:

[![DOI](https://zenodo.org/badge/610934658.svg)](https://zenodo.org/badge/latestdoi/610934658)