Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed dask.optimize on datasets #4438

Merged
merged 4 commits into from Sep 20, 2020
Merged

Fixed dask.optimize on datasets #4438

merged 4 commits into from Sep 20, 2020

Conversation

TomAugspurger
Copy link
Contributor

@TomAugspurger TomAugspurger commented Sep 18, 2020 •
edited by dcherian

Another attempt to fix #3698. The issue with my fix in is that we hit
Variable._dask_finalize in both dask.optimize and dask.persist. We
want to do the culling of unnecessary tasks (test_persist_Dataset) but
only in the persist case, not optimize (test_optimize).

@TomAugspurger
Fixed dask.optimize on datasets
39bb240 
Another attempt to fix pydata#3698. The issue with my fix in is that we hit
`Variable._dask_finalize` in both `dask.optimize` and `dask.persist`. We
want to do the culling of unnecessary tasks (`test_persist_Dataset`) but
only in the persist case, not optimize (`test_optimize`).
@dcherian
Copy link
Contributor

The RTD build failure is unrelated and may result from these warnings?

waiting for workers...
Matplotlib is building the font cache; this may take a moment.
/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/4438/lib/python3.8/site-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
  warnings.warn('Downloading: {}'.format(url), DownloadWarning)
/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/4438/lib/python3.8/site-packages/cartopy/io/__init__.py:260: DownloadWarning: Downloading: https://naciscdn.org/naturalearth/50m/physical/ne_50m_coastline.zip
  warnings.warn('Downloading: {}'.format(url), DownloadWarning)
/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/4438/lib/python3.8/site-packages/cartopy/mpl/gridliner.py:741: UserWarning: The labels of this gridliner do not belong to the gridliner axes
  warnings.warn('The labels of this gridliner do not belong to '
/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/4438/doc/whats-new.rst:92: WARNING: Bullet list ends without a blank line; unexpected unindent.
/home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/4438/doc/whats-new.rst:93: WARNING: Bullet list ends without a blank line; unexpected unindent.
looking for now-outdated files... none found
pickling environment... done

@keewis
Copy link
Collaborator

keewis commented Sep 19, 2020 •
edited

these warnings should be fixed by #4435

Edit: let's merge this before #4435 because ideally, that PR should be the last commit before the release.

@max-sixty
Copy link
Collaborator

I think it's OK to merge #4435 first and then this. I'll do another PR if we need any more touch-ups. That way we can see if RTD is fixed there...

max-sixty
max-sixty reviewed Sep 20, 2020
View reviewed changes
doc/whats-new.rst Outdated Show resolved Hide resolved
@max-sixty max-sixty merged commit 13c09dc into pydata:master Sep 20, 2020
dcherian added a commit to dcherian/xarray that referenced this pull request Oct 9, 2020
@dcherian
Merge branch 'propagate-attrs' of github.com:dcherian/xarray into pro…
1f998f0 
…pagate-attrs

* 'propagate-attrs' of github.com:dcherian/xarray: (22 commits)
  silence sphinx warnings about broken rst (pydata#4448)
  Xarray open_mfdataset with engine Zarr (pydata#4187)
  Fix release notes formatting (pydata#4443)
  fix typo in io.rst (pydata#4250)
  Fix typo (pydata#4181)
  Fix release notes typo
  New whatsnew section
  Add notes re doctests (pydata#4440)
  Fixed dask.optimize on datasets (pydata#4438)
  Release notes for 0.16.1 (pydata#4435)
  Small updates to How-to-release + lint (pydata#4436)
  Fix doctests (pydata#4439)
  add a ci for doctests (pydata#4437)
  preserve original dimension, coordinate and variable order in ``concat`` (pydata#4419)
  Fix for h5py deepcopy issues (pydata#4426)
  Keep the original ordering of the coordinates (pydata#4409)
  Clearer Vectorized Indexing example (pydata#4433)
  Revert "Fix optimize for chunked DataArray (pydata#4432)" (pydata#4434)
  Fix optimize for chunked DataArray (pydata#4432)
  fix doc dataarray to netcdf (pydata#4424)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@max-sixty max-sixty max-sixty left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

dask.optimize on xarray objects
4 participants
@TomAugspurger @dcherian @keewis @max-sixty