Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved rasterize API #2465

Merged
merged 6 commits into from Mar 21, 2018
Merged

Improved rasterize API #2465

merged 6 commits into from Mar 21, 2018

Conversation

@philippjfr
Copy link
Member

@philippjfr philippjfr commented Mar 20, 2018

This PR ensures that the rasterize and datashade operations correctly apply datashading operations to all elements that support it, specifically regridding of Image types can now also be invoked in this way. Secondly this PR improves handling for the aggregator functions, which may now be defined as strings or ds.reductions.Reduction types.

Adding documentation is a separate task which @jbednar volunteered for, specifically these two issues should be addressed by documentation updates:

#1909
#2158

The rasterize operation works by dispatching each Element type to a specific underlying operation with a specific default aggregator:

  • Curve/Scatter/Points/Path: aggregate (count)

  • Image/RGB/HSV: regrid (mean)

  • TriMesh: without vdims aggregate (count), with vdims trimesh_rasterize (mean)

  • QuadMesh: quadmesh_rasterize (mean)

  • Addresses #2461

  • Add tests

@jbednar
Copy link
Member

@jbednar jbednar commented Mar 20, 2018

Looks great! I'm slightly confused about why regrid still has so much code in it. I grudgingly can see why it still exists, in that regrid involves interpolation and the others don't, but I don't see why it has a lot of very specific code that does more than that. Maybe some explanation of that would help.

@philippjfr
Copy link
Member Author

@philippjfr philippjfr commented Mar 20, 2018

Sure, I'll add that. It's almost exclusively datetime handling and converting non-xarray data to xarray.

@philippjfr philippjfr force-pushed the rasterize_api branch 5 times, most recently from 84ce627 to 16f6c3e Mar 20, 2018
@philippjfr philippjfr force-pushed the rasterize_api branch from 16f6c3e to 7991077 Mar 21, 2018
@philippjfr
Copy link
Member Author

@philippjfr philippjfr commented Mar 21, 2018

Ready to review and merge.

@philippjfr
Copy link
Member Author

@philippjfr philippjfr commented Mar 21, 2018

Topics to cover in documentation:

  • Explain rasterize and datashade operation being extra clear about 'eq_hist', I think it's currently not clear enough that shade does histogram normalization, leading people to draw false conclusions.
  • Explain what rasterization does for Image types, including expand and upsample options which stop the regridded Image from expanding beyond bounds of original data, and disable upsampling by default, respectively
  • Explain what precompute does and when to use it, i.e. it collapses the data into a monolithic datastructure which is faster to aggregate, if plotting a single element containing pandas/dask dataframe it's probably not worth it. When rasterizing images precompute does nothing.
@jlstevens
Copy link
Contributor

@jlstevens jlstevens commented Mar 21, 2018

Looks good to me though you might want to consider moving ResamplingOperation out of this file as it is more general than just datashader.

Merging.

@jlstevens jlstevens merged commit 8f13372 into master Mar 21, 2018
4 checks passed
4 checks passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage decreased (-0.02%) to 82.455%
Details
@philippjfr
s3-reference-data-cache Test data is cached.
Details
jlstevens added a commit that referenced this pull request Mar 21, 2018
@philippjfr philippjfr deleted the rasterize_api branch Mar 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants