New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved rasterize API #2465

Merged
merged 6 commits into from Mar 21, 2018

Conversation

Projects
None yet
3 participants
@philippjfr
Member

philippjfr commented Mar 20, 2018

This PR ensures that the rasterize and datashade operations correctly apply datashading operations to all elements that support it, specifically regridding of Image types can now also be invoked in this way. Secondly this PR improves handling for the aggregator functions, which may now be defined as strings or ds.reductions.Reduction types.

Adding documentation is a separate task which @jbednar volunteered for, specifically these two issues should be addressed by documentation updates:

#1909
#2158

The rasterize operation works by dispatching each Element type to a specific underlying operation with a specific default aggregator:

  • Curve/Scatter/Points/Path: aggregate (count)

  • Image/RGB/HSV: regrid (mean)

  • TriMesh: without vdims aggregate (count), with vdims trimesh_rasterize (mean)

  • QuadMesh: quadmesh_rasterize (mean)

  • Addresses #2461

  • Add tests

@philippjfr philippjfr added this to the v1.10 milestone Mar 20, 2018

@jbednar

This comment has been minimized.

Member

jbednar commented Mar 20, 2018

Looks great! I'm slightly confused about why regrid still has so much code in it. I grudgingly can see why it still exists, in that regrid involves interpolation and the others don't, but I don't see why it has a lot of very specific code that does more than that. Maybe some explanation of that would help.

@philippjfr

This comment has been minimized.

Member

philippjfr commented Mar 20, 2018

Sure, I'll add that. It's almost exclusively datetime handling and converting non-xarray data to xarray.

@philippjfr

This comment has been minimized.

Member

philippjfr commented Mar 21, 2018

Ready to review and merge.

@philippjfr

This comment has been minimized.

Member

philippjfr commented Mar 21, 2018

Topics to cover in documentation:

  • Explain rasterize and datashade operation being extra clear about 'eq_hist', I think it's currently not clear enough that shade does histogram normalization, leading people to draw false conclusions.
  • Explain what rasterization does for Image types, including expand and upsample options which stop the regridded Image from expanding beyond bounds of original data, and disable upsampling by default, respectively
  • Explain what precompute does and when to use it, i.e. it collapses the data into a monolithic datastructure which is faster to aggregate, if plotting a single element containing pandas/dask dataframe it's probably not worth it. When rasterizing images precompute does nothing.
@jlstevens

This comment has been minimized.

Member

jlstevens commented Mar 21, 2018

Looks good to me though you might want to consider moving ResamplingOperation out of this file as it is more general than just datashader.

Merging.

@jlstevens jlstevens merged commit 8f13372 into master Mar 21, 2018

4 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage decreased (-0.02%) to 82.455%
Details
s3-reference-data-cache Test data is cached.
Details

jlstevens added a commit that referenced this pull request Mar 21, 2018

@philippjfr philippjfr deleted the rasterize_api branch Mar 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment