Log axis when plotting holoviews object through datashader

[papahabla]

With the following code I'm trying to maintain the log scale of the x-axis which seems to work when using the pure holoview plot (top plot), but not when plotting the holoview.NdOverlay object through the datashader (bottom plot).

import numpy as np
import holoviews as hv
import datashader as ds
from holoviews.operation.datashader import datashade, dynspread
hv.extension('bokeh')

%%opts NdOverlay  [height=256 width=1024 show_grid=True logx=True]
%%opts RGB [height=256 width=1024 show_grid=True logx=True]

frequencies = [0.5, 0.75, 1.0, 1.25]
def sine_curve(phase, freq):
    xvals = [0.1*i+.0001 for i in range(100)]
    return hv.Curve((xvals, [np.sin(phase+freq*x) for x in xvals]))
overlay = hv.NdOverlay({f: sine_curve(0, f) for f in frequencies}, kdims='channel_keys')
curve_plot = dynspread(datashade(overlay, aggregator=ds.count_cat('channel_keys')))
(overlay + curve_plot).cols(1)

[jlstevens]

Thanks for reporting this!

I suspect datashader might be lacking an option to apply a log transform before rasterizing -
though I might be wrong!

I am assuming that the example you post is a simplified example, as I don't see why you would use datashader for a small number of simple curves like that...

[papahabla]

@jlstevens, yes, this example is just a simple modification of an example in the user guide. For actual application, I would like to plot many simultaneous high resolution FFT transforms of audio signals. Currently using bokeh, but it's very slow plotting so I'm looking for an alternative. So far, holoviews+datashader+bokeh is amazingly fast with time domain signals (not needing log axis).

[jbednar]

Datashader does have an option for doing the log transformation before rasterizing; it's datashader.Canvas(..., x_axis_type='log'). I don't know if that's been exposed in HoloViews, but it might also be possible for it to be broken. I'll have to look into that.

[jbednar]

Looks like it isn't broken in datashader itself:

def sine_curve_df(phase,freq):
    c = sine_curve(phase,freq).data
    return pd.DataFrame(dict(x=c[:,0],y=c[:,1]))

opts = dict(x_range=(0.8,math.pi*2), plot_height=200, plot_width=800)
tf.Images(tf.shade(ds.Canvas(x_axis_type='linear', **opts).line(sine_curve_df(0, 1.25),x='x',y='y')),
          tf.shade(ds.Canvas(x_axis_type='log',    **opts).line(sine_curve_df(0, 1.25),x='x',y='y'))).cols(1)

However, I don't see anywhere in HoloViews that the value of logx or logy is passed down to datashader, so it looks like this was an oversight in the HoloViews code for datashading.

[jbednar]

I've discussed this issue with @philippjfr and @jlstevens, and unfortunately it will take some work to fix it.

There are a couple of underlying issues:

1. The datashader operation does not currently have access to plot options like logx,logy. In some cases passing such options down into the operation would be possible, but it would be very tricky in practice, because e.g. any options provided as Jupyter cell magics take effect after the operation completes. One could feed in the options via a stream that grabs values from the plot as it is displayed, but it would take a good bit of thought and will be awkward, probably requiring a redraw once the plot is available. A more practical approach is to provide explicit logx,logy options to the operation, and then validate that they match the plot options. That way you'd get an error with the code above, but could make it work by adding ", logx=True" when calling the operation.
2. The current code for Rasters and RGBs assumes that the data is linearly and regularly spaced, and it doesn't check that this spacing matches the coordinates provided by xarray (Image should warn about unevenly spaced dimensions #1869). We are about to add validation that those values are regularly sampled (GridInterface bin support for Histogram and QuadMesh #2160), but that will fail for log-spaced sampling. So, we'll need to extend the xarray coords validation to validate against either linear or log spacing as appropriate.

To-do items:

• 1. Make datashader.shade() preserve the xarray coordinates created by the aggregation functions; right now I believe that information is discarded.
• 2. After 1., make slicing of xarray-based Image and RGB elements respect xarray's x and y coordinates, not our computed values based on assumptions of equal sampling (Image should warn about unevenly spaced dimensions #1869).
• 3. Before merging GridInterface bin support for Histogram and QuadMesh #2160, extend it to allow validating either log or linear axes on Image and RGB types. For both axis types the values must be regularly sampled, just regular along a different axis type.
• 4. Add code to check that aggregate(logx,logy) matches the plot options logx,logy, at the first point those two sources of information both become available. I.e. if any plot contains a datashaded object with log_=True, the plot options must specify a log axis, and vice versa, or an error or scary warning should be raised.
• 5. Add support for slicing xarray-based RGB and Image types, respecting the coordinates stored by xarray and not trying to compute slicing as a fraction of the overall array (which assumes and requires linear spacing, right now).
• 6. Could support multiple log bases some day, as matplotlib does, but that's not relevant here because datashader only currently supports base 10.
• 7. Someday, could add an RGB QuadMesh type, supporting display of truly irregularly sampled RGB values, but that's not what's needed here; datashader only generates regular sampling, it's just regular on either a linear or a log axis. Irregular sampling could be useful for other purposes, but is an orthogonal consideration.

In any case we have to do something, even if it's just to raise an error in the above case, because generating an incorrect plot is not ever ok.

[jetilton]

I think this is the same issue so I will post it here, but %%opts Curve[shared_axes=False]{+axiswise +framewise} does not work with datashader either.

Try

%%opts Curve[shared_axes=False]{+axiswise +framewise}
import pandas as pd
x = [1,2,3,4]
data = pd.DataFrame({'x':x, 'y':[i**2 for i in x]})
data['z'] = data['y'] *1000
c1 = hv.Curve((data['x'], data['y']))
c2 = hv.Curve((data['x']*1000, data['z']))
c3 = datashade(hv.Curve((data['x'], data['y'])))
c4 = datashade(hv.Curve((data['x']*1000, data['z'])))
(c1+c2+c3+c4).cols(2)

[philippjfr]

@jetilton datashade turns the Curve elements into RGB elements, try this:

%%opts Curve RGB [shared_axes=False] {+axiswise +framewise}

[jetilton]

@philippjfr That does it, just getting going in holoviews thanks for the quick feedback

[wmayner]

Just got bitten by this. Maybe in the interim it would make sense to throw a warning until a proper fix can be implemented? Right now, it fails silently and could mislead users into thinking they're looking at the log scaled plot when they actually aren't.

[jbednar]

Yes, I agree. I don't quite know how to detect this case cleanly, but if it's obvious to anyone else, I'd be very happy to see a PR appear that warns or even raises an exception (given how unlikely it is that someone would find the current behavior useful or appropriate).