Relax categorical coordinates for heatmap?

[jbednar]

Right now, heatmaps treat their coordinates as categorical, which only works well for small sizes because all labels are always shown for categoricals (since the ordering is not necessarily meaningful, unlike a numerical axis). For larger heatmaps, having categorical values generates unreadable axes:

One option is for heatmap to support non-categorical dimensions, so that Bokeh's usual better-performing axis numbering algorithm can be used. Another option is to add hover tool support to images in Bokeh, which would let HeatMap be reserved for the small-number categorical case, while Image could be used for cases like the above. In this case, however, one of the axes is truly categorical, while the other one is numerical, so allowing Heatmaps to have non-categorical dimensions seems like the only general solution.

[jbednar]

Note that the WSJ original appears to be using a numeric year axis, as requested here:

Or maybe just skipping categories, as there seem to be states missing as well!

[TomBugnon]

Any progress or workaround on this?

[NumesSanguis]

I would call this a bug, because hv.help(hv.HeatMap) shows that there is a xticks and yticks, but even if you write xticks=None, it is ignored.

Based on this example (in Jupyter Notebook): # https://stackoverflow.com/questions/38701969/dynamic-spectrum-using-plotly
(vignesh.wav: https://github.com/MTG/essentia-audio/raw/65d8eded3d8145df74625b0c60e3d605c21fa860/recorded/vignesh.wav)

import numpy as np
import holoviews as hv
hv.extension('bokeh')

import numpy as np

from scipy.io import wavfile # scipy library to read wav files
AudioName = "vignesh.wav" # Audio File
fs, Audiodata = wavfile.read(AudioName)
Audiodata = Audiodata / (2.**15) # Normalized between [-1,1]

 #Spectrogram
from scipy import signal
N = 512 #Number of point in the fft
w = signal.blackman(N)
freqs, bins, Pxx = signal.spectrogram(Audiodata, fs, window=w, nfft=N)

heatmap = hv.HeatMap((bins, freqs, 10*np.log10(Pxx)))
heatmap.opts(colorbar=True, width=800, xticks=None)  # , xticks=[0, 1, 2, 3]
heatmap

Gives an insane amount of x and y tick labels:

True, that hv.Image can be used as pointed out here: https://stackoverflow.com/questions/54584670/holoviews-heatmap-x-axis-formatting
However, that doesn't match HoloViews abstraction principles, because Plotly for example does support HeatMap with changing ticks.

The same HeatMap only using Plotly:

# plotly offline
import plotly.offline as pyo
from plotly.offline import init_notebook_mode #to plot in jupyter notebook
import plotly.graph_objs as go
init_notebook_mode() # init plotly in jupyter notebook

# Plot with plotly
trace = [go.Heatmap(
    x= bins,
    y= freqs,
    z= 10*np.log10(Pxx),
    colorscale='Jet',
    )]
layout = go.Layout(
    title = 'Spectrogram with plotly',
    yaxis = dict(title = 'Frequency'), # x-axis label
    xaxis = dict(title = 'Time'), # y-axis label
    )
fig = go.Figure(data=trace, layout=layout)
pyo.iplot(fig, filename='Spectrogram')

[NumesSanguis]

Also found an answer (https://stackoverflow.com/questions/49211006/changing-ticks-mark-period-in-holoviews-heatmap) how to do it in a hacky way (untested):

from bokeh.models import CustomJS
import holoviews as hv
hv.extension("bokeh")

def change_formatter(p, o):
    fig = p.handles["plot"]

    def callback(fig=fig):
        def do_format(t, e):
            return [label if i % 2 == 0 else "" for i, label in enumerate(t)]

        fig.renderers[0].formatter.doFormat = do_format    
    fig.js_on_change("inner_width", CustomJS.from_py_func(callback))

%opts HeatMap [finalize_hooks=[change_formatter]]

data = [(i, j,  i*j) for i in range(10) for j in range(10)]
hv.HeatMap(data)    

[HuangJunye]

This issues seems to unsolved after more than one year. Specifiying xticks does not do anything for both bokeh and plotly backends. The hacky way mention above works on Mac (with warnings) but does not work on Windows (with warnings but no image shown) for some reasons.

BokehDeprecationWarning:
'from_py_func' is deprecated and will be removed in an eventual 2.0 release. 
Use CustomJS directly instead.

Is there a way to make this work using plotly as backend instead of using plotly directly?

[ThetomekK]

It should be solved, espacially for spectrogram plots and these are a fundamental representation of time frequency spaces, not only in audio signal processing.

Is there a way to get a working example with hv.extension('plotly') backend?

[philippjfr]

@ThetomekK You can use both QuadMesh and Image for those kinds of plots. HeatMap is very specifically for categorical axes and while lifting that restriction would be nice, e.g. to have one categorical and one continuous axis, HeatMap is absolutely the wrong element for time/frequency spaces.

[philippjfr]

@NumesSanguis the comment above is also relevant to you, using HeatMap for the plot you posted is definitely the wrong choice of element, you want a QuadMesh.

[jbednar]

Note that Image now does support hover, so part of the original reason for this issue has been addressed (see original comment). However, although an Image or QuadMesh fully addresses the spectrogram case, where both axes are numerical, it doesn't seem like it properly addresses the original example above, where the x axis is numerical and the y axis is categorical. Image and QuadMesh will work well with the original example's x axis, while HeatMap works well with the original example's y axis, but it doesn't seem like we currently have a good solution for having one axis of each type.

[philippjfr]

it doesn't seem like we currently have a good solution for having one axis of each type.

That's exactly right and what I meant by:

lifting that restriction would be nice e.g. to have one categorical and one continuous axis

[philippjfr]

Maybe we should just say that if the values are floats it shouldn't force a categorical axis? To force a categorical axis you could then define Dimension(..., type=str) or something (that's already covered by #2468).

[jbednar]

That sounds ok to me. Integers are a trickier question, and here Year is integer, right? Integers could be categorical, which works fine for small ranges, but for large ranges a numerical axis is more appropriate. Still, numerical seems like the best default for both integer and float axes for a HeatMap, to me.

In any case, it seems like @NumesSanguis and @ThetomekK should be happy already, and I think using Image also addresses @HuangJunye's question about xticks (though maybe there is still the issue of HeatMap ignoring them?).

[philippjfr]

(though maybe there is still the issue of HeatMap ignoring them?).

Yes, that's been a longstanding annoyance I've had with categorical axes, at minimum there should be a way to reduce their frequency.

[HuangJunye]

@philippjfr @jbednar Thanks a lot for the quick responses. Indeed, Image is what I need because both axes in my plot are numerical.

There is one thing I don’t understand. Why does Image require gridded data? It would be better if it can also support columnar data such as pandas dataframe. Currently I am using a workaround by converting dataframe to xarray
#3686