Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of ImageStack #5751

Merged
merged 44 commits into from Sep 14, 2023
Merged

Implementation of ImageStack #5751

merged 44 commits into from Sep 14, 2023

Conversation

Hoxbro
Copy link
Member

@Hoxbro Hoxbro commented Jun 9, 2023

WIP

import holoviews as hv
import numpy as np
import panel as pn

hv.extension("bokeh")


def categorical_data():
    n = 20
    samples = 1000
    rng = np.random.default_rng(92478)

    centers = [(0.3, 0.3), (0.7, 0.5), (0.3, 0.7)]
    radii = [0.2, 0.3, 0.15]
    ncats = len(radii)

    dx = 1.0 / n
    data = np.zeros((n, n, ncats), dtype=np.float32)
    data[:] = np.nan

    for k in range(ncats):
        x = rng.normal(centers[k][0], radii[k], samples)
        y = rng.normal(centers[k][1], radii[k], samples)
        i = (x / dx).astype(int)
        j = (y / dx).astype(int)
        for ii, jj in zip(i, j):
            if 0 <= ii < n and 0 <= jj < n:
                if np.isnan(data[jj, ii, k]):
                    data[jj, ii, k] = 1.0
                else:
                    data[jj, ii, k] += 1.0
    return data


data = categorical_data()

colors = ["red", "blue", "green"]
a = hv.ImageStack(data).opts(
    # colorbar=True,
    width=800,
    height=800,
    cmap=colors,
    num_colors=100,
)

pn.panel(a).servable()

Reference for me:
#4806
bokeh/bokeh#12356

@Hoxbro Hoxbro marked this pull request as draft June 9, 2023 18:00
@codecov-commenter
Copy link

codecov-commenter commented Jun 9, 2023

Codecov Report

Merging #5751 (61c8fb7) into main (2033109) will decrease coverage by 0.08%.
Report is 2 commits behind head on main.
The diff coverage is 94.93%.

@@            Coverage Diff             @@
##             main    #5751      +/-   ##
==========================================
- Coverage   88.35%   88.27%   -0.08%     
==========================================
  Files         310      311       +1     
  Lines       63983    64340     +357     
==========================================
+ Hits        56529    56793     +264     
- Misses       7454     7547      +93     
Flag Coverage Δ
ui-tests 23.36% <11.67%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
holoviews/plotting/mpl/__init__.py 81.48% <ø> (ø)
holoviews/plotting/plotly/__init__.py 93.10% <ø> (ø)
holoviews/plotting/util.py 68.86% <75.00%> (+0.07%) ⬆️
holoviews/element/raster.py 79.54% <85.71%> (+0.57%) ⬆️
holoviews/tests/plotting/bokeh/test_rasterplot.py 94.84% <93.85%> (-5.16%) ⬇️
...views/tests/plotting/matplotlib/test_rasterplot.py 97.36% <94.28%> (-2.64%) ⬇️
holoviews/tests/plotting/plotly/test_imagestack.py 94.44% <94.44%> (ø)
holoviews/core/data/xarray.py 89.65% <100.00%> (+0.23%) ⬆️
holoviews/operation/datashader.py 83.83% <100.00%> (+0.02%) ⬆️
holoviews/plotting/__init__.py 100.00% <100.00%> (ø)
... and 4 more

... and 23 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@philippjfr
Copy link
Member

philippjfr commented Aug 28, 2023

  • shade now has to support ingesting an ImageStack as well as an NdOverlay of Image
  • Add Compositor definitions for matplotlib and plotly to convert an ImageStack -> RGB
  • Finish plot implementation by finishing support for invert_axes, invert_xaxis and inter_yaxis.
  • Add reference notebook for ImageStack
  • Add tests

Nice to have:

  • Automatically add a legend for the categories in the ImageStack

@ahuang11 ahuang11 self-assigned this Aug 28, 2023
@ahuang11
Copy link
Collaborator

ahuang11 commented Aug 29, 2023

import holoviews as hv
import xarray as xr
import numpy as np
import panel as pn

hv.extension("bokeh")


da = xr.DataArray(coords={"x": [1, 2, 3], "y": [6, 7, 8]}, dims=["x", "y"])

da_1 = da.copy()
da_1.name = "a"
da_1.values = [[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3]

da_2 = da.copy()
da_2.name = "b"
da_2.values = [[np.nan] * 3, [1, 1, np.nan], [np.nan] * 3]

da_3 = da.copy()
da_3.name = "c"
da_3.values = [[np.nan] * 3, [np.nan] * 3, [1, 1, 1]]

ds = xr.merge([da_1, da_2, da_3])

colors = ["red", "blue", "green"]
a = hv.ImageStack(ds).opts(
    # colorbar=True,
    cmap=colors,
    num_colors=100,
    title="original",
)
b = a.clone().opts(invert_axes=True, title="invert axes")

c = a.clone().opts(invert_yaxis=True, title="invert yaxis")
d = a.clone().opts(invert_xaxis=True, title="invert xaxis")
pn.GridBox(
    pn.panel(a, linked_axes=False),
    pn.panel(b, linked_axes=False),
    pn.panel(c, linked_axes=False),
    pn.panel(d, linked_axes=False),
    ncols=2,
)
image

@ahuang11
Copy link
Collaborator

Does hover need support?

@Hoxbro
Copy link
Member Author

Hoxbro commented Aug 29, 2023

Does hover need support?

Hover support is out of scope for this PR as it also needs work in Bokeh (or a new release). See bokeh/bokeh#13193, bokeh/bokeh#13200, and bokeh/bokeh#13354.

@ahuang11
Copy link
Collaborator

import holoviews as hv
import xarray as xr
import numpy as np
import panel as pn
from holoviews.operation.datashader import shade, datashade

hv.extension("bokeh")


da = xr.DataArray(coords={"x": [1, 2, 3], "y": [6, 7, 8]}, dims=["x", "y"])

da_1 = da.copy()
da_1.name = "a"
da_1.values = [[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3]

da_2 = da.copy()
da_2.name = "b"
da_2.values = [[np.nan] * 3, [1, 1, np.nan], [np.nan] * 3]

da_3 = da.copy()
da_3.name = "c"
da_3.values = [[np.nan] * 3, [np.nan] * 3, [1, 1, 1]]

ds = xr.merge([da_1, da_2, da_3])

colors = ["red", "blue", "green"]
a = hv.ImageStack(ds).opts(
    # colorbar=True,
    cmap=colors,
    num_colors=100,
    title="original",
)
# pn.GridBox(
#     pn.panel(a, linked_axes=False),
#     pn.panel(b, linked_axes=False),
#     pn.panel(c, linked_axes=False),
#     pn.panel(d, linked_axes=False),
#     ncols=2,
# )

shade(a)
image

@ahuang11
Copy link
Collaborator

mpl & plotly

image image

@ahuang11
Copy link
Collaborator

Weirdly using numpy array transposes the plot

import holoviews as hv
import xarray as xr
import numpy as np
import panel as pn

hv.extension("bokeh")

x = np.arange(0, 3)
y = np.arange(5, 8)
a = np.array([[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3])
b = np.array([[np.nan] * 3, [1, 1, np.nan], [np.nan] * 3])
c = np.array([[np.nan] * 3, [np.nan] * 3, [1, 1, 1]])

ds = xr.Dataset(
    {"a": (["x", "y"], a), "b": (["x", "y"], b), "c": (["x", "y"], c)},
    coords={"x": x, "y": y},
)

colors = ["green", "red", "blue"]
(
    hv.ImageStack(ds, kdims=["x", "y"], vdims=["a", "b", "c"]).opts(cmap=colors) + 
    hv.ImageStack((x, y, a, b, c), kdims=["x", "y"], vdims=["a", "b", "c"]).opts(cmap=colors)
)
image

@ahuang11
Copy link
Collaborator

ahuang11 commented Aug 29, 2023

Okay, I guess Image does the same, so maybe it's not an issue / separate issue
image

import holoviews as hv
import xarray as xr
import numpy as np
import panel as pn

hv.extension("bokeh")

x = np.arange(0, 3)
y = np.arange(5, 8)
a = np.array([[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3])
b = np.array([[np.nan] * 3, [1, 1, np.nan], [np.nan] * 3])
c = np.array([[np.nan] * 3, [np.nan] * 3, [1, 1, 1]])

ds = xr.Dataset(
    {"a": (["x", "y"], a), "b": (["x", "y"], b), "c": (["x", "y"], c)},
    coords={"x": x, "y": y},
)

colors = ["green", "red", "blue"]
(
    hv.ImageStack(ds, kdims=["x", "y"], vdims=["a", "b", "c"]).opts(cmap=colors) + 
    hv.ImageStack((x, y, a, b, c), kdims=["x", "y"], vdims=["a", "b", "c"]).opts(cmap=colors) +
    hv.Image(ds["a"], kdims=["x", "y"], vdims=["a"]) +
    hv.Image((x, y, a))
).cols(2)

@philippjfr
Copy link
Member

Yeah definitely an issue but a separate issue.

@ahuang11
Copy link
Collaborator

I believe this is ready for initial review, but I don't understand the invert_x/yaxis tests because although the tests pass, the inverted data matches the original data and it renders as expected.

@ahuang11 ahuang11 changed the title [WIP] Implementation of ImageStack Implementation of ImageStack Aug 29, 2023
@ahuang11 ahuang11 marked this pull request as ready for review August 29, 2023 22:36
Copy link
Member

@philippjfr philippjfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments, otherwise this looks good.

@ahuang11
Copy link
Collaborator

Hmm this error seems a bit tricky:

Traceback ```python
______________ examples\user_guide\15-Large_Data.ipynb::Cell 16 _______________

[gw1] win32 -- Python 3.8.17 C:\Miniconda3\envs\test-environment\python.exe

Notebook cell execution failed

Cell 16: Cell execution caused an exception



Input:

gaussians = {i: hv.Points(rand_gauss2d(i), kdims, "i") for i in range(num_ks)}

c = dynspread(datashade(hv.NdOverlay(gaussians, kdims='k'), aggregator=ds.by('k', ds.count())))

m = dynspread(datashade(hv.NdOverlay(gaussians, kdims='k'), aggregator=ds.by('k', ds.mean("i"))))



c.opts(width=400) + m.opts(width=400)



Traceback:



---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

File C:\Miniconda3\envs\test-environment\lib\site-packages\IPython\core\formatters.py:974, in MimeBundleFormatter.__call__(self, obj, include, exclude)

    971     method = get_real_method(obj, self.print_method)

    973     if method is not None:

--> 974         return method(include=include,exclude=exclude)

    975     return None

    976 else:



File d:\a\holoviews\holoviews\holoviews\core\dimension.py:1287, in Dimensioned._repr_mimebundle_(self, include, exclude)

1280 def _repr_mimebundle_(self, include=None, exclude=None):

1281     """

1282     Resolves the class hierarchy for the class rendering the

1283     object using any display hooks registered on Store.display

1284     hooks.  The output of all registered display_hooks is then

1285     combined and returned.

1286     """

-> 1287     return Store.render(self)



File d:\a\holoviews\holoviews\holoviews\core\options.py:1423, in Store.render(cls, obj)

1421 data, metadata = {}, {}

1422 for hook in hooks:

-> 1423     ret = hook(obj)

1424     if ret is None:

1425         continue



File d:\a\holoviews\holoviews\holoviews\ipython\display_hooks.py:280, in pprint_display(obj)

    278 if not ip.display_formatter.formatters['text/plain'].pprint:

    279     return None

--> 280 return display(obj,raw_output=True)



File d:\a\holoviews\holoviews\holoviews\ipython\display_hooks.py:251, in display(obj, raw_output, **kwargs)

    249 elif isinstance(obj, (Layout, NdLayout, AdjointLayout)):

    250     with option_state(obj):

--> 251         output = layout_display(obj)

    252 elif isinstance(obj, (HoloMap, DynamicMap)):

    253     with option_state(obj):



File d:\a\holoviews\holoviews\holoviews\ipython\display_hooks.py:142, in display_hook.<locals>.wrapped(element)

    140 try:

    141     max_frames = OutputSettings.options['max_frames']

--> 142     mimebundle = fn(element,max_frames=max_frames)

    143     if mimebundle is None:

    144         return {}, {}



File d:\a\holoviews\holoviews\holoviews\ipython\display_hooks.py:216, in layout_display(layout, max_frames)

    213     max_frame_warning(max_frames)

    214     return None

--> 216 return render(layout)



File d:\a\holoviews\holoviews\holoviews\ipython\display_hooks.py:69, in render(obj, **kwargs)

    66 if renderer.fig == 'pdf':

    67     renderer = renderer.instance(fig='png')

---> 69 return renderer.components(obj,**kwargs)



File d:\a\holoviews\holoviews\holoviews\plotting\renderer.py:397, in Renderer.components(self, obj, fmt, comm, **kwargs)

    394 embed = (not (dynamic or streams or self.widget_mode == 'live') or config.embed)

    3[96](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:97) if embed or config.comms == 'default':

--> 3[97](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:98)     return self._render_panel(plot,embed,comm)

    3[98](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:99) return self._render_ipywidget(plot)



File d:\a\holoviews\holoviews\holoviews\plotting\renderer.py:404, in Renderer._render_panel(self, plot, embed, comm)

    402 doc = Document()

    403 with config.set(embed=embed):

--> 404     model = plot.layout._render_model(doc,comm)

    405 if embed:

    406     return render_model(model, comm)



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\viewable.py:737, in Viewable._render_model(self, doc, comm)

    735 if comm is None:

    736     comm = state._comm_manager.get_server_comm()

--> 737 model = self.get_root(doc,comm)

    739 if self._design and self._design.theme.bokeh_theme:

    740     doc.theme = self._design.theme.bokeh_theme



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\layout\base.py:306, in Panel.get_root(self, doc, comm, preprocess)

    302 def get_root(

    303     self, doc: Optional[Document] = None, comm: Optional[Comm] = None,

    304     preprocess: bool = True

    305 ) -> Model:

--> 306     root = super().get_root(doc,comm,preprocess)

    307     # ALERT: Find a better way to handle this

    308     if hasattr(root, 'styles') and 'overflow-x' in root.styles:



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\viewable.py:659, in Renderable.get_root(self, doc, comm, preprocess)

    657 wrapper = self._design._wrapper(self)

    658 if wrapper is self:

--> 659     root = self._get_model(doc,comm=comm)

    660     if preprocess:

    661         self._preprocess(root)



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\layout\base.py:174, in Panel._get_model(self, doc, root, parent, comm)

    172 root = root or model

    173 self._models[root.ref['id']] = (model, parent)

--> 174 objects, _ = self._get_objects(model,[],doc,root,comm)

    175 props = self._get_properties(doc)

    176 props[self._property_mapping['objects']] = objects



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\layout\base.py:156, in Panel._get_objects(self, model, old_objects, doc, root, comm)

    154 else:

    155     try:

--> 156         child = pane._get_model(doc,root,model,comm)

    157     except RerenderError as e:

    158         if e.layout is not None and e.layout is not self:



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\pane\holoviews.py:411, in HoloViews._get_model(self, doc, root, parent, comm)

    409     plot = self.object

    410 else:

--> 411     plot = self._render(doc,comm,root)

    413 plot.pane = self

    414 backend = plot.renderer.backend



File C:\Miniconda3\envs\test-environment\lib\site-packages\panel\pane\holoviews.py:506, in HoloViews._render(self, doc, comm, root)

    503     if comm:

    504         kwargs['comm'] = comm

--> 506 return renderer.get_plot(self.object,**kwargs)



File d:\a\holoviews\holoviews\holoviews\plotting\bokeh\renderer.py:70, in BokehRenderer.get_plot(self_or_cls, obj, doc, renderer, **kwargs)

    63 @bothmethod

    64 def get_plot(self_or_cls, obj, doc=None, renderer=None, **kwargs):

    65     """

    66     Given a HoloViews Viewable return a corresponding plot instance.

    67     Allows supplying a document attach the plot to, useful when

    68     combining the bokeh model with another plot.

    69     """

---> 70     plot = super().get_plot(obj,doc,renderer,**kwargs)

    71     if plot.document is None:

    72         plot.document = Document() if self_or_cls.notebook_context else curdoc()



File d:\a\holoviews\holoviews\holoviews\plotting\renderer.py:218, in Renderer.get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)

    215     raise SkipRendering(msg.format(dims=dims))

    217 # Initialize DynamicMaps with first data item

--> 218 initialize_dynamic(obj)

    220 if not renderer:

    221     renderer = self_or_cls



File d:\a\holoviews\holoviews\holoviews\plotting\util.py:256, in initialize_dynamic(obj)

    254     continue

    255 if not len(dmap):

--> 256     dmap[dmap._initial_key()]



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:1212, in DynamicMap.__getitem__(self, key)

1210 # Not a cross product and nothing cached so compute element.

1211 if cache is not None: return cache

-> 1212 val = self._execute_callback(*tuple_key)

1213 if data_slice:

1214     val = self._dataslice(val, data_slice)



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:979, in DynamicMap._execute_callback(self, *args)

    976     kwargs['_memoization_hash_'] = hash_items

    978 with dynamicmap_memoization(self.callback, self.streams):

--> 979     retval = self.callback(*args,**kwargs)

    980 return self._style(retval)



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:549, in Callable.__call__(self, *args, **kwargs)

    547 kwarg_hash = kwargs.pop('_memoization_hash_', ())

    548 (self.args, self.kwargs) = (args, kwargs)

--> 549 if not args and not kwargs and not any(kwarg_hash): return self.callable()

    550 inputs = [i for i in self.inputs if isinstance(i, DynamicMap)]

    551 streams = []



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:1024, in Dynamic._dynamic_operation.<locals>.dynamic_operation(*key, **kwargs)

1023 def dynamic_operation(*key, **kwargs):

-> 1024     key, obj = resolve(key,kwargs)

1025     return apply(obj, *key, **kwargs)



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:1013, in Dynamic._dynamic_operation.<locals>.resolve(key, kwargs)

1011 elif isinstance(map_obj, DynamicMap) and map_obj._posarg_keys and not key:

1012     key = tuple(kwargs[k] for k in map_obj._posarg_keys)

-> 1013 return key, map_obj[key]



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:1212, in DynamicMap.__getitem__(self, key)

1210 # Not a cross product and nothing cached so compute element.

1211 if cache is not None: return cache

-> 1212 val = self._execute_callback(*tuple_key)

1213 if data_slice:

1214     val = self._dataslice(val, data_slice)



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:979, in DynamicMap._execute_callback(self, *args)

    976     kwargs['_memoization_hash_'] = hash_items

    978 with dynamicmap_memoization(self.callback, self.streams):

--> 979     retval = self.callback(*args,**kwargs)

    980 return self._style(retval)



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:549, in Callable.__call__(self, *args, **kwargs)

    547 kwarg_hash = kwargs.pop('_memoization_hash_', ())

    548 (self.args, self.kwargs) = (args, kwargs)

--> 549 if not args and not kwargs and not any(kwarg_hash): return self.callable()

    550 inputs = [i for i in self.inputs if isinstance(i, DynamicMap)]

    551 streams = []



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:1024, in Dynamic._dynamic_operation.<locals>.dynamic_operation(*key, **kwargs)

1023 def dynamic_operation(*key, **kwargs):

-> 1024     key, obj = resolve(key,kwargs)

1025     return apply(obj, *key, **kwargs)



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:1013, in Dynamic._dynamic_operation.<locals>.resolve(key, kwargs)

1011 elif isinstance(map_obj, DynamicMap) and map_obj._posarg_keys and not key:

1012     key = tuple(kwargs[k] for k in map_obj._posarg_keys)

-> 1013 return key, map_obj[key]



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:1212, in DynamicMap.__getitem__(self, key)

1210 # Not a cross product and nothing cached so compute element.

1211 if cache is not None: return cache

-> 1212 val = self._execute_callback(*tuple_key)

1213 if data_slice:

1214     val = self._dataslice(val, data_slice)



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:979, in DynamicMap._execute_callback(self, *args)

    976     kwargs['_memoization_hash_'] = hash_items

    978 with dynamicmap_memoization(self.callback, self.streams):

--> 979     retval = self.callback(*args,**kwargs)

    980 return self._style(retval)



File d:\a\holoviews\holoviews\holoviews\core\spaces.py:579, in Callable.__call__(self, *args, **kwargs)

    576     args, kwargs = (), dict(pos_kwargs, **kwargs)

    578 try:

--> 579     ret = self.callable(*args,**kwargs)

    580 except KeyError:

    581     # KeyError is caught separately because it is used to signal

    582     # invalid keys on DynamicMap and should not warn

    583     raise



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:1025, in Dynamic._dynamic_operation.<locals>.dynamic_operation(*key, **kwargs)

1023 def dynamic_operation(*key, **kwargs):

1024     key, obj = resolve(key, kwargs)

-> 1025     return apply(obj,*key,**kwargs)



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:1017, in Dynamic._dynamic_operation.<locals>.apply(element, *key, **kwargs)

1015 def apply(element, *key, **kwargs):

1016     kwargs = dict(util.resolve_dependent_kwargs(self.p.kwargs), **kwargs)

-> 1017     processed = self._process(element,key,kwargs)

1018     if (self.p.link_dataset and isinstance(element, Dataset) and

1019         isinstance(processed, Dataset) and processed._dataset is None):

1020         processed._dataset = element.dataset



File d:\a\holoviews\holoviews\holoviews\util\__init__.py:[99](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:100)9, in Dynamic._process(self, element, key, kwargs)

    997 elif isinstance(self.p.operation, Operation):

    998     kwargs = {k: v for k, v in kwargs.items() if k in self.p.operation.param}

--> 999     return self.p.operation.process_element(element,key,**kwargs)

[100](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:101)0 else:

1001     return self.p.operation(element, **kwargs)



File d:\a\holoviews\holoviews\holoviews\core\operation.py:194, in Operation.process_element(self, element, key, **params)

    191 else:

    192     self.p = param.ParamOverrides(self, params,

    193                                   allow_extra_keywords=self._allow_extra_keywords)

--> 194 return self._apply(element,key)



File d:\a\holoviews\holoviews\holoviews\core\operation.py:141, in Operation._apply(self, element, key)

    139     if not in_method:

    140         element._in_method = True

--> 141 ret = self._process(element,key)

    142 if hasattr(element, '_in_method') and not in_method:

    143     element._in_method = in_method



File d:\a\holoviews\holoviews\holoviews\operation\datashader.py:1501, in datashade._process(self, element, key)

1500 def _process(self, element, key=None):

-> 1501     agg = rasterize._process(self,element,key)

1502     shaded = shade._process(self, agg, key)

1503     return shaded



File d:\a\holoviews\holoviews\holoviews\operation\datashader.py:1480, in rasterize._process(self, element, key)

1477     op = transform.instance(**{k:v for k,v in extended_kws.items()

1478                                if k in transform.param})

1479     op._precomputed = self._precomputed

-> 1480     element = element.map(op,predicate)

1481     self._precomputed = op._precomputed

1483 unused_params = list(all_supplied_kws - all_allowed_kws)



File d:\a\holoviews\holoviews\holoviews\core\dimension.py:695, in LabelledData.map(self, map_fn, specs, clone)

    693         if new_val is not None:

    694             deep_mapped[k] = new_val

--> 695     if applies: deep_mapped = map_fn(deep_mapped)

    696     return deep_mapped

    697 else:



File d:\a\holoviews\holoviews\holoviews\core\operation.py:214, in Operation.__call__(self, element, **kwargs)

    210         return element.clone([(k, self._apply(el, key=k))

    211                               for k, el in element.items()])

    212     elif ((self._per_element and isinstance(element, Element)) or

    213           (not self._per_element and isinstance(element, ViewableElement))):

--> 214         return self._apply(element)

    215 elif 'streams' not in kwargs:

    216     kwargs['streams'] = self.p.streams



File d:\a\holoviews\holoviews\holoviews\core\operation.py:141, in Operation._apply(self, element, key)

    139     if not in_method:

    140         element._in_method = True

--> 141 ret = self._process(element,key)

    142 if hasattr(element, '_in_method') and not in_method:

    143     element._in_method = in_method



File d:\a\holoviews\holoviews\holoviews\operation\datashader.py:391, in aggregate._process(self, element, key)

    389 else:

    390     params['vdims'] = list(agg.coords[agg_fn.column].data)

--> 391     return ImageStack(agg,**params)



File d:\a\holoviews\holoviews\holoviews\element\raster.py:529, in ImageStack.__init__(self, data, kdims, vdims, **params)

    528 def __init__(self, data, kdims=None, vdims=None, **params):

--> 529     super().__init__(data,kdims,vdims,**params)

    530     # I think we don't have to consider pandas here

    531     # len(xr.Dataset) simply provides the number of variables

    532     # len(dict) provides the number of keys

    533     unused_dims = len(data) - len(self.kdims + self.vdims)



File d:\a\holoviews\holoviews\holoviews\element\raster.py:277, in Image.__init__(self, data, kdims, vdims, bounds, extents, xdensity, ydensity, rtol, **params)

    274 else:

    275     params['rtol'] = config.image_rtol

--> 277 Dataset.__init__(self,data,kdims=kdims,vdims=vdims,extents=extents,**params)

    278 if not self.interface.gridded:

    279     raise DataError("{} type expects gridded data, {} is columnar. "

    280                     "To display columnar data as gridded use the HeatMap "

    281                     "element or aggregate the data (e.g. using rasterize "

    282                     "or np.histogram2d).".format(type(self).__name__, self.interface.__name__))



File d:\a\holoviews\holoviews\holoviews\core\data\__init__.py:323, in Dataset.__init__(self, data, kdims, vdims, **kwargs)

    320     if input_transforms is None:

    321         input_transforms = data._transforms

--> 323 kwargs.update(process_dimensions(kdims,vdims))

    324 kdims, vdims = kwargs.get('kdims'), kwargs.get('vdims')

    326 validate_vdims = kwargs.pop('_validate_vdims', True)



File d:\a\holoviews\holoviews\holoviews\core\dimension.py:117, in process_dimensions(kdims, vdims)

    [112](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:113)     elif not isinstance(dims, list):

    [113](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:114)         raise ValueError("{} argument expects a Dimension or list of dimensions, "

    [114](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:115)                          "specified as tuples, strings, dictionaries or Dimension "

    [115](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:116)                          "instances, not a {} type. Ensure you passed the data as the "

    [116](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:117)                          "first argument.".format(group, type(dims).__name__))

--> [117](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:118)     dimensions[group] = [asdim(d) for d in dims]
--> 117     dimensions[group] = [asdim(d) for d in dims]
--> 117     dimensions[group] = [asdim(d) for d in dims]
--> 117     dimensions[group] = [asdim(d) for d in dims]
--> 117     dimensions[group] = [asdim(d) for d in dims]
--> 117     dimensions[group] = [asdim(d) for d in dims]

    [118](https://github.com/holoviz/holoviews/actions/runs/6030322490/job/16361879215?pr=5751#step:5:119) return dimensions



File d:\a\holoviews\holoviews\holoviews\core\dimension.py:61, in asdim(dimension)

    51 def asdim(dimension):

    52     """Convert the input to a Dimension.

    53 

    54     Args:

    (...)

    59         copy is performed if the input is already a Dimension.

    60     """

---> 61     return dimension if isinstance(dimension, Dimension) else Dimension(dimension)



File d:\a\holoviews\holoviews\holoviews\core\dimension.py:269, in Dimension.__init__(self, spec, **params)

    265         raise ValueError(

    266             'Dimension specified as a dict must contain a "name" key'

    267         ) from exc

    268 else:

--> 269     raise ValueError(

    270         '%s type could not be interpreted as Dimension.  Dimensions must be '

    271         'declared as a string, tuple, dictionary or Dimension type.'

    272         % type(spec).__name__

    273     )

    274 all_params.update(params)

    276 if not all_params['name']:



ValueError: int64 type could not be interpreted as Dimension.  Dimensions must be declared as a string, tuple, dictionary or Dimension type.

</summary>

Copy link
Member Author

@Hoxbro Hoxbro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just had a quick look at the code and left some comments.

For the hover problem, try to filter the _hover_opts method. For now, I think we should remove these extra dimensions, as we can only get one layer of information for now. When Bokeh 3.3 is released, we can extract more information because of bokeh/bokeh#13193 (maybe we can already get this with a dev release). Further down the line, we should be able to get even better support with bokeh/bokeh#13200.

(Merged in main to be able only to be able to see the changes made in this PR)

holoviews/operation/datashader.py Show resolved Hide resolved
holoviews/plotting/bokeh/raster.py Outdated Show resolved Hide resolved
holoviews/core/data/xarray.py Show resolved Hide resolved
holoviews/element/raster.py Outdated Show resolved Hide resolved
@ahuang11
Copy link
Collaborator

ahuang11 commented Sep 12, 2023

I don't quite understand the behavior of the ImageStack hover

Alone, it's understandable

import holoviews as hv
import xarray as xr
import numpy as np
import panel as pn
from holoviews.operation.datashader import rasterize

hv.extension("bokeh")


da = xr.DataArray(coords={"x": [1, 2, 3], "y": [6, 7, 8]}, dims=["x", "y"])

da_1 = da.copy()
da_1.name = "a"
da_1.values = [[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3]

ds = xr.merge([da_1]).fillna(0)

colors = ["red", "blue", "green"]
a = hv.ImageStack(ds).opts(
    cmap=colors,
    title="original",
).opts(tools=["hover"])

a
image

However, stacked on another image, it doesn't match expectations.

import holoviews as hv
import xarray as xr
import numpy as np
import panel as pn
from holoviews.operation.datashader import rasterize

hv.extension("bokeh")


da = xr.DataArray(coords={"x": [1, 2, 3], "y": [6, 7, 8]}, dims=["x", "y"])

da_1 = da.copy()
da_1.name = "a"
da_1.values = [[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3]

da_2 = da.copy()
da_2.name = "b"
da_2.values = [[np.nan, np.nan, 1], [np.nan] * 3, [np.nan] * 3]

ds = xr.merge([da_1, da_2]).fillna(0)

colors = ["red", "blue", "green"]
a = hv.ImageStack(ds).opts(
    cmap=colors,
    title="original",
).opts(tools=["hover"])

a

Why does this show 0?
image

@Hoxbro
Copy link
Member Author

Hoxbro commented Sep 12, 2023

It is a limitation in Bokeh. Try to read the issues I mentioned in my last post.

@ahuang11
Copy link
Collaborator

ahuang11 commented Sep 12, 2023

For the hover problem, try to filter the _hover_opts method. For now, I think we should remove these extra dimensions, as we can only get one layer of information for now.

Okay. To build upon that though, I don't think we can even get one layer though:

e.g. I expect 1 instead of 0 here:
image

So do we just keep x and y?

@Hoxbro
Copy link
Member Author

Hoxbro commented Sep 12, 2023

So do we just keep x and y?

yes

@ahuang11
Copy link
Collaborator

ahuang11 commented Sep 12, 2023

Should I be worried about the core tests?

Eh I just wrapped try/except.

@Hoxbro
Copy link
Member Author

Hoxbro commented Sep 13, 2023

Eh I just wrapped try/except.

I would rather see something like this:

try:
import xarray as xr
except ImportError:
raise SkipTest("Test requires xarray")

holoviews/plotting/bokeh/raster.py Show resolved Hide resolved
holoviews/operation/datashader.py Show resolved Hide resolved
holoviews/tests/plotting/bokeh/test_rasterplot.py Outdated Show resolved Hide resolved
Copy link
Member

@philippjfr philippjfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from the minor comments left by @Hoxbro and I this looks great.

@philippjfr philippjfr merged commit 923cd27 into main Sep 14, 2023
9 checks passed
@philippjfr philippjfr deleted the categorical branch September 14, 2023 18:14
@Hoxbro Hoxbro restored the categorical branch November 21, 2023 13:52
@Hoxbro Hoxbro deleted the categorical branch November 21, 2023 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants