[BUG] range no longer works with ColumnDataSource #12313

hoxbro · 2022-08-19T11:44:00Z

Software versions

Python version      :  3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:56:21) 
IPython version     :  8.4.0
Tornado version     :  6.2
Bokeh version       :  3.0.0dev11+3.g23e05ae31
BokehJS static path :  /home/shh/Repos/bokeh3/bokeh/bokeh/server/static
node.js version     :  v18.7.0
npm version         :  8.16.0
Operating system    :  Linux-5.18.10-76051810-generic-x86_64-with-glibc2.35

Browser name and version

No response

Jupyter notebook / Jupyter Lab version

No response

Expected behavior

The code should be able to run, or a better error message is needed.

Observed behavior

The example code does not run on bokeh 3.0, it works on version 2.4.3.

Example code

from bokeh.models import ColumnDataSource
from bokeh.plotting import figure
from bokeh.io import output_notebook, show
output_notebook()

source = ColumnDataSource({"x": range(10), "y": range(10)})
p = figure()
p.line(x="x", y="y", source=source)
show(p)

Stack traceback or browser console output

AttributeError                            Traceback (most recent call last)
Input In [2], in <cell line: 9>()
      7 p = figure()
      8 p.line(x="x", y="y", source=source)
----> 9 show(p)

File ~/Repos/bokeh3/bokeh/bokeh/io/showing.py:144, in show(obj, browser, new, notebook_handle, notebook_url, **kwargs)
    141 state = curstate()
    143 if isinstance(obj, LayoutDOM):
--> 144     return _show_with_state(obj, state, browser, new, notebook_handle=notebook_handle)
    146 def is_application(obj: Any) -> TypeGuard[Application]:
    147     return getattr(obj, '_is_a_bokeh_application_class', False)

File ~/Repos/bokeh3/bokeh/bokeh/io/showing.py:191, in _show_with_state(obj, state, browser, new, notebook_handle)
    189 if state.notebook:
    190     assert state.notebook_type is not None
--> 191     comms_handle = run_notebook_hook(state.notebook_type, 'doc', obj, state, notebook_handle)
    192     shown = True
    194 if state.file or not shown:

File ~/Repos/bokeh3/bokeh/bokeh/io/notebook.py:367, in run_notebook_hook(notebook_type, action, *args, **kwargs)
    365 if _HOOKS[notebook_type][action] is None:
    366     raise RuntimeError(f"notebook hook for {notebook_type!r} did not install {action!r} action")
--> 367 return _HOOKS[notebook_type][action](*args, **kwargs)

File ~/Repos/bokeh3/bokeh/bokeh/io/notebook.py:584, in show_doc(obj, state, notebook_handle)
    582 from ..embed.notebook import notebook_content
    583 comms_target = make_id() if notebook_handle else None
--> 584 (script, div, cell_doc) = notebook_content(obj, comms_target)
    586 publish_display_data({HTML_MIME_TYPE: div})
    587 publish_display_data({JS_MIME_TYPE: script, EXEC_MIME_TYPE: ""}, metadata={EXEC_MIME_TYPE: {"id": obj.id}})

File ~/Repos/bokeh3/bokeh/bokeh/embed/notebook.py:90, in notebook_content(model, notebook_comms_target, theme)
     87 # Comms handling relies on the fact that the new_doc returned here
     88 # has models with the same IDs as they were started with
     89 with OutputDocumentFor([model], apply_theme=theme, always_new=True) as new_doc:
---> 90     (docs_json, [render_item]) = standalone_docs_json_and_render_items([model])
     92 div = div_for_render_item(render_item)
     94 render_item = render_item.to_json()

File ~/Repos/bokeh3/bokeh/bokeh/embed/util.py:331, in standalone_docs_json_and_render_items(models, suppress_callback_warning)
    329 docs_json: dict[ID, DocJson] = {}
    330 for doc, (docid, _) in docs.items():
--> 331     docs_json[docid] = doc.to_json(deferred=False)
    333 render_items: list[RenderItem] = []
    334 for _, (docid, roots) in docs.items():

File ~/Repos/bokeh3/bokeh/bokeh/document/document.py:733, in Document.to_json(self, deferred)
    731 serializer = Serializer(deferred=deferred)
    732 defs = serializer.encode(data_models)
--> 733 roots = serializer.encode(self._roots)
    735 doc_json = DocJson(
    736     version=__version__,
    737     title=self.title,
    738     defs=defs,
    739     roots=roots,
    740 )
    742 self.models.flush()

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:281, in Serializer._encode(self, obj)
    279     return self._encode_tuple(obj)
    280 elif isinstance(obj, list):
--> 281     return self._encode_list(obj)
    282 elif isinstance(obj, dict):
    283     return self._encode_dict(obj)

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:322, in Serializer._encode_list(self, obj)
    321 def _encode_list(self, obj: list[Any]) -> ArrayRepLike:
--> 322     return [self.encode(item) for item in obj]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:322, in <listcomp>(.0)
    321 def _encode_list(self, obj: list[Any]) -> ArrayRepLike:
--> 322     return [self.encode(item) for item in obj]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:265, in Serializer._encode(self, obj)
    263 def _encode(self, obj: Any) -> AnyRep:
    264     if isinstance(obj, Serializable):
--> 265         return obj.to_serializable(self)
    266     elif (encoder := self._encoders.get(type(obj))) is not None:
    267         return encoder(obj, self)

File ~/Repos/bokeh3/bokeh/bokeh/model/model.py:521, in Model.to_serializable(self, serializer)
    518 def to_serializable(self, serializer: Serializer) -> ObjectRefRep:
    519     serializer.add_ref(self, self.ref)
--> 521     super_rep = super().to_serializable(serializer)
    522     rep = ObjectRefRep(
    523         type="object",
    524         name=super_rep["name"],
    525         id=self.id,
    526     )
    528     attributes = super_rep.get("attributes")

File ~/Repos/bokeh3/bokeh/bokeh/core/has_props.py:377, in HasProps.to_serializable(self, serializer)
    371 rep = ObjectRep(
    372     type="object",
    373     name=self.__qualified_model__,
    374 )
    376 properties = self.properties_with_values(include_defaults=False)
--> 377 attributes = {key: serializer.encode(val) for key, val in properties.items()}
    379 if attributes:
    380     rep["attributes"] = attributes

File ~/Repos/bokeh3/bokeh/bokeh/core/has_props.py:377, in <dictcomp>(.0)
    371 rep = ObjectRep(
    372     type="object",
    373     name=self.__qualified_model__,
    374 )
    376 properties = self.properties_with_values(include_defaults=False)
--> 377 attributes = {key: serializer.encode(val) for key, val in properties.items()}
    379 if attributes:
    380     rep["attributes"] = attributes

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:281, in Serializer._encode(self, obj)
    279     return self._encode_tuple(obj)
    280 elif isinstance(obj, list):
--> 281     return self._encode_list(obj)
    282 elif isinstance(obj, dict):
    283     return self._encode_dict(obj)

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:322, in Serializer._encode_list(self, obj)
    321 def _encode_list(self, obj: list[Any]) -> ArrayRepLike:
--> 322     return [self.encode(item) for item in obj]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:322, in <listcomp>(.0)
    321 def _encode_list(self, obj: list[Any]) -> ArrayRepLike:
--> 322     return [self.encode(item) for item in obj]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:265, in Serializer._encode(self, obj)
    263 def _encode(self, obj: Any) -> AnyRep:
    264     if isinstance(obj, Serializable):
--> 265         return obj.to_serializable(self)
    266     elif (encoder := self._encoders.get(type(obj))) is not None:
    267         return encoder(obj, self)

File ~/Repos/bokeh3/bokeh/bokeh/model/model.py:521, in Model.to_serializable(self, serializer)
    518 def to_serializable(self, serializer: Serializer) -> ObjectRefRep:
    519     serializer.add_ref(self, self.ref)
--> 521     super_rep = super().to_serializable(serializer)
    522     rep = ObjectRefRep(
    523         type="object",
    524         name=super_rep["name"],
    525         id=self.id,
    526     )
    528     attributes = super_rep.get("attributes")

File ~/Repos/bokeh3/bokeh/bokeh/core/has_props.py:377, in HasProps.to_serializable(self, serializer)
    371 rep = ObjectRep(
    372     type="object",
    373     name=self.__qualified_model__,
    374 )
    376 properties = self.properties_with_values(include_defaults=False)
--> 377 attributes = {key: serializer.encode(val) for key, val in properties.items()}
    379 if attributes:
    380     rep["attributes"] = attributes

File ~/Repos/bokeh3/bokeh/bokeh/core/has_props.py:377, in <dictcomp>(.0)
    371 rep = ObjectRep(
    372     type="object",
    373     name=self.__qualified_model__,
    374 )
    376 properties = self.properties_with_values(include_defaults=False)
--> 377 attributes = {key: serializer.encode(val) for key, val in properties.items()}
    379 if attributes:
    380     rep["attributes"] = attributes

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:265, in Serializer._encode(self, obj)
    263 def _encode(self, obj: Any) -> AnyRep:
    264     if isinstance(obj, Serializable):
--> 265         return obj.to_serializable(self)
    266     elif (encoder := self._encoders.get(type(obj))) is not None:
    267         return encoder(obj, self)

File ~/Repos/bokeh3/bokeh/bokeh/model/model.py:521, in Model.to_serializable(self, serializer)
    518 def to_serializable(self, serializer: Serializer) -> ObjectRefRep:
    519     serializer.add_ref(self, self.ref)
--> 521     super_rep = super().to_serializable(serializer)
    522     rep = ObjectRefRep(
    523         type="object",
    524         name=super_rep["name"],
    525         id=self.id,
    526     )
    528     attributes = super_rep.get("attributes")

File ~/Repos/bokeh3/bokeh/bokeh/core/has_props.py:377, in HasProps.to_serializable(self, serializer)
    371 rep = ObjectRep(
    372     type="object",
    373     name=self.__qualified_model__,
    374 )
    376 properties = self.properties_with_values(include_defaults=False)
--> 377 attributes = {key: serializer.encode(val) for key, val in properties.items()}
    379 if attributes:
    380     rep["attributes"] = attributes

File ~/Repos/bokeh3/bokeh/bokeh/core/has_props.py:377, in <dictcomp>(.0)
    371 rep = ObjectRep(
    372     type="object",
    373     name=self.__qualified_model__,
    374 )
    376 properties = self.properties_with_values(include_defaults=False)
--> 377 attributes = {key: serializer.encode(val) for key, val in properties.items()}
    379 if attributes:
    380     rep["attributes"] = attributes

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:283, in Serializer._encode(self, obj)
    281     return self._encode_list(obj)
    282 elif isinstance(obj, dict):
--> 283     return self._encode_dict(obj)
    284 elif isinstance(obj, bytes):
    285     return self._encode_bytes(obj)

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:327, in Serializer._encode_dict(self, obj)
    324 def _encode_dict(self, obj: dict[Any, Any]) -> MapRep:
    325     return MapRep(
    326         type="map",
--> 327         entries=[(self.encode(key), self.encode(val)) for key, val in obj.items()],
    328     )

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:327, in <listcomp>(.0)
    324 def _encode_dict(self, obj: dict[Any, Any]) -> MapRep:
    325     return MapRep(
    326         type="map",
--> 327         entries=[(self.encode(key), self.encode(val)) for key, val in obj.items()],
    328     )

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:256, in Serializer.encode(self, obj)
    254 self._circular[ident] = obj
    255 try:
--> 256     return self._encode(obj)
    257 finally:
    258     del self._circular[ident]

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:295, in Serializer._encode(self, obj)
    293     return self._encode_dataclass(obj)
    294 else:
--> 295     return self._encode_other(obj)

File ~/Repos/bokeh3/bokeh/bokeh/core/serialization.py:444, in Serializer._encode_other(self, obj)
    441 if np.issubdtype(type(obj), np.bool_):
    442     return self._encode_bool(bool(obj))
--> 444 if _HAS_PANDAS and obj.__module__ is not None and obj.__module__.startswith("pandas"):
    445     pd = import_optional("pandas")
    446     if pd and isinstance(obj, (pd.Series, pd.Index)):

AttributeError: 'range' object has no attribute '__module__'

Screenshots

No response

mattpap · 2022-08-19T12:24:31Z

The are a few issues here.

First, the error should be:

bokeh.core.serialization.SerializationError: can't serialize <class 'range'>

That's an obvious bug. I assumed __module__ would be set to None for stdlib, but it's Python, so let's involve hasattr().

Second, serialization of iterators as arrays is a bad idea (this was done in 2.4). If anything, iterators have to be serialized as objects that can be deserialized to iterators (back and forth), but that's equivalent to serialization of functions in the most generic setup. Thus, if we want to keep iterator support in APIs like ColumnDataSource.data, then a transform has to happen either in ColumnDataSource or in data property (most likely the later, as data has a specialized type already).

hoxbro · 2022-08-19T12:34:29Z

I assumed module would be set to None

It seems to be missing because it is initialized:

>>> range.__module__
'builtins'
>>> range(10).__module__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'range' object has no attribute '__module__'
>>> type(range(10)).__module__
'builtins'

mattpap · 2022-08-19T12:39:04Z

Sure, the problem is that this is different behavior from user defined types:

n [1]: class Foo:
   ...:     pass
   ...: 

In [2]: o = Foo()

In [3]: o.__module__
Out[3]: '__main__'

In [4]: type(o).__module__
Out[4]: '__main__'

In [5]: Foo.__module__
Out[5]: '__main__'

In [6]: r = range(10)

In [7]: r.__module__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [7], in <cell line: 1>()
----> 1 r.__module__

AttributeError: 'range' object has no attribute '__module__'

In [8]: type(r).__module__
Out[8]: 'builtins'

In [9]: range.__module__
Out[9]: 'builtins'

hoxbro added the TRIAGE label Aug 19, 2022

mattpap added type: bug and removed TRIAGE labels Aug 19, 2022

mattpap added this to the 3.0 milestone Aug 19, 2022

mattpap added the tag: regression label Aug 19, 2022

mattpap mentioned this issue Aug 19, 2022

Fail gracefully when serializing unsupported builtins #12314

Merged

hoxbro mentioned this issue Aug 19, 2022

[WIP] Compatibility with Bokeh 3.0 (round 2) holoviz/panel#3752

Closed

8 tasks

mattpap closed this as completed in #12314 Aug 21, 2022

mattpap added the reso: completed label Aug 21, 2022

m-temp mentioned this issue Feb 14, 2023

Deprecated bokeh parameter used in cw/util/plot.py lowRISC/ot-sca#117

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] range no longer works with ColumnDataSource #12313

[BUG] range no longer works with ColumnDataSource #12313

hoxbro commented Aug 19, 2022

mattpap commented Aug 19, 2022

hoxbro commented Aug 19, 2022

mattpap commented Aug 19, 2022

[BUG] range no longer works with ColumnDataSource #12313

[BUG] range no longer works with ColumnDataSource #12313

Comments

hoxbro commented Aug 19, 2022

Software versions

Browser name and version

Jupyter notebook / Jupyter Lab version

Expected behavior

Observed behavior

Example code

Stack traceback or browser console output

Screenshots

mattpap commented Aug 19, 2022

hoxbro commented Aug 19, 2022

mattpap commented Aug 19, 2022