[BUG] DeserializationError when plotting graphs

[agrav]

Hi,

I am getting a DeserializationError when plotting graphs.

The issue can be reproduced with the code from the first example here: https://docs.bokeh.org/en/latest/docs/user_guide/graph.html. The code example below is run with the bokeh serve command. Traceback and error messages when loading the page is shown below.

This issue is related to:
https://discourse.bokeh.org/t/deserializationerror-for-graph-render/8508
holoviz/holoviews#5107

ALL software version info (bokeh, python, notebook, OS, browser, any other relevant packages)

bokeh: 2.4.1
OS: Ubuntu 20.04 (on WSL2)
Python: 3.9.5
Chrome: 95.0.4638.69 (on Win 10)

Complete, minimal, self-contained example code that reproduces the issue

import math
from bokeh.io import curdoc
from bokeh.models import GraphRenderer, Ellipse, StaticLayoutProvider
from bokeh.palettes import Spectral8
from bokeh.plotting import figure

# list the nodes and initialize a plot
N = 8
node_indices = list(range(N))

plot = figure(title="Graph layout demonstration", x_range=(-1.1, 1.1),
              y_range=(-1.1, 1.1), tools="", toolbar_location=None)

graph = GraphRenderer()

# replace the node glyph with an ellipse
# set its height, width, and fill_color
graph.node_renderer.glyph = Ellipse(height=0.1, width=0.2,
                                    fill_color="fill_color")

# assign a palette to ``fill_color`` and add it to the data source
graph.node_renderer.data_source.data = dict(
    index=node_indices,
    fill_color=Spectral8)

# add the rest of the assigned values to the data source
graph.edge_renderer.data_source.data = dict(
    start=[0] * N,
    end=node_indices)

# generate ellipses based on the ``node_indices`` list
circ = [i * 2 * math.pi / 8 for i in node_indices]

# create lists of x- and y-coordinates
x = [math.cos(i) for i in circ]
y = [math.sin(i) for i in circ]

# convert the ``x`` and ``y`` lists into a dictionary of 2D-coordinates
# and assign each entry to a node on the ``node_indices`` list
graph_layout = dict(zip(node_indices, zip(x, y)))

# use the provider model to supply coourdinates to the graph
graph.layout_provider = StaticLayoutProvider(graph_layout=graph_layout)

# render the graph
plot.renderers.append(graph)

curdoc().add_root(plot)

Complete output from server on error:

 message: Message 'PATCH-DOC' content: {'events': [{'kind': 'ModelChanged', 'model': {'id': '1022'}, 'attr': 'edge_renderer', 'new': {'id': '1034'}}], 'references': []}
 error: DeserializationError("Instance(GlyphRenderer) failed to deserialize reference to {'id': '1034'}")
Traceback (most recent call last):
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/server/protocol_handler.py", line 97, in handle
    work = await handler(message, connection)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/server/session.py", line 93, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/server/session.py", line 287, in _handle_patch
    message.apply_to_document(self.document, self)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/protocol/messages/patch_doc.py", line 115, in apply_to_document
    invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.content, setter))
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/callbacks.py", line 408, in invoke_with_curdoc
    return f()
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/protocol/messages/patch_doc.py", line 115, in <lambda>
    invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.content, setter))
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/document.py", line 391, in apply_json_patch
    DocumentPatchedEvent.handle_json(self, event_json, references, setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/events.py", line 259, in handle_json
    handler(doc, event_json, references, setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/events.py", line 467, in _handle_json
    patched_obj.set_from_json(attr, value, models=references, setter=setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/core/has_props.py", line 384, in set_from_json
    descriptor.set_from_json(self, json, models=models, setter=setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/core/property/descriptors.py", line 376, in set_from_json
    value = self.property.prepare_value(obj, self.name, self.property.from_json(json, models=models))
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/core/property/instance.py", line 112, in from_json
    raise DeserializationError(f"{self} failed to deserialize reference to {json}")
bokeh.core.property.bases.DeserializationError: Instance(GlyphRenderer) failed to deserialize reference to {'id': '1034'}
2021-11-13 10:06:56,989 error handling message
 message: Message 'PATCH-DOC' content: {'events': [{'kind': 'ModelChanged', 'model': {'id': '1022'}, 'attr': 'node_renderer', 'new': {'id': '1026'}}], 'references': []}
 error: DeserializationError("Instance(GlyphRenderer) failed to deserialize reference to {'id': '1026'}")
Traceback (most recent call last):
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/server/protocol_handler.py", line 97, in handle
    work = await handler(message, connection)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/server/session.py", line 93, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/server/session.py", line 287, in _handle_patch
    message.apply_to_document(self.document, self)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/protocol/messages/patch_doc.py", line 115, in apply_to_document
    invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.content, setter))
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/callbacks.py", line 408, in invoke_with_curdoc
    return f()
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/protocol/messages/patch_doc.py", line 115, in <lambda>
    invoke_with_curdoc(doc, lambda: doc.apply_json_patch(self.content, setter))
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/document.py", line 391, in apply_json_patch
    DocumentPatchedEvent.handle_json(self, event_json, references, setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/events.py", line 259, in handle_json
    handler(doc, event_json, references, setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/document/events.py", line 467, in _handle_json
    patched_obj.set_from_json(attr, value, models=references, setter=setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/core/has_props.py", line 384, in set_from_json
    descriptor.set_from_json(self, json, models=models, setter=setter)
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/core/property/descriptors.py", line 376, in set_from_json
    value = self.property.prepare_value(obj, self.name, self.property.from_json(json, models=models))
  File "/home/audun/.cache/pypoetry/virtualenvs/lifting-app-MZN96enu-py3.9/lib/python3.9/site-packages/bokeh/core/property/instance.py", line 112, in from_json
    raise DeserializationError(f"{self} failed to deserialize reference to {json}")
bokeh.core.property.bases.DeserializationError: Instance(GlyphRenderer) failed to deserialize reference to {'id': '1026'}

Console output in browser

The following two error messages are logged in the browser

[bokeh] Unhandled ERROR reply to D736BF40C061492A96D194593A19F4B9: DeserializationError("Instance(GlyphRenderer) failed to deserialize reference to {'id': '1034'}")

[bokeh] Unhandled ERROR reply to EBD938D9EEB04488A0CFC23CBDA7667F: DeserializationError("Instance(GlyphRenderer) failed to deserialize reference to {'id': '1026'}")

[g-parki]

This traces back to #11547. Thanks for the detailed bug report @agrav

[mattpap]

This appears to work in branch-3.0, though it needs to be confirmed that the underlying issue is actually fixed.

[philippjfr]

Can we backport whatever the fix was to 2.4.2 or apply some other fix? Seems like an important regression.

[mattpap]

Technically this is a duplicate of issue #11422 with is fixed by PR #11469. However, there is another issue to be resolved, that was introduced by PR #11547 (unnecessary reassignment). Things "work" on branch-3.0, because of PR #10623.