[BUG] Serialization of MessageSent document event includes all model references

[matbryan52]

Observed a drop in performance while upgrading from release 2.2.3 to 2.3.0, on a page using MouseMove events to update a line plot. Fast mouse movement causes events to 'queue' and complete after mouse has stopped.

Issue discussed on Discourse at https://discourse.bokeh.org/t/mousemove-events-queuing-up-after-2-3-0-update/7361.

From @mateusz on Discourse:

Currently serialization of MessageSent document event includes all model references this event’s model field refers to, whereas it should include only previously unknown models. This is a regression since PR #10602.

An MWE is available on the Discourse thread.

Versions:
Bokeh 2.3.0 (upgrading from 2.2.3)
Python 3.9.2 on Ubuntu 20.04 / Firefox 86.0

Video of behaviour:

vokoscreenNG-2021-03-08_23-01-29.mp4

[bryevdv]

@mattpap we should make sure this can go in a 2.3.1 release next week. Pretty sure this is also affecting cuxfilter cc @AjayThorve

[AjayThorve]

@mattpap we should make sure this can go in a 2.3.1 release next week. Pretty sure this is also affecting cuxfilter cc @AjayThorve

yep, getting a huge log of events which slows down the performance for lasso-select and box-select events with datashader charts, and infact getting a deserialization error at the end:

 error: DeserializationError('Float expected Real, got None of type NoneType')
Traceback (most recent call last):
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/server/protocol_handler.py", line 90, in handle
    work = await handler(message, connection)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/server/session.py", line 67, in _needs_document_lock_wrapper
    result = func(self, *args, **kwargs)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/server/session.py", line 261, in _handle_patch
    message.apply_to_document(self.document, self)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 100, in apply_to_document
    doc._with_self_as_curdoc(lambda: doc.apply_json_patch(self.content, setter))
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/document/document.py", line 1198, in _with_self_as_curdoc
    return f()
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/protocol/messages/patch_doc.py", line 100, in <lambda>
    doc._with_self_as_curdoc(lambda: doc.apply_json_patch(self.content, setter))
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/document/document.py", line 394, in apply_json_patch
    initialize_references_json(references_json, references, setter)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/document/util.py", line 88, in initialize_references_json
    instance.update_from_json(obj_attrs, models=references, setter=setter)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/core/has_props.py", line 469, in update_from_json
    self.set_from_json(k, v, models, setter)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/core/has_props.py", line 411, in set_from_json
    descriptor.set_from_json(self, json, models, setter)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/core/property/descriptors.py", line 623, in set_from_json
    return super().set_from_json(obj, self.property.from_json(json, models), models, setter)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/core/property/bases.py", line 472, in from_json
    return self.type_param.from_json(json, models=models)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/core/property/bases.py", line 472, in from_json
    return self.type_param.from_json(json, models=models)
  File "/anaconda3/envs/rapids-0.19/lib/python3.8/site-packages/bokeh/core/property/bases.py", line 520, in from_json
    raise DeserializationError(msg)
bokeh.core.property.bases.DeserializationError: Float expected Real, got None of type NoneType

[mattpap]

bokeh.core.property.bases.DeserializationError: Float expected Real, got None of type NoneType

This is most likely unrelated and a side effect of making properties non-nullable by default in 2.3. I need to add model/attr information to those deserialization errors, because currently I don't know whether you're setting a bokeh's property with incorrect type of value, or you need to update your properties to make them nullable where needed. We already did that for validation errors.

[philippjfr]

Probably not related but it seems data models are also all sent and/or registered on each render:

Do you want an issue for that?

[mattpap]

Yes please. I guess I didn't test that with server.

[bjarthur]

here is another MWE. vastly slower in 2.3 vs 2.2. great to hear this might be fixed in 2.3.1. thanks!

from bokeh.models import ColumnDataSource
from bokeh.plotting import figure, curdoc
from bokeh.events import PanStart, Pan

p = figure()
p.toolbar.active_drag = None
p.circle([1, 2, 3, 4, 5], [6, 7, 2, 4, 5], size=20, color="navy", alpha=0.5)
quad = ColumnDataSource(data=dict(left=[], right=[], bottom=[], top=[]))
p.quad(left='left', right='right', top='top', bottom='bottom', source=quad, fill_color="black")

def cb_panstart(evt):
    quad.data.update(left=[evt.x], right=[evt.x], bottom=[evt.y], top=[evt.y])

def cb_pan(evt):
    quad.data.update(right=[evt.x], top=[evt.y])

p.on_event(PanStart, cb_panstart)
p.on_event(Pan, cb_pan)

doc = curdoc()
doc.add_root(p)