Change the binary types for custom serializers to memoryview, bytearr…

…ay, or python 3 bytes.

We disregard the python 2 buffer type - it is obsolete, even on python 2.

See #1194

docs/source/changelog.md

@@ -37,15 +37,16 @@ Major user-visible changes in ipywidgets 7.0 include:
 
 Major changes developers should be aware of include:
 
-- Custom serializers in either Python or Javascript can now return a structure which contains binary buffers. If a binary buffer is in the serialized data structure, the message will be synced in binary, which is much more efficient. ([#1194](https://github.com/jupyter-widgets/ipywidgets/pull/1194))
-- On the python/kernel side:
-  - The python `@register` decorator for widget classes no longer takes a string argument, but registers a widget class using the `_model_*` and `_view_*` traits in the class. Using the decorator as `@register('name')` is deprecated and should be changed to just `@register`. [#1228](https://github.com/jupyter-widgets/ipywidgets/pull/1228), [#1276](https://github.com/jupyter-widgets/ipywidgets/pull/1276)
+- On the Python/kernel side:
+  - The Python `@register` decorator for widget classes no longer takes a string argument, but registers a widget class using the `_model_*` and `_view_*` traits in the class. Using the decorator as `@register('name')` is deprecated and should be changed to just `@register`. [#1228](https://github.com/jupyter-widgets/ipywidgets/pull/1228), [#1276](https://github.com/jupyter-widgets/ipywidgets/pull/1276)
   - Widgets will now need correct `_model_module` and `_view_module` Unicode traits defined.
   - Selection widgets now sync the index of the selected item, rather than the label. ([#1262](https://github.com/jupyter-widgets/ipywidgets/pull/1262))
-  - The python `ipywidget.domwidget.LabeledWidget` is now `ipywidget.widget_description.DescriptionWidget`, and there is a new `ipywidget.widget_description.DescriptionStyle` that lets the user set the CSS width of the description.
+  - The Python `ipywidget.domwidget.LabeledWidget` is now `ipywidget.widget_description.DescriptionWidget`, and there is a new `ipywidget.widget_description.DescriptionStyle` that lets the user set the CSS width of the description.
+  - Custom serializers can now return a structure that contains binary objects (`memoryview`, `bytearray`, or Python 3 `bytes` object). In this case, the sync message will be a binary message, which is much more efficient for binary data than base64-encoding. ([#1194](https://github.com/jupyter-widgets/ipywidgets/pull/1194), [#1595](https://github.com/jupyter-widgets/ipywidgets/pull/1595))
 - On the Javascript side:
   - The `jupyter-js-widgets` Javascript package has been split into `@jupyter-widgets/base` package (containing base widget classes, the DOM widget, and the associated layout and style classes), and the `@jupyter-widgets/controls` package (containing the rest of the Jupyter widgets controls). Authors of custom widgets will need to update to depend on `@jupyter-widgets/base` instead of `jupyter-js-widgets` (if you use a class from the controls package, you will also need to depend on `@jupyter-widgets/controls`). See the [cookie cutter](https://github.com/jupyter-widgets/widget-cookiecutter) to generate a simple example custom widget using the new packages.
   - Custom serializers in Javascript are now synchronous, and should return a snapshot of the widget state. The default serializer makes a copy of JSONable objects. ([#1270](https://github.com/jupyter-widgets/ipywidgets/pull/1270))
+  - Custom serializers can now return a structure that contains binary objects (`ArrayBuffer`, `DataView`, or a typed array such as `Int8Array`, `Float64Array`, etc.). In this case, the sync message will be a binary message, which is much more efficient for binary data than base64-encoding. ([#1194](https://github.com/jupyter-widgets/ipywidgets/pull/1194))
   - A custom serializer is given the widget instance as its second argument, and a custom deserializer is given the widget manager as its second argument.
   - The Javascript model `.id` attribute has been renamed to `.model_id` to avoid conflicting with the Backbone `.id` attribute. ([#1410](https://github.com/jupyter-widgets/ipywidgets/pull/1410))
 - Regarding widget managers and the syncing message protocol:

ipywidgets/widgets/tests/test_set_state.py

@@ -11,9 +11,6 @@
 
 from ..widget import Widget
 
-
-byte_type = bytes if PY3 else buffer
-
 #
 # First some widgets to test on:
 #
@@ -48,7 +45,7 @@ def mview_serializer(instance, widget):
     return { 'data': memoryview(instance.data) if instance.data else None }
 
 def bytes_serializer(instance, widget):
-    return { 'data': byte_type(memoryview(instance.data).tobytes()) if instance.data else None }
+    return { 'data': bytearray(memoryview(instance.data).tobytes()) if instance.data else None }
 
 def deserializer(json_data, widget):
     return DataInstance( memoryview(json_data['data']).tobytes() if json_data else None )
@@ -125,4 +122,4 @@ def test_set_state_data_truncate():
 
     # Sanity:
     nt.assert_equal(len(buffers), 1)
-    nt.assert_equal(buffers[0], byte_type(data[:20].tobytes()))
+    nt.assert_equal(buffers[0], data[:20].tobytes())

ipywidgets/widgets/widget.py

@@ -50,9 +50,9 @@ def _json_to_widget(x, obj):
 }
 
 if PY3:
-    _binary_types = (memoryview, bytes)
+    _binary_types = (memoryview, bytearray, bytes)
 else:
-    _binary_types = (memoryview, buffer)
+    _binary_types = (memoryview, bytearray)
 
 def _put_buffers(state, buffer_paths, buffers):
     """The inverse of _remove_buffers, except here we modify the existing dict/lists.
@@ -115,6 +115,8 @@ def _separate_buffers(substate, path, buffer_paths, buffers):
 def _remove_buffers(state):
     """Return (state_without_buffers, buffer_paths, buffers) for binary message parts
 
+    A binary message part is a memoryview, bytearray, or python 3 bytes object.
+
     As an example:
     >>> state = {'plain': [0, 'text'], 'x': {'ar': memoryview(ar1)}, 'y': {'shape': (10,10), 'data': memoryview(ar2)}}
     >>> _remove_buffers(state)
@@ -486,8 +488,8 @@ def _compare(self, a, b):
     def _buffer_list_equal(self, a, b):
         """Compare two lists of buffers for equality.
 
-        Used to decide whether two sequences of buffers (memoryviews) differ,
-        such that a sync is needed.
+        Used to decide whether two sequences of buffers (memoryviews,
+        bytearrays, or python 3 bytes) differ, such that a sync is needed.
 
         Returns True if equal, False if unequal
         """
@@ -501,9 +503,9 @@ def _buffer_list_equal(self, a, b):
             # e.g. memoryview(np.frombuffer(ia, dtype='float32')) !=
             # memoryview(np.frombuffer(b)), since the format info differs.
             # However, since we only transfer bytes, we use `tobytes()`.
-            iabytes = ia.tobytes() if isinstance(ia, memoryview) else ia
-            ibbytes = ib.tobytes() if isinstance(ib, memoryview) else ib
-            if ia != ib:
+            ia_bytes = ia.tobytes() if isinstance(ia, memoryview) else ia
+            ib_bytes = ib.tobytes() if isinstance(ib, memoryview) else ib
+            if ia_bytes != ib_bytes:
                 return False
         return True