Binary support in JSON

[oberstet]

WAMP defines an extension to standard JSON that allows transparent support for binary value:

https://wamp-proto.org/_static/gen/wamp_latest_ietf.html#rfc.section.15

AutobahnPython and AutobahnJS support that - would be great to have it in connectanum-dart as well. It's pretty useful, and (in practice) doesn't cause issues (in most host languages)

[konsultaner]

Do you mean that in conjunction with wamp-proto/wamp-proto#228 ? Or in general?

[konsultaner]

I'll implement it in general, just saw that I forgot about transparent payload as well 🤯

[oberstet]

No, in general. Payload transparency is a mode where the whole args/kwargs is not a structured object, but itself just a binary value opaque to the router. Hence the name "payload transparency".

JSON binary supports allow you to eg do a call with

args = ["foo", 23, b'\xc0\xfc\xde\x84']

or

kwargs = {"foo": 23, "bar": b'\xc0\xfc\xde\x84'}

Hence the JSON serializer/deserializer must be able to traverse/process binary values deep inside structured app payload.

In AutobahnPython, this is implemented by overriding the "string item processing" hook on the standard JSON serializer/deserializer:

https://github.com/crossbario/autobahn-python/blob/eda067aeb5cf06f88f548a10ba05383c431247f8/autobahn/wamp/serializer.py#L382

In general, this requires that the JSON serializer/deserializer has hooks to do such things ... or it requires a full custom one

[konsultaner]

Yes I Did it that way 😊

[konsultaner]

I already pushed and released it

[oberstet]

^ ok, I tested it: seems to work!

with JSON (sever side), notice 'protocol': 'wamp.2.json':

network       | 2021-03-21T00:02:29+0000 [Router        106] Authenticator.authenticate(realm="xbrnetwork", authid="None", details={'transport': {'type': 'websocket', 'protocol': 'wamp.2.json', 'peer': 'tcp4:213.170.219.39:27618', 'http_headers_received': {'user-agent': 'Dart/2.13 (dart:io)', 'connection': 'Upgrade', 'cache-control': 'no-cache', 'accept-encoding': 'gzip', 'content-length': '0', 'sec-websocket-version': '13', 'host': 'thingcloud-box-aws.sthngs.crossbario.com:8090', 'sec-websocket-extensions': 'permessage-deflate; client_max_window_bits', 'sec-websocket-key': 'lWDsHf0VFQy7IHyQIs3Mxg==', 'sec-websocket-protocol': 'wamp.2.json', 'upgrade': 'websocket'}, 'http_headers_sent': {}, 'websocket_extensions_in_use': [{'extension': 'permessage-deflate', 'is_server': True, 'server_no_context_takeover': False, 'client_no_context_takeover': False, 'server_max_window_bits': 13, 'client_max_window_bits': 13, 'mem_level': 5}], 'cbtid': None, 'channel_id': '0000000000000000000000000000000000000000000000000000000000000000'}, 'session': 8036318037757571, 'authmethod': 'cryptosign', 'authextra': {'pubkey': 'f878b6f07db66bdcc64ed6f5c8d7251100355d95da2210ab14c3f7c6f589aeb2', 'channel_binding': None}, 'authid': None, 'authrole': None})
network       | 2021-03-21T00:02:29+0000 [Router        106] Authenticator.authenticate(..) => {'pubkey': 'f878b6f07db66bdcc64ed6f5c8d7251100355d95da2210ab14c3f7c6f589aeb2', 'realm': 'xbrnetwork', 'authid': 'member-93538308-8a43-43c5-a411-ae3b4bdd5bd7', 'role': 'member', 'extra': None, 'cache': True}
network       | 2021-03-21T00:02:29+0000 [Router        106] attached session 8036318037757571 to realm "xbrnetwork" (authid="member-93538308-8a43-43c5-a411-ae3b4bdd5bd7", authrole="member") <crossbar.router.router.Router.attach>

with MsgPack, 'protocol': 'wamp.2.msgpack':

network       | 2021-03-21T00:03:13+0000 [Router        106] Authenticator.authenticate(realm="xbrnetwork", authid="None", details={'transport': {'type': 'websocket', 'protocol': 'wamp.2.msgpack', 'peer': 'tcp4:213.170.219.39:27710', 'http_headers_received': {'user-agent': 'Dart/2.13 (dart:io)', 'connection': 'Upgrade', 'cache-control': 'no-cache', 'accept-encoding': 'gzip', 'content-length': '0', 'sec-websocket-version': '13', 'host': 'thingcloud-box-aws.sthngs.crossbario.com:8090', 'sec-websocket-extensions': 'permessage-deflate; client_max_window_bits', 'sec-websocket-key': 'h/94NSyeCc691z4vJ8wRPQ==', 'sec-websocket-protocol': 'wamp.2.msgpack', 'upgrade': 'websocket'}, 'http_headers_sent': {}, 'websocket_extensions_in_use': [{'extension': 'permessage-deflate', 'is_server': True, 'server_no_context_takeover': False, 'client_no_context_takeover': False, 'server_max_window_bits': 13, 'client_max_window_bits': 13, 'mem_level': 5}], 'cbtid': None, 'channel_id': '0000000000000000000000000000000000000000000000000000000000000000'}, 'session': 925287054320347, 'authmethod': 'cryptosign', 'authextra': {'pubkey': 'f878b6f07db66bdcc64ed6f5c8d7251100355d95da2210ab14c3f7c6f589aeb2', 'channel_binding': None}, 'authid': None, 'authrole': None})
network       | 2021-03-21T00:03:13+0000 [Router        106] Authenticator.authenticate(..) => {'pubkey': 'f878b6f07db66bdcc64ed6f5c8d7251100355d95da2210ab14c3f7c6f589aeb2', 'realm': 'xbrnetwork', 'authid': 'member-93538308-8a43-43c5-a411-ae3b4bdd5bd7', 'role': 'member', 'extra': None, 'cache': True}
network       | 2021-03-21T00:03:13+0000 [Router        106] attached session 925287054320347 to realm "xbrnetwork" (authid="member-93538308-8a43-43c5-a411-ae3b4bdd5bd7", authrole="member") <crossbar.router.router.Router.attach>