Skip to content
This repository has been archived by the owner on Jun 3, 2023. It is now read-only.

None keys don't get serialised to transit #38

Open
nblumoe opened this issue Oct 1, 2021 · 0 comments
Open

None keys don't get serialised to transit #38

nblumoe opened this issue Oct 1, 2021 · 0 comments

Comments

@nblumoe
Copy link

nblumoe commented Oct 1, 2021

Description

Python dictionaries that use None as a key, cannot be serialised to transit.
The Clojure implementation in contrast, deals with nil as key without any issues.

Example and reproduction

Here is a minimal example:

from transit.writer import Writer
from io import StringIO

io = StringIO()
writer = Writer(io, "json")

val = {None: 42}
writer.write(val)

It will throw this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/var/folders/rm/7dw9bw5d5pj7lg41n5j77mww0000gn/T/ipykernel_34111/3646707664.py in <module>
      8 val = {None: 42}
      9 print(val)
---> 10 writer.write(val)

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in write(self, obj)
     59         the 'io' source.
     60         """
---> 61         self.marshaler.marshal_top(obj)
     62 
     63     def register(self, obj_type, handler_class):

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in marshal_top(self, obj, cache)
    222                 self.marshal(TaggedValue(QUOTE, obj), False, cache)
    223             else:
--> 224                 self.marshal(obj, False, cache)
    225             self.flush()
    226         else:

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in marshal(self, obj, as_map_key, cache)
    203 
    204         if f:
--> 205             f(self, obj, handler.string_rep(obj) if as_map_key else handler.rep(obj), as_map_key, cache)
    206         else:
    207             self.emit_encoded(tag, handler, obj, as_map_key, cache)

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in <lambda>(self, obj, rep, as_map_key, cache)
    252                     "'": lambda self, obj, rep, _, cache: self.emit_tagged("'", rep, cache),
    253                     "array": lambda self, obj, rep, as_map_key, cache: self.emit_array(rep, as_map_key, cache),
--> 254                     "map": lambda self, obj, rep, as_map_key, cache: self.dispatch_map(rep, as_map_key, cache)}
    255 
    256 

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in dispatch_map(self, rep, as_map_key, cache)
    233         """
    234         if self.are_stringable_keys(rep):
--> 235             return self.emit_map(rep, as_map_key, cache)
    236         return self.emit_cmap(rep, as_map_key, cache)
    237 

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in emit_map(self, m, _, cache)
    356         self.marshal(MAP_AS_ARR, False, cache)
    357         for k, v in m.items():
--> 358             self.marshal(k, True, cache)
    359             self.marshal(v, False, cache)
    360         self.emit_array_end()

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in marshal(self, obj, as_map_key, cache)
    203 
    204         if f:
--> 205             f(self, obj, handler.string_rep(obj) if as_map_key else handler.rep(obj), as_map_key, cache)
    206         else:
    207             self.emit_encoded(tag, handler, obj, as_map_key, cache)

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in <lambda>(self, obj, rep, as_map_key, cache)
    244         self.handlers[obj_type] = handler_class
    245 
--> 246 marshal_dispatch = {"_": lambda self, obj, rep, as_map_key, cache: self.emit_nil(rep, as_map_key, cache),
    247                     "?": lambda self, obj, rep, as_map_key, cache: self.emit_boolean(rep, as_map_key, cache),
    248                     "s": lambda self, obj, rep, as_map_key, cache: self.emit_string("", "", escape(rep), as_map_key, cache),

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in emit_nil(self, _, as_map_key, cache)
    120 
    121     def emit_nil(self, _, as_map_key, cache):
--> 122         return self.emit_string(ESC, "_", None, True, cache) if as_map_key else self.emit_object(None)
    123 
    124     def emit_string(self, prefix, tag, string, as_map_key, cache):

~/miniconda3/envs/dummy-env/lib/python3.7/site-packages/transit/writer.py in emit_string(self, prefix, tag, string, as_map_key, cache)
    123 
    124     def emit_string(self, prefix, tag, string, as_map_key, cache):
--> 125         encoded = cache.encode(str(prefix)+tag+string, as_map_key)
    126         # TODO: Remove this optimization for the time being - it breaks cache
    127         #if "cache_enabled" in self.opts and is_cacheable(encoded, as_map_key):

TypeError: can only concatenate str (not "NoneType") to str

Comment

The issue surfaced when reading and writing transit encoded EDN that works flawlessly with Clojure, but not Python. The original EDN includes maps which use nil as a key. This doesn't lead to any problems when processing the data with Clojure, but it does with Python.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant