Dict keys

[FoxKeys]

Have a big trouble with dict keys.
Now, only unicode-string keys supported (int keys converts to unicode string and remains unicode string after decode). So, dict's after encode/decode change the type of keys. That erroneously in much applications.

[davvid]

Can you test the latest changes in the jsonpickle "dictkeys" branch?

eoghanmurray had a fork that implemented support for dictionaries (and consequently, other objects) some time ago. I was going to get back to looking into the issue and had some time tonight. I merged his changes and fixed up a few things.

Let me know if it also consequently makes int keys work for you. There's other options available such as installing a custom handler for your object's class or implementing getstate and setstate.

Let me know about the branch though, curious to see if it works for you.

BTW Can you write a simple testcase that shows your particular use case? (I wouldn't want to break it in the future).

Thanks for the report.

[FoxKeys]

Heh... "dictkeys" branch seems works fine!

And, yes, i will do test case.
But this will require some time (day, may be two) - for code extract and
cleanup.

[FoxKeys]

Seems, appropriate test already exists: "test_non_string_dict_keys"

[davvid]

Thank you! Great. I'll start a small thread on the jsonpickle mailing list with a heads up that we'll be merging this branch soon.

[Dieterbe]

Hi, what's the status of this?

[FoxKeys]

Hi. It was a looong time ago.
The project where i'm use jsonpickle was finished more than one year
ago. So, can't remember details. But it works :-)

[Dieterbe]

That's odd. I just started using jsonpickle (latest version, 3.1) and I'm experiencing this bug where if i serialize a dict with key 1, i need to use '1' (i.e. as string, not as int) after unserializing. as int, the key is not found.

I.e. it seems that the branch with the fix was never merged into master.

[davvid]

Dieterbe -- the "dictkeys" branch was about being able to have dictionaries as keys in a dictionary. That required special work since that is inherently un-json-like. The problem you're facing is that int keys are also unlike json and we don't really have a general solution for it.

The "dictkeys" branch felt a little hacky which is why it hasn't been merged. There's a certain point where the impedance mismatch between python and json becomes too large and trying to bridge them is harmful to performance or simplicity of the code.

I don't remember whether that branch would address this topic, but it might. Does that branch allow you to have ints as dictionary keys? I agree that such a data structure is a very common one so I understand the desire to support it. Perhaps it would be okay if the code was as careful and simple as possible. I remember the code introduced a (non-import-time) circular dependency which I wanted to try and unravel.

[FoxKeys]

Hi. One thing i remember - my code based on "dictkeys" branch works fine.
Unfortunately, now i have no this code within striking distance. I try to find it on my work PC on monday (19/16/2011)

[Dieterbe]

@david, well I mentioned it here because the OP described Now, only unicode-string keys supported (int keys converts to unicode string and remains unicode string after decode). so I thought this would be the appropriate place for all "my key type is not supported" issues.

[davvid]

Yes, this is the right place. I just reworked the jsonpickle format in a way that we can build upon for later supporting more object types as keys. I'll need to think about the most maintainable and simple way to do it. There have been a few slightly-working methods in the past but they were each a little hacky, and I'd like jsonpickle to be robust and simple whenever possible.

[jaraco]

I encountered this bug in our application recently. I would really like to see a fix for it, even if it is hacky. At the very least, jsonpickle should raise a warning or even refuse to encode something that it can't store rather than translate it silently to something it can.