`json`'s `default` callable/method ignores keys.

[Matiiss]

Bug report

Bug description:

It's possible to provide a default callable or override a JSONEncoder's default method to support serializing objects that by default are not supported, however, this appears to ignore the keys which is something I would expect it to not ignore (see the documentation reference below) and I don't see a reason why it shouldn't try to convert them as well if the callable can return a valid JSON representation of the key.

The documentation is somewhat vague in this regard, stating that

To extend this to recognize other objects, subclass and implement a default() method with another method that returns a serializable object for o if possible, otherwise it should call the superclass implementation (to raise TypeError).

https://docs.python.org/3/library/json.html#json.JSONEncoder

import json


def tuple_to_json(obj):
    if isinstance(obj, tuple):
        return ",".join(map(str, obj))
    return json.dumps(obj)


dct = {(1, 2): {"x": 5}}
print(json.dumps(dct, default=tuple_to_json))

This raises a TypeError:

TypeError: keys must be str, int, float, bool or None, not tuple

The expected output would be:

{"1,2": {"x": 5}}

That said, this can be implemented somewhat nicely the other way around

import json


def string_to_tuple(string: str, delimiter: str = ",", cast: type = str):
    if delimiter in string:
        return tuple(map(cast, string.split(delimiter)))
    return string


def key_to_tuple_hook(obj, delimiter: str = ",", cast: type = str):
    if isinstance(obj, dict):
        return {string_to_tuple(key, delimiter=delimiter, cast=cast): value for key, value in obj.items()}
    return obj


json_string = '{"1,2": {"3,4": 5}}'
print(json.loads(json_string, object_hook=lambda obj: key_to_tuple_hook(obj, cast=int)))

So, to sort of make it more symmetrical, either default should attempt to convert the keys or there could be another keyword parameter specifically for keys (supposedly that would also mean another method for the JSONEncoder).

CPython versions tested on:

3.12

Operating systems tested on:

Windows

[sanjitkumar2016]

The object_hook callback function takes in only strings, so a dev could handle string parsing for both key and value formats fairly easily, but the default callback function takes in any object (currently only for values) that causes the TypeError. If the suggested change was made to pass unserializable keys to the default handler, it would require a good amount of rework for devs who rely on catching the TypeError in the json.dumps call, rather than in the default handler. Plus, the default handler would now have to handle checking for multiple objects in case the key and value are different objects. I think your second suggestion of creating "another keyword parameter specifically for keys" would be better.

[gaogaotiantian]

I believe this is basically a dup for #117391

And there are some discussions about it in #117392

We can have a way to serialize keys for json, we need to have a good design though as this is a heavily used library and we want to keep good maintainability (as well as backward compatibility).

[nineteendo]

@picnixz, this is a duplicate of #63020 (and #117391).

[picnixz]

I'm going to close this one then. But I don't know which one to close for the two older issues.