JSON library unable to provide index during parse operations.

[smaudet]

Feature or enhancement

Proposal:

import json

def some_hook_logic(pairs: []):
    # some logic here

if __name__ == '__main__':
    with open('some.json', 'r') as fp:
        json.load(fp, object_pairs_hook=some_hook_logic)

Currently hooks are provided to allow for some custom processing of json files. However, there is no way to obtain the index of the hooked operation. This would be especially useful for providing validation feedback on files, but there may be other benefits to providing this information to the hooks.

JSONDecoder has a raw_decode method on it that can be overwritten, which could be passed the index, however currently this parameter is essentially unused, and the index is available only to the scanner implementation (which can either be pure python or the _json.c module.

The proposal/feature is to add the index as a parameter to the hooks, or make it available as a part of JSONDecoder, possibly as new index-aware hooks so as to leave existing code interoperable.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[abstractedfox]

By index do you mean the last index read from the file?

(Also, if anyone can answer, is this something I can try to do as a new contributor?)

[encukou]

Hello!
Could you discuss new ideas on Discourse, as the issue template says?

Some variant of this could be a good idea, but as they say, “the devil is in the details”.
Is “index” a byte offset, codepoint offset, or a line:column pair? Which of that is useful, and how can the user convert to the other one(s)?

As a new contributor, go ahead, but be aware that at this point, this is an API design issue. The next step is opening the discussion.

[smaudet]

@encukou Sure I can provide some discussion. Some discussion was already head, albeit in an irc channel.

@abstractedfox Yes, "index" here meaning codepoint index (of the parsed string, not the index of the parsed set of bytes).