add json.tool option to avoid alphabetic sort of fields

[PavelKazlou]

BPO	21650
Nosy	@rhettinger, @pitrou, @ezio-melotti, @bitdancer, @berkerpeksag, @vadmium, @serhiy-storchaka
Files	issue21650.diff issue21650_v2.diff issue21650_v3.diff issue21650_v4.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/berkerpeksag'
closed_at = <Date 2014-11-10.07:58:05.540>
created_at = <Date 2014-06-03.10:11:31.342>
labels = ['type-feature', 'library']
title = 'add json.tool option to avoid alphabetic sort of fields'
updated_at = <Date 2014-11-10.07:58:05.539>
user = 'https://bugs.python.org/PavelKazlou'

bugs.python.org fields:

activity = <Date 2014-11-10.07:58:05.539>
actor = 'berker.peksag'
assignee = 'berker.peksag'
closed = True
closed_date = <Date 2014-11-10.07:58:05.540>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2014-06-03.10:11:31.342>
creator = 'Pavel.Kazlou'
dependencies = []
files = ['35468', '35631', '37110', '37119']
hgrepos = []
issue_num = 21650
keywords = ['patch']
message_count = 18.0
messages = ['219675', '219677', '219678', '219804', '219834', '219835', '219889', '219891', '219893', '220556', '230431', '230433', '230434', '230511', '230563', '230669', '230939', '230940']
nosy_count = 9.0
nosy_names = ['rhettinger', 'pitrou', 'ezio.melotti', 'r.david.murray', 'python-dev', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'Pavel.Kazlou']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21650'
versions = ['Python 3.5']

[PavelKazlou]

Currently when you use json.tool, fields are reordered alphabetically.
In source code the value of sort_keys is hardcoded to be true.
It should be easy to expose this option as command line parameter.

[PavelKazlou]

This is the line in module I'm talking about:
json.dump(obj, outfile, sort_keys=True, indent=4)

[berkerpeksag]

Here's a patch with a test case.

[pitrou]

I don't really understand the point of this. The "unsorted" output order will be unpredictable for the user (it isn't necessarily the same as the order of fields in the input data).

[bitdancer]

It should be possible to also change the tool to use OrderDicts, though.

[bitdancer]

Or does the data get decoded to a dict *before* it gets passed to the object_hook? Probably, in which case nevermind...

[PavelKazlou]

The idea is to keep the same order as in input.

[bitdancer]

Yes but the input is turned into a dict, and dicts do not preserve order. Further, what is passed to the object_hook is already a dict, so the order is already lost before object_hook is called.

Since the parser (or at least the Python version of the parser) first turns the input into a list of pairs, and a Python OrderedDict can be instantiated from a list of pairs, it would theoretically be possible to extend the object_hook API to allow an OrderedDict to be used on decode. Exactly how to do that so as to retain backward compatibility is a bit of a question.

So, it is *possible* to achieve your aim, but it isn't as simple as allowing sort= to be set False.

IMO being able to preserve the order of the input when desired (ie: use an OrderedDict object_hook) would be a nice feature to have.

[bitdancer]

Wait, I read the code wrong.

You can define object_pairs_hook, and use that to return an OrderedDict. So it should be possible to do this without changing the json module itself. This is actually documented as a place to use OrderedDict. Guess I should have read the docs instead of the code :)

[berkerpeksag]

Updated patch attached based on feedback from David. Thanks!

[rhettinger]

To me, "--unsorted" implies arbitrary ordering such as the order generated by regular dictionaries. I think the option should be "--order-preserving" or some such. The option name needs to make it clear that the output is in the same order an the input.

As side from the option name, I'm +1 on the patch. It is very annoying to look at scrambled JSON when the original presentation order made more logical sense.

[serhiy-storchaka]

May be make sorting keys not default?

[rhettinger]

Maybe make sorting keys not default?

If you mean preserve order by default, then yes that would be a nice default.