Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Option to allow pretty print without alpha sorting #128

Open
patcon opened this Issue Feb 10, 2013 · 19 comments

Comments

Projects
None yet

patcon commented Feb 10, 2013

Right now, the options are colors, format, and all, but often the order of the json returned is intentional and for clarify. It would be nice to be able to pretty print the json without having it's keys sorted :)

Awesome tool, and thanks very much for everything!

Results from my introspection.

➜  httpie git:(issue-128) http http://headers.jsontest.com/
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: private
Content-Type: application/json; charset=ISO-8859-1
Date: Sat, 27 Jul 2013 07:01:24 GMT
Server: Google Frontend
Transfer-Encoding: chunked

{
    "Accept": "*/*",
    "Host": "headers.jsontest.com",
    "User-Agent": "HTTPie/0.6.0"
}

➜  httpie git:(issue-128) curl http://headers.jsontest.com/
{
   "Host": "headers.jsontest.com",
   "User-Agent": "curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5",
   "Accept": "*/*"
}

Since httpie uses requests guessing r.json() is responsible for this, trying requests.get.

In [36]: r = requests.get("http://headers.jsontest.com/")

In [37]: r.json()
Out[37]:
{u'Accept': u'*/*',
 u'Host': u'headers.jsontest.com',
 u'User-Agent': u'python-requests/1.2.3 CPython/2.7.2 Darwin/12.4.0'}

Same result. Now looking intor.json code.

In [38]: r.json??
Type:       instancemethod
String Form:<bound method Response.json of <Response [200]>>
File:       /Library/Python/2.7/site-packages/requests/models.py
Definition: r.json(self, **kwargs)
Source:
    def json(self, **kwargs):
        """Returns the json-encoded content of a response, if any.

        :param \*\*kwargs: Optional arguments that ``json.loads`` takes.
        """

        if not self.encoding and len(self.content) > 3:
            # No encoding set. JSON RFC 4627 section 3 states we should expect
            # UTF-8, -16 or -32. Detect which one to use; If the detection or
            # decoding fails, fall back to `self.text` (using chardet to make
            # a best guess).
            encoding = guess_json_utf(self.content)
            if encoding is not None:
                return json.loads(self.content.decode(encoding), **kwargs)
        return json.loads(self.text or self.content, **kwargs)

Look into requests code json is imported from compat.py, the exact code is

try:
    import simplejson as json
except ImportError:
    import json

Trying other library like ujson provides same result

n [43]: ujson.loads("""{
   "Host": "headers.jsontest.com",
   "User-Agent": "curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5",
   "Accept": "*/*"
}""")
Out[43]:
{u'Accept': u'*/*',
 u'Host': u'headers.jsontest.com',
 u'User-Agent': u'curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5'}

Since the the returned object is dictionary it isn't possible to maintain the order due to hashing. If it is possible to maintain the order it would be great.

Seems like I made a mistake. There is a JSONProcessor but still it might not solve the problem.

In [48]: r.content
Out[48]: '{\n   "Host": "headers.jsontest.com",\n   "User-Agent": "python-requests/1.2.3 CPython/2.7.2 Darwin/12.4.0",\n   "Accept": "*/*"\n}\n'

In [49]: json.loads(r.content)
Out[49]:
{u'Accept': u'*/*',
 u'Host': u'headers.jsontest.com',
 u'User-Agent': u'python-requests/1.2.3 CPython/2.7.2 Darwin/12.4.0'}

Trying with JSONProcessor

In [61]: j = JSONProcessor()

In [62]: j.process_body(r.content, r.headers['Content-Type'], subtype='json', encoding='utf8')
Out[62]: u'{\n    "Accept": "*/*", \n    "Host": "headers.jsontest.com", \n    "User-Agent": "python-requests/1.2.3 CPython/2.7.2 Darwin/12.4.0"\n}'

One way to get around the problem is use PygmentsProcessor.

In [64]: p = PygmentsProcessor()

In [65]: p.process_body(r.content, r.headers['Content-Type'], subtype='json', encoding='utf8')
Out[65]: u'\x1b[38;5;245m{\x1b[39m\n\x1b[38;5;245m   \x1b[39m\x1b[38;5;33m"Host"\x1b[39m\x1b[38;5;245m:\x1b[39m\x1b[38;5;245m \x1b[39m\x1b[38;5;37m"headers.jsontest.com"\x1b[39m\x1b[38;5;245m,\x1b[39m\n\x1b[38;5;245m   \x1b[39m\x1b[38;5;33m"User-Agent"\x1b[39m\x1b[38;5;245m:\x1b[39m\x1b[38;5;245m \x1b[39m\x1b[38;5;37m"python-requests/1.2.3 CPython/2.7.2 Darwin/12.4.0"\x1b[39m\x1b[38;5;245m,\x1b[39m\n\x1b[38;5;245m   \x1b[39m\x1b[38;5;33m"Accept"\x1b[39m\x1b[38;5;245m:\x1b[39m\x1b[38;5;245m \x1b[39m\x1b[38;5;37m"*/*"\x1b[39m\n\x1b[38;5;245m}\x1b[39m'

In [66]: print(p.process_body(r.content, r.headers['Content-Type'], subtype='json', encoding='utf8'))
{
   "Host": "headers.jsontest.com",
   "User-Agent": "python-requests/1.2.3 CPython/2.7.2 Darwin/12.4.0",
   "Accept": "*/*"
}

patcon commented Jul 27, 2013

Nice. I recently used something called an OrderedHash is ruby that solved a similar problem. Thinking that OrderedDict might be the solution in python...
http://stackoverflow.com/questions/6921699/can-i-get-json-to-load-into-an-ordereddict-in-python

@patcon That was easier solution than mine. I will modify the code accordingly and update the pull request.

patcon commented Jul 29, 2013

👍 Closing this one :)

@patcon patcon closed this Jul 29, 2013

@patcon Why?, pull request isn't merged though.

patcon commented Jul 29, 2013

Ah. Oops. Misunderstood the context of the referring issue above.

@patcon patcon reopened this Jul 29, 2013

carueda commented Oct 2, 2014

👍 veeery useful option, please merge/fix at the next convenient opportunity. If not "reasonably" soon, any idea of by when approximately?

+1
Please add this option. Sorting is a deal breaker for me.

+1
Yes, please add this option. I would like to have pretty print but without sorting the keys

Any news about this?

+1

hartym commented Sep 12, 2016

This problem is indeed misleading. I struggled a bit to understand why my (server side) OrderedDict was not correctly serialised in my app, then I finally understood that the culprit was not the server but indeed the client re-sorting the keys. It is sometimes useful to have the keys sorted, but for an HTTP client, having a default that changes the response content does not sound like the right thing.

Thanks for the amazing tool anyway ;)

Contributor

msabramo commented Feb 22, 2017

@jkbrzt: You mention a while back in jkbrzt#151 (comment) that you wanted a more generic solution to this problem; was any progress made on that?

Owner

jakubroztocil commented Feb 22, 2017

@msabramo it's still on my to do list but not progress yet

NN88 commented Jun 9, 2017 edited

+1 this really needs to become a feature.

Thanks for this amazing tool!

I wrote an HTTP client for Sublime Text that's also built on top of Requests, called Requester. I also wanted to allow users to choose whether or not keys for JSON responses are sorted, so I added a 'fmt' argument to calls to requests.

I solved it like this. If the response body is JSON, calling res.json() on the response object res returns a Python dictionary. At this point you can pass the dict to json.dumps to specify indentation and whether keys are sorted. However, if you just return res.text (equivalent to returning the raw response body) the keys won't be sorted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment