Skip to content
This repository

notepad + jsonlib: TypeError: Only whitespace may be used for indentation. #1037

Closed
wolever opened this Issue November 24, 2011 · 8 comments

3 participants

David Wolever Fernando Perez Min RK
David Wolever

When jsonlib is installed, the notepad crashes with a TypeError:

Traceback (most recent call last):
...
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/kernelmanager.py", line 752, in start_kernel
    self.write_connection_file()
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/kernelmanager.py", line 716, in write_connection_file
    shell_port=self.shell_port, hb_port=self.hb_port)
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/entry_point.py", line 86, in write_connection_file
    f.write(json.dumps(cfg, indent=2))
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/zmq/utils/jsonapi.py", line 67, in jsonlib_dumps
    return _squash_unicode(jsonmod.dumps(o,**kwargs))
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/jsonlib.py", line 799, in write
    return write_impl (value, sort_keys, validate_indent (indent), ascii_only,
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/jsonlib.py", line 812, in validate_indent
    raise TypeError ("Only whitespace may be used for indentation.")
TypeError: Only whitespace may be used for indentation.

It looks like this is because jsonlib expects the indent argument to be string, while the stdlib's json expects indent to be a number. The offending code is below:

> /Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/entry_point.py(86)write_connection_file()
-> f.write(json.dumps(cfg, indent=2))
(Pdb) list
 81                   )
 82         cfg['ip'] = ip
 83         cfg['key'] = bytes_to_str(key)
 84         
 85         with open(fname, 'wb') as f:
 86  ->         f.write(json.dumps(cfg, indent=2))
 87         
 88         return fname, cfg
 89         
 90     
 91     def base_launch_kernel(code, fname, stdin=None, stdout=None, stderr=None,

The above is Python 2.6.7 on OS X 10.7 with jsonlib 1.6.1.

Uninstalling jsonlib (presumably forcing a fallback to stdlib json?) fixes this issue.

Fernando Perez
Owner

Yup, good catch. @minrk, we should probably work around this one in ipython regardless, I'll make the fixes now so that we support jsonlib as well. But ultimately it seems to me the deeper fix should be made in zmq.jsonapi, right?

Fernando Perez fperez closed this issue from a commit November 24, 2011
Fernando Perez Work around incompatibilities between jsonlib and json.
Note that there seems to be a deeper problem with jsonlib; at least on
my system it doesn't pass the full test suite.  But with this fix,
regular interactive use is now OK.

Closes #1037.
8cacf9c
Fernando Perez fperez closed this in 8cacf9c November 24, 2011
Fernando Perez
Owner

OK, I've fixed it quickly, but I noticed that with jsonlib installed, the full test suite doesn't quite complete.

@minrk, I had to manually interrupt it at:

test_retries (IPython.parallel.tests.test_lbview.TestLoadBalancedView) ... ^CTraceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/iptest.py", line 313, in run
    return self._run_cmd()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/iptest.py", line 306, in _run_cmd
    retcode = subp.wait()
  File "/usr/lib/python2.7/subprocess.py", line 1281, in wait
    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
  File "/usr/lib/python2.7/subprocess.py", line 478, in _eintr_retry_call
    return func(*args)
KeyboardInterrupt

I don't know if you want to consider that an issue for us, for zmq or for jsonlib, though, so I won't open another one quite yet. If you think it's on our side, then let's open a new issue for it.

Min RK
Owner
Fernando Perez
Owner

Ok, thanks. In the meantime, the workaround I committed means people won't get burned too badly if they have jsonlib installed; I'll leave the decision of what to do in zmq up to you (though dropping jsonlib support seems like the right approach: I looked and the api is so different that any but the most trivial uses are likely to cause problems similar to this one).

Min RK
Owner
Fernando Perez
Owner
Min RK
Owner

The main point of the zmq.utils.jsonapi is that it serializes to/from utf8-encoded bytes, and json libraries are inconsistent in whether they deal in unicode or str. If that's not what we need (e.g. talking to files), we should be using the stdlib json. And the fact that jsonlib does not guarantee kwarg equivalence means that anyone using jsonapi with kwargs must take this into account (jsonapi.jsonmod.__name__ is the appropriate value to check).

The practical result of this is that we (IPython) should essentially never pass kwargs when using zmq.jsonapi. There is exactly one case where we are currently using a kwarg where we should be using zmq.jsonapi at all, and it has always properly handled the difference in jsonlib. The real bug here is that we were using jsonapi in places that don't make any sense.

I did find the bug with jsonlib in the test suite - it unserializes floats to decimal.Decimal (?!), so I have to cast it to float before passing it to datetime.timedelta. There are few cases where the difference will matter (and exactly one case where a float is ever sent via json), but we will have to be wary if we work with floats while jsonlib is a possible actor.

This may be the last straw for dropping jsonlib support in pyzmq, but that will be irrelevant to IPython until we bump minimum pyzmq version past 2.1.11.

A monkeypatch for prohibiting pyzmq from using jsonlib, and forcing fallback to stdlib:

import json
from zmq.utils import jsonapi

if jsonapi.jsonmod.__name__ == 'jsonlib':
    jsonapi.jsonmod =  json

But this may not be ideal, as jsonlib is significantly faster than stdlib, and with the small fix in PR #1040, well behaved.

Fernando Perez
Owner

OK, I like the approach in #1040, so let's go with that. Thanks!

Fernando Perez fperez referenced this issue from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.