notepad + jsonlib: TypeError: Only whitespace may be used for indentation. #1037

wolever opened this Issue Nov 24, 2011 · 8 comments

3 participants


When jsonlib is installed, the notepad crashes with a TypeError:

Traceback (most recent call last):
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/", line 752, in start_kernel
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/", line 716, in write_connection_file
    shell_port=self.shell_port, hb_port=self.hb_port)
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/", line 86, in write_connection_file
    f.write(json.dumps(cfg, indent=2))
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/zmq/utils/", line 67, in jsonlib_dumps
    return _squash_unicode(jsonmod.dumps(o,**kwargs))
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/", line 799, in write
    return write_impl (value, sort_keys, validate_indent (indent), ascii_only,
  File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/", line 812, in validate_indent
    raise TypeError ("Only whitespace may be used for indentation.")
TypeError: Only whitespace may be used for indentation.

It looks like this is because jsonlib expects the indent argument to be string, while the stdlib's json expects indent to be a number. The offending code is below:

> /Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/
-> f.write(json.dumps(cfg, indent=2))
(Pdb) list
 81                   )
 82         cfg['ip'] = ip
 83         cfg['key'] = bytes_to_str(key)
 85         with open(fname, 'wb') as f:
 86  ->         f.write(json.dumps(cfg, indent=2))
 88         return fname, cfg
 91     def base_launch_kernel(code, fname, stdin=None, stdout=None, stderr=None,

The above is Python 2.6.7 on OS X 10.7 with jsonlib 1.6.1.

Uninstalling jsonlib (presumably forcing a fallback to stdlib json?) fixes this issue.

IPython member

Yup, good catch. @minrk, we should probably work around this one in ipython regardless, I'll make the fixes now so that we support jsonlib as well. But ultimately it seems to me the deeper fix should be made in zmq.jsonapi, right?

@fperez fperez added a commit that closed this issue Nov 24, 2011
@fperez fperez Work around incompatibilities between jsonlib and json.
Note that there seems to be a deeper problem with jsonlib; at least on
my system it doesn't pass the full test suite.  But with this fix,
regular interactive use is now OK.

Closes #1037.
@fperez fperez closed this in 8cacf9c Nov 24, 2011
IPython member

OK, I've fixed it quickly, but I noticed that with jsonlib installed, the full test suite doesn't quite complete.

@minrk, I had to manually interrupt it at:

test_retries (IPython.parallel.tests.test_lbview.TestLoadBalancedView) ... ^CTraceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/", line 313, in run
    return self._run_cmd()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/", line 306, in _run_cmd
    retcode = subp.wait()
  File "/usr/lib/python2.7/", line 1281, in wait
    pid, sts = _eintr_retry_call(os.waitpid,, 0)
  File "/usr/lib/python2.7/", line 478, in _eintr_retry_call
    return func(*args)

I don't know if you want to consider that an issue for us, for zmq or for jsonlib, though, so I won't open another one quite yet. If you think it's on our side, then let's open a new issue for it.

IPython member
IPython member

Ok, thanks. In the meantime, the workaround I committed means people won't get burned too badly if they have jsonlib installed; I'll leave the decision of what to do in zmq up to you (though dropping jsonlib support seems like the right approach: I looked and the api is so different that any but the most trivial uses are likely to cause problems similar to this one).

IPython member
IPython member
IPython member

The main point of the zmq.utils.jsonapi is that it serializes to/from utf8-encoded bytes, and json libraries are inconsistent in whether they deal in unicode or str. If that's not what we need (e.g. talking to files), we should be using the stdlib json. And the fact that jsonlib does not guarantee kwarg equivalence means that anyone using jsonapi with kwargs must take this into account (jsonapi.jsonmod.__name__ is the appropriate value to check).

The practical result of this is that we (IPython) should essentially never pass kwargs when using zmq.jsonapi. There is exactly one case where we are currently using a kwarg where we should be using zmq.jsonapi at all, and it has always properly handled the difference in jsonlib. The real bug here is that we were using jsonapi in places that don't make any sense.

I did find the bug with jsonlib in the test suite - it unserializes floats to decimal.Decimal (?!), so I have to cast it to float before passing it to datetime.timedelta. There are few cases where the difference will matter (and exactly one case where a float is ever sent via json), but we will have to be wary if we work with floats while jsonlib is a possible actor.

This may be the last straw for dropping jsonlib support in pyzmq, but that will be irrelevant to IPython until we bump minimum pyzmq version past 2.1.11.

A monkeypatch for prohibiting pyzmq from using jsonlib, and forcing fallback to stdlib:

import json
from zmq.utils import jsonapi

if jsonapi.jsonmod.__name__ == 'jsonlib':
    jsonapi.jsonmod =  json

But this may not be ideal, as jsonlib is significantly faster than stdlib, and with the small fix in PR #1040, well behaved.

IPython member

OK, I like the approach in #1040, so let's go with that. Thanks!

@mattvonrocketstein mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014
@fperez fperez Work around incompatibilities between jsonlib and json.
Note that there seems to be a deeper problem with jsonlib; at least on
my system it doesn't pass the full test suite.  But with this fix,
regular interactive use is now OK.

Closes #1037.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment