When jsonlib is installed, the notepad crashes with a TypeError:
Traceback (most recent call last):
File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/kernelmanager.py", line 752, in start_kernel
File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/kernelmanager.py", line 716, in write_connection_file
File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/IPython/zmq/entry_point.py", line 86, in write_connection_file
File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/zmq/utils/jsonapi.py", line 67, in jsonlib_dumps
File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/jsonlib.py", line 799, in write
return write_impl (value, sort_keys, validate_indent (indent), ascii_only,
File "/Users/wolever/code/sandbox/env/sandbox/lib/python2.6/site-packages/jsonlib.py", line 812, in validate_indent
raise TypeError ("Only whitespace may be used for indentation.")
TypeError: Only whitespace may be used for indentation.
It looks like this is because jsonlib expects the indent argument to be string, while the stdlib's json expects indent to be a number. The offending code is below:
-> f.write(json.dumps(cfg, indent=2))
82 cfg['ip'] = ip
83 cfg['key'] = bytes_to_str(key)
85 with open(fname, 'wb') as f:
86 -> f.write(json.dumps(cfg, indent=2))
88 return fname, cfg
91 def base_launch_kernel(code, fname, stdin=None, stdout=None, stderr=None,
The above is Python 2.6.7 on OS X 10.7 with jsonlib 1.6.1.
Uninstalling jsonlib (presumably forcing a fallback to stdlib json?) fixes this issue.
Yup, good catch. @minrk, we should probably work around this one in ipython regardless, I'll make the fixes now so that we support jsonlib as well. But ultimately it seems to me the deeper fix should be made in zmq.jsonapi, right?
Work around incompatibilities between jsonlib and json.
Note that there seems to be a deeper problem with jsonlib; at least on
my system it doesn't pass the full test suite. But with this fix,
regular interactive use is now OK.
OK, I've fixed it quickly, but I noticed that with jsonlib installed, the full test suite doesn't quite complete.
@minrk, I had to manually interrupt it at:
test_retries (IPython.parallel.tests.test_lbview.TestLoadBalancedView) ... ^CTraceback (most recent call last):
File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/iptest.py", line 313, in run
File "/home/fperez/usr/lib/python2.7/site-packages/IPython/testing/iptest.py", line 306, in _run_cmd
retcode = subp.wait()
File "/usr/lib/python2.7/subprocess.py", line 1281, in wait
pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
File "/usr/lib/python2.7/subprocess.py", line 478, in _eintr_retry_call
I don't know if you want to consider that an issue for us, for zmq or for jsonlib, though, so I won't open another one quite yet. If you think it's on our side, then let's open a new issue for it.
Ok, thanks. In the meantime, the workaround I committed means people won't get burned too badly if they have jsonlib installed; I'll leave the decision of what to do in zmq up to you (though dropping jsonlib support seems like the right approach: I looked and the api is so different that any but the most trivial uses are likely to cause problems similar to this one).
The main point of the zmq.utils.jsonapi is that it serializes to/from utf8-encoded bytes, and json libraries are inconsistent in whether they deal in unicode or str. If that's not what we need (e.g. talking to files), we should be using the stdlib json. And the fact that jsonlib does not guarantee kwarg equivalence means that anyone using jsonapi with kwargs must take this into account (jsonapi.jsonmod.__name__ is the appropriate value to check).
The practical result of this is that we (IPython) should essentially never pass kwargs when using zmq.jsonapi. There is exactly one case where we are currently using a kwarg where we should be using zmq.jsonapi at all, and it has always properly handled the difference in jsonlib. The real bug here is that we were using jsonapi in places that don't make any sense.
I did find the bug with jsonlib in the test suite - it unserializes floats to decimal.Decimal (?!), so I have to cast it to float before passing it to datetime.timedelta. There are few cases where the difference will matter (and exactly one case where a float is ever sent via json), but we will have to be wary if we work with floats while jsonlib is a possible actor.
This may be the last straw for dropping jsonlib support in pyzmq, but that will be irrelevant to IPython until we bump minimum pyzmq version past 2.1.11.
A monkeypatch for prohibiting pyzmq from using jsonlib, and forcing fallback to stdlib:
from zmq.utils import jsonapi
if jsonapi.jsonmod.__name__ == 'jsonlib':
jsonapi.jsonmod = json
But this may not be ideal, as jsonlib is significantly faster than stdlib, and with the small fix in PR #1040, well behaved.
OK, I like the approach in #1040, so let's go with that. Thanks!