Test the Message Spec #1627

merged 12 commits into from Apr 18, 2012


None yet

2 participants

IPython member

This adds a few preliminary tests for the message spec.

It uses Traitlets to perform validation of keys.

Checks right now are not very strict, as (almost) any key is allowed to be None, as long as it is defined. This is because I simply do not know which keys are allowed to be None, and this is not discussed in the specification. If no keys are allowed to be None, we violate that all over the place.

Parametric tests are used, so every key validation counts as a test (147!).

Message spec doc was found to misrepresent code in a few points, and some changes were made:

  • spec had error keys as exc_name/value, but we are actually using ename/value (docs updated to match code)
  • payloads were inaccurate - list of dicts, rather than single dict, and transformed_output is a payload, not top-level in exec-reply (docs update to match code).
  • in oinfo_request, detail_level was in message spec, but not actually implemented (code updated to match docs).

History messages are not yet tested, but I think I get at least elementary coverage of everything else in the doc.

IPython member

Are you getting these tests run by a simple iptest? I think IPython.zmq is still in our exclusions list... If I run it manually via iptest IPython.zmq, I get dropped into an interactive editor a couple of times... It seems that the zmq subpackage has a couple of nasty tests in it and maybe initially we just punted and disabled the lot. As part of this PR, we should fix that so that iptest picks up all of IPython.zmq for testing as well...

IPython member

No, I'd been running the individual test file manually (iptest IPython.zmq.tests.test_message_spec).

I'll get on fixing IPython.zmq for general testing.

IPython member

Awesome. That way this PR will really have the (fantastic) impact of getting all of our zmq stuff into the regular testing workflow.

minrk added some commits Apr 18, 2012
@minrk minrk close KernelManager channel sockets when they stop
Otherwise there are dangling sockets on the Context, which cannot terminate.
@minrk minrk skip magic_edit doctest in zmqshell
it opens a GUI editor, which is obviously inappropriate
@minrk minrk include IPython.zmq in iptest groups bf52088
IPython member

okay, iptest now runs IPython.zmq (with gtk/matplotlib exclusions, as appropriate, I believe).

IPython member

Awesome! All tetsts pass on my box; we may get a few odd failures tomorrow from the buildbots, but if that's the case we'll track them one by one with the info from Shining Pandas. Heads-up to @takluyver in case anything out of the ordinary happens with py3. But let's merge this as-is, you did a terrific job and I'm thrilled to have zmq and messaging test coverage; we can fine-tune things later as needed.

@fperez fperez merged commit 232fa81 into ipython:master Apr 18, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment