--fast flag for test suite!

[ivanov]

runs the test suite in parallel, closes #1880

[minrk]

don't forget to rebase on PR #4165.

[ivanov]

actually just updated to be standalone. This PR doesn't interfere with #4165

[minrk]

parallel is the slowest of all, shouldn't it be one of the first as well?

[ivanov]

the output now looks something like this:

**********************************************************************

IPython test group: IPython.kernel..................................OK

IPython test group: IPython.kernel.inprocess........................OK

IPython test group: IPython.config..................................OK

IPython test group: IPython.core....................................OK

IPython test group: IPython.extensions..............................OK

IPython test group: IPython.lib.....................................OK

IPython test group: IPython.terminal................................OK

IPython test group: IPython.testing.................................OK

IPython test group: IPython.utils...................................OK

IPython test group: IPython.nbformat................................OK

IPython test group: IPython.qt......................................OK

IPython test group: IPython.html....................................OK

IPython test group: IPython.nbconvert...............................OK

IPython test group: autoreload......................................OK

**********************************************************************

and iptest --fast finishes in 22 seconds, whereas without the --fast flag, it takes 75 seconds here.

[ivanov]

and with iptest --all --fast I get 70 seconds, whereas without it, it's 217.261.

[takluyver]

Looks good to me.

[bfroehle]

This is a wonderful addition, and with minimal work would probably suffice to close #1880 which I opened a year ago.

The only thing I'd consider changing is the name of the flag, from --fast which is pretty ambiguous to -j [jobs] / --jobs=[jobs] as is standard in most common unix tools:

• iptest (use 1 process)
• iptest -j (use all available processes, or, perhaps, start all subtests simultaneously)
• iptest -j 4 (use 4 processes)

Does anybody else have thoughts on the name?

[takluyver]

@ivanov @minrk and I discussed the flag name in person - adding a parameter would require proper argument parsing, whereas iptest is currently just doing if '--fast' in sys.argv, so we said we'd leave that for when I refactor iptest, which I plan to start working on once this lands.

[ivanov]

Yeah, I'm all for a -j flag going forward, it was just going to make this code a lot uglier on the parser side of things, as @takluyver pointed out. In the meantime, the --fast flag works eponymously, and can later be an alias for just -j when the number of threads is specified. I added that it closes #1880 to the PR description, so GH can do it's magic in auto-closing that when this lands

[minrk]

do you want to use imap_unordered, so that results will arrive as they finish? They are still executed in the same order.

[takluyver]

The zip() just below currently relies on getting the results in the same order that they were submitted, though I'm sure there are ways around that if we want to use imap_unordered.

[minrk]

okay, then no big deal. But if @ivanov wants it, if do_run just returned (x[0], x[1], ret), everything would be in sync and the zip would be unnecessary.