expedite parallel tests

[minrk]

Two changes, which significantly reduce the time it takes to run iptest IPython.parallel:

• changed the heartbeat from the default 3s to 1/4s
• changed the add_engines code, so that in most cases engines up to a minimum number are added, rather than always adding to the total. This means that most previous calls to add engines are now no-ops, because there were already enough for the given situation.

On my machine, this reduced time iptest IPython.parallel from two minutes to forty seconds.

[takluyver]

Will changing the heartbeat affect ShiningPanda? I remember we've occassionally seen failures where processes didn't start up as fast as it expected.

[minrk]

It shouldn't, or at least not negatively. There are two levels of timeout related to registration and the heartbeat:

1. in the Hub, the timeout is the longer out of: two heartbeat periods or five seconds. This measures the time between the initial registration_request message, and the first beat of the engine's heart. If this timeout is reached, the engine is considered DOA, and can never register. I am not aware if this timeout ever having been reached by an engine that hasn't died.
2. in the test suite itself, there is a timeout of five (or ten?) seconds for engines to be added to the list before the test assumes there was a problem with registration. This timeout includes the time between requesting the ipengine subprocess to start, and its sending of the first message, which is why it sometimes fails on slow machines/VMs. Since the time to successfully register is proportional to the heartbeat period, this change should actually make the ShiningPanda timeout less likely.