add %%px --local for local execution

[minrk]

So that a cell can be run on engines as well as the local namespace.

Prompted by SO Question 14184621.

[bfroehle]

Wow, neat idea.

1. If the command is slow to execute, might it be worthwhile to start processing on the remotes before running locally?
2. Or, in the current implementation, should we leave a note saying that the command will be first executed locally and will run on the engines only if the local execution succeeds?

[ahmadia]

Agreed with @bfroehle. Ideally, the command might return a future from remote execution, starts local execution in a new thread, then waits on both.

[minrk]

There are advantages to both:

1. local first - errors will raise immediately, preventing the much more disruptive remote errors
2. remote first - everything is executing concurrently

@ahmadia - there is really no reason at all to do the local execution in a thread. If we do local execution after remote, all n+1 execuctions will be concurrent.

I chose 1., since I really intended this for interactive definitions, where the first choice is always preferable,
but if people are goind to use it for execution (would be weird, but not unlikely), then I suppose I should change to 2.

[minrk]

local is now run after async submission of remote

[ahmadia]

@minrk - I suggested that you execute locally in a thread so that in the case the local execution spinlocks but the remote execution does not, you will be able to retrieve the remote results in a "good" state even if you have to signal/interrupt local execution. If you are not using threads to protect against this sort of behavior elsewhere in IPython I agree there's not much point in using one here.

[minrk]

@ahmadia - yes, it makes sense in the abstract, just not in the Context of IPython, where there is no async local execution.

[ogrisel]

+1 for merging the current impl from my point of view.