RemoteException not re-thrown by pmap()

[samoconnor]

@everywhere function do_work(i)
    r = [1,2,3][i]
    println(r)
    return r
end

for r in pmap(do_work, [3,2,1,0])
    println("pmap got result")
    println(typeof(r))
end

The pmap() call above returns a RemoteException object in the result array instead of re-throwing it.

The output is:

./julia -p4 test.jl
    From worker 3:  2
    From worker 4:  1
    From worker 2:  3
pmap got result
Int64
pmap got result
Int64
pmap got result
Int64
pmap got result
RemoteException

Changing pmap() to map() results in BoundsError as expected.

[simonster]

Related to/dup of #7808. Also IMO err_stop=true and rethrowing the error should be the default behavior for pmap.

[samoconnor]

As far as I can tell err_stop=true is just a performance optimisation to save time when an error occurs.

However, re-throwing the exception (after re-tries) seems like a must unless the documentation clearly says that pmap() returns exceptions in the output collection.

[simonster]

The problem is that there's no way to get the incomplete output of pmap if it throws an exception. If you set err_stop=err_rety=false, you clearly want the output even if it's incomplete. (With the defaults err_stop=false, err_retry=true it's a little less clear what you want.) The behavior should certainly be documented, but I imagine there are cases where it's desirable to return the exceptions instead of throwing them.

[samoconnor]

To me it seems very clear: a function should either return a "complete" result or throw an exception.
A function that returns exception objects in-line with an output collection seems odd to me.

If "incomplete output of pmap" is useful in some particular application, that application could just do:

pmap([1,2,3]) do i
    try
        [1,2][i]
    catch e
        return e
    end
end

[simonster]

But then failures never get retried with err_retry=true, because you don't throw an error.

[malmaud]

This is definitely an issue, but closing in favor of the earlier #7808.