I imagine a common usecase will be end users using all engines in a direct view to create a context, using that context to create distarrays. What about the following simplification to mimick NumPy more closely?
import distarray as da # Create default context
dist_zeros = da.zeros((10**5, 10**5), dist={0:'c'}) # uses default Context with all engines
This will have a much closer feel to NumPy for end users.
If users want to create their own Context and use that instead, then that avenue is available:
from distarray import Context
from IPython.parallel import Client
dv = Client()[::2]
ctx = Context(dv)
other_zeros = ctx.zeros(...) # creates distributed array on even-numbered engines...
Cons:
- Hidden global object created when importing the
distarray package.
- How handle case when not all engines are created with MPI?
I imagine a common usecase will be end users using all engines in a direct view to create a context, using that context to create distarrays. What about the following simplification to mimick NumPy more closely?
This will have a much closer feel to NumPy for end users.
If users want to create their own Context and use that instead, then that avenue is available:
Cons:
distarraypackage.