Improve context API

[joaander]

Description

There are many improvements to be made to HOOMD's context management. We should support

1. Multiple simulations of different types in a single script (e.g. CPU and GPU)
2. Pythonic initialization for libraries
3. (if possible) Implicit automatic context creation (as in hoomd v1.x)

Proposed solution

I do not see any obvious targets for refactoring the C++ side ExecutionConfiguration. I propose
a pure python improvement to the python API using the existing C++ structure.

At the python level, device initialization will be separate from the simulation context:

dev = hoomd.device.cpu()
dev = hoomd.device.cpu(nthreads=8)
dev = hoomd.device.gpu(1)
dev = hoomd.device.gpu([0,1,2], nrank=3)
dev = hoomd.device.auto()

Constructors will take fixed and mutable parameters, but fewer than context.initialize currently accepts.
MPI domain decomposition options can already be passed to hoomd.comm.decomposition.

Mutable execution device parameters will be exposed as properties:

dev.nthreads = 12
dev.notice_level = 3

Devices will be passed by variable, not stored in global state:

sim1 = hoomd.context.SimulationContext(dev1)
sim2 = hoomd.context.SimulationContext(dev2)

SimulationContext uses CamelCase, unlike most of the HOOMD user API. It should be hoomd.context.simulation.

Compared to context.initialize, this can also be a concise one-liner:

hoomd.context.simulation(hoomd.device.gpu())

There are a few options for implicit initialization.

1. No implicit initialization at the library level
2. On hoomd.init.*, create a hoomd.device.auto() if the context is missing a device. In practice, this can only occur for the first
   init command in the script - the one that uses the implicit empty context. Later ones would require an explicit user
   instantiation of a simulation context:

   hoomd.init.read_gsd(...)   # uses implied auto device
   sim2 = hoomd.context.simulation(hoomd.device.gpu(2))
   with sim2:
       hoomd.init.read_gsd(...)   # uses explicit device
   

3. More drastically: do away with explicit user calls to hoomd.context.simulation and instead hoomd.init.* methods each create
   and return a simulation context. The device would need to be passed as an argument:

   sim1 = hoomd.init.read_gsd(filename='one.gsd', device=hoomd.device.gpu())
   sim2 = hoomd.init.read_gsd(filename='two.gsd', device=hoomd.device.cpu())
   

4. Implicit initialization through a hoomd command line entry point. This tool would process command line options,
   create a device, simulation, and decomposition, then execute the user script:

   hoomd script.py --gpu=1 --nrank=3
   hoomd script.py --mode=cpu
   

   (can coexist with one of options 1-3)

Additional context

HOOMD started as a command line tool that ran scripts (which happened to be python scripts).
By v2, it evolved into first class python library, but kept some of its old tendencies. This proposal
is to remove those and make the API more flexible.

With the possibility for multiple and reusable devices, we could potentially move to a pytest testing framework. pytest tests
are simpler and cleaner than existing tests, plus it imports all tests into one process which would improve performance of many small
tests. I'm not certain how we would handle the long validation tests, or non-MPI tests vs test of different numbers of MPI ranks - but
that can be worked out later.

When implemented completely, this proposal would remove context.initialize, option, and potentially change other APIs in v3.
We could ease the transition by implementing the new library API alongside context.initialize in a 2.x
release which would also deprecate the old API (this is not possible with option 3 as it would change the return type of init.*).

Questions to consider

• Which of the 4 options for implicit initialization should we go with?
• Is the command line entry point useful?
• Busy messenger output makes sense for the command line application, but should the library usage pattern be so noisy by default?

[joaander]

Seeking input from other developers and the community.

[joaander]

With #392 we can now separate out MPI initialize cleanly at the user level:

partition = hoomd.comm.partition(nrank=3)
partition = hoomd.comm.partition(mpi_comm=...)

Devices must be attached to a partition

dev = hoomd.device.cpu(nthreads=8, partition=partition)
dev = hoomd.device.gpu([0,1,2], partition=partition)

A default None value for partition will create an default MPI configuration:

dev = hoomd.device.gpu()
dev = hoomd.device.auto()

[joaander]

Multiple simulation contexts do not work cleanly with the proposed command line entrypoint.

init.read_gsd(...)
...
hoomd.context.simulation(hoomd.context.current.device)
hoomd.init.read_gsd(...)

To reuse the same command line options, the user would need to explicitly reuse the device from the current context. MPI options like --linear go into the domain decomposition which would also need an API to forward.

This fault is an argument against implementing the command line entrypoint. We should provide just one consistent API for defining devices. Users that which command line options for their scripts are free to implement them using this API.

[csadorf]

Many Python libraries and applications provide a main() entry point. We could provide a default HOOMD command line interface, however users would need to explicitly call hoomd.main() in their scripts.

[bdice]

This is a good proposal! In particular, I like option 2. I think that would offer a nice balance of intelligent defaults and simple learning curve for new users while still offering users plenty of control.

[joaander]

Bug potential: We generally cache the output of cudaFuncGetAttributes to determine the max possible block size in static variables. If the user switches from one GPU to another in a script, this maximum could be invalid and lead to launch errors.

[jglaser]

I support removing context.initialize() for the reasons mentioned in issue #473.

[b-butler]

The proposed API for v.3.0 will use Simulation objects to manage context. This means that multiple contexts are immediately supported since there is not concept of a global state. @csadorf A command line interface of something like Simulation.main(ops=ops, filename='init.gsd', snapshot=None) could be provided that would grab any command line options, choose a device, and run the simulation with the specified operations from the script.

[joaander]

Implemented on next and feature/new-object-API