Allow an array of plugins to be assigned to an array of ensemble members

[jmhays]

I would like to assign different potentials to different ensemble members in my current workflow. It is not clear to me whether this can currently be done through add_dependency feature, whether there is an alternate way of achieving this, or whether this is simply not yet supported. It would be great if it could be supported!

[eirrgang]

Would these be different configurations of the same plugin or different implementations altogether?

The simplest solution would be to have all plugins on all ensemble members, but some inactive in some places, but I don't remember off the top of my head how the ensemble parameter inputs work right now. I'll look into it tomorrow and follow up.

There is a philosophical point that could become a technical point: if a simulation node has different plugins on different ranks, is it really the same node? I think the purest conceptual way to handle this is for the graph topology to handle multiple ensembles (one for each plugin/simulator combination). gmxapi 0.0.6 could not handle multiple ensembles, but I'll check tomorrow where 0.0.7.1 ended up and look at where we are with the 0.1 alpha.

[jmhays]

To clarify, this would be distinct from something like restrained-ensemble in that each ensemble member would get its own set of different, independent plugins. I can already build the plugins, construct an array of N plugins, and now I want to attach each one to one of N independent ensemble members (just a simple mapping here). So the parameters are not really an issue...
Fwiw, I'm currently working with 0.0.7.

[eirrgang]

In 0.0.7 the work graph is very "flat". Whether or not a WorkElement is part of an ensemble is an implementation detail. That is, when the simulation is launched, the builder for each operation receives the same input, and it is up to the launch() function to do something different on different ranks, if it wants to. (See implementation notes in context.py:Context.) But the builder from the sample_restraint code does not add a node to the graph, since the MD operation just takes over responsibility for it.

The most obvious solution would be to add a node with a compatible "launch" function in RestraintBuilder::build(py::object graph), but it is easier said than done.

A workaround could be to add logic that adjusts the behavior (including how to process parameters) based on the Potential object's "name" field. That's super kludgey, but it should work.

These two assume, again, that every rank is instantiating the same objects.

I think a reasonable alternative would be to create a WorkElement for each unique potential and allow add_dependency to have different behavior when it gets a list. I'm going to test that here in a few minutes...

[jmhays]

I don't understand the first two solutions as well as the very last; I'm not sure how the first two would solve the problem. But the add_dependency solution sounds great. Let me know how it goes!

[eirrgang]

I uploaded a potential fix, but it looks like my Travis-CI configuration has rotted... https://travis-ci.org/eirrgang/gmxapi/builds/548299908

Feel free to try the patch or I can try to test it more thoroughly tomorrow: https://github.com/eirrgang/gmxapi/tree/gmxapi-221

[eirrgang]

The solution to this issue will require that restraints check whether their subscriber list is empty during their build() methods, so the sample_restraint repository should be updated. This is a big enough change to warrant an API version bump... We may need to bump gmxapi to version 0.0.8 for builds against GROMACS 2019 and bump the advertised libgmxapi in pre-GROMACS 2020 master branch to 0.0.9.

This is also a reminder that in some future API revision, we should more clearly answer the question of how the framework evaluates the state of output to determine whether something needs to be run, but that requires more design in terms of operations declaring the sort of output they produce.

Note: Currently, everything WorkElement that has been added to a WorkSpec will have its builder called, and it is up to the builder to decide whether to add a launcher to the execution graph, etc.