-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reusable output node #40
Comments
Just checking--this is a workflow node but not one that requires communication between contexts, right? i.e. it could effectively be a map() operation so contexts do their output independently. |
I see what you mean about a map() operation, but that's not quite what I was getting at here. I've updated the description in the hopes of clarifying. Map operation is a good thing to bring up, though. The grand plan is to have parallel execution essentially work that way, but right now it is rigidly structured as an array of pipelines with possible operations across the full array (ensemble operations). That is, we have simulation scope and ensemble scope for operations or data flow. I think the first step in allowing something more map() like would be to explicitly expand the parallel edges and nodes in the execution DAG. Right now we have it more as "array operations of a given width can feed array operations of the same width, with the exception of many-to-one and one-to-many". Otherwise, we start dealing with more scheduling logic, which I would rather defer to specific context implementations (and try to pawn it off to third-party data flow graph execution managers). I think some of the necessary metadata will become clearer as we implement full workflow checkpointing. |
Relates to issue kassonlab#40 Functionally, add a boolean `append_output` property to MD simulations that defaults to `true`. Implemented with an optional keyword / param that, if set `false`, gives the `mdrun -noappend` behavior.
Recording some comments from the pull request pull request that may not be resolved in that pull request: In current plans for gmxapi 0.1.0, each operation in each identifiably distinct work description is uniquely tagged. In order to implement [more output flexibility], we need to be specific about behavior and artifact handling.
An update should specify behavior in a sufficient level of detail that we can discuss whether the specified behavior as written should be different. |
Relates to issue kassonlab#40 and to bug kassonlab#130 Resolves task in kassonlab#130 when merged.
Relates to issue #40 Functionally, add a boolean `append_output` property to MD simulations that defaults to `true`. Implemented with an optional keyword / param that, if set `false`, gives the `mdrun -noappend` behavior.
We need to be able to append trajectories sensibly and robustly along a single simulation pipeline.
Most basically, some workflows will included several stages of simulation that should produce a single continuous trajectory. Whether the lower level implementation involves multiple GROMACS program launches or a single launch (that runs for a bit, changes parameters, and runs more) is not relevant at the higher-level interface. So this issue has a few parts:
One way is to have a single output node represented in the workflow. Multiple simulation nodes in the workflow graph could run as a pipeline. In order for the output node to be responsible for writing the entire trajectory, the intermediate nodes would "pass through" trajectory frames for time steps before the simulation node is active.
Two obvious alternatives are
I like the latter and it seems more TensorFlow-ish, but I have thought about it less and it implies introducing more formalism into one or both of our graph schema: the workflow specification graph (specified in the high-level API) and the execution / data flow DAG (currently evolving fluidly).
The text was updated successfully, but these errors were encountered: