Implement the usage of plum PortNamespaces #1099
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The ProcessSpec class in plum has been changed quite a lot to support generic port namespacing in the inputs and outputs. This requires small changes in the interface
The only calculation that was being tested against the JobProcess was the TemplatereplacerCalculation, which doesn't have a use method with an additional parameter. As such the InputGroup functionality was not being tested
…pace The InputGroup functionality has been made obsolete by an improved and generalized implementation of port namespacing through the PortNamespace class. To create the necessary port namespace for a use method containing an additional parameter, we can now use ProcessSpec.input_namespace()
Methods like dynamic_input and dynamic_output have been deprecated in the latest plum due to the introduction of PortNamespaces. To make the input or output ports of a ProcessSpec dynamic or not one has to now go directly through the relevant port namespace.
muhrin
suggested changes
Feb 2, 2018
aiida/work/processes.py
Outdated
| if port.non_db: | ||
| continue | ||
|
|
||
| items.append((prefixed_key, value)) |
There was a problem hiding this comment.
This will be called a second time if the port does not exist in the spec (i.e. is dynamic).
Changing this brings a new version of kiwipy which required a small change in the RmqConnector API to close it
When setting up the database record for a Process, the dictionary of inputs, which may be nested, has to be flattened, skipping any inputs whose corresponding port is marked as non_db, i.e. an input that should not be stored as a node in the database. The link labels will be created by concatenating the namespaces with underscores. Also make sure that the port types of the plum ProcessSpec are overridden with the versions of aiida core, which have the WithNonDb mixin.
…nd thus immutable Plum is currently not recursing down nested input dictionaries when creating the parsed input data structure of a Process, which means that only the top level inputs (reachable through self.inputs from within the Process) are wrapped in an AttributesFrozendict and therefore immutable. Any inputs in an input namespace will be bare dictionaries and therefore mutable. This requires a fix in plum, but two tests have been added that test this behavior. The one with the namespaced inputs currently fails.
|
This PR will also address the issues raised in #919 . In commit |
muhrin
approved these changes
Feb 5, 2018
This was referenced Feb 5, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1096
This implements the changes necessary to leverage the recently added
PortNamespaceinplum.The inputs and outputs of a
ProcessSpecare now port namespaces, which means that they can be arbitrarily nested. This also allowed us to get rid of theinput_groupmethod and replace it by a properPortNamespacewhich can now be used forJobProcessesthat have use methods with additional parameters.