add file retrieval lists to job template #5636
Conversation
In some scenarios, retrieving files after job completion may need to be handled by pushing them back via the compute job rather than by AiiDA trying to fetch them. By adding the various retrieval/copy lists to the JobTemplate, this is made possible.
|
@sphuber We needed this information at the scheduler level for an internal plugin. In general, I think this will be necessary for supporting a mode where the compute node is responsible for pushing results back, rather than AiiDA being responsible for pulling results in. Can you please let me know whether there are any concerns for adding these at the top-level of the job template? If not I can add a test and get this PR ready for review. |
How is this information read by the compute node though? The code of the |
|
Should this maybe be converted into an issue with a feature request describing the use case? |
The scheduler plugin uses the information to add corresponding commands to the batch script. |
Yes, I'll do that, but it will require a bit of thought. |
I see that, but what is the idea of the PR? Were you looking to get it merged? I am asking since you opened it as a draft and like you say it might merit some discussion before changing this. If you agree with opening an issue, maybe this can be closed for the time being |
|
I will close this for now as the requested feature can be discussed in #5892 |
In some scenarios, retrieving files after job completion may need to be handled by pushing them back via the compute job rather than by AiiDA trying to fetch them.
By adding the various retrieval/copy lists to the JobTemplate, this is made possible.