let remote compute resource "push back" output files

[ltalirz]

Is your feature request related to a problem? Please describe

File transfer between the machine running the AiiDA daemon and the compute resource is currently handled by the AiiDA daemon via scp (which puts load on the daemon).

In cloud environments, this file transfer is best accomplished by other means, e.g. via a shared file system or object store that is mounted both locally on the VM running the AiiDA daemon and remotely on the compute resource.

A workaround to have AiiDA support this use case today is to configure your remote compute resource as the localhost, let AiiDA copy files onto the locally mounted storage, and then let a scheduler plugin interact with the scheduler of the remote compute resource.

This works as long as the files created in the "scratch folder" are transferred automatically as they are created by the simulation code.

There are, however, use cases where one wants to avoid transferring all of the files as they are created during the run of the calculation (e.g. because some of them are large and temporary, and network bandwidth to the shared storage is limited).
In such cases, you may want to run the calculation in a temporary local file system on the compute resource.
Afterwards, the remote compute resource needs to know which files should be "pushed back" to shared storage.

Describe the solution you'd like

#5636 implements a simple solution for this: by making the various copy lists available in AiiDA's job template, the scheduler plugin can make sure to write a job script that, after running the calculation, takes care of copying back to shared storage only those files that are actually needed by AiiDA to parse the calculation.

Of course, this is still a bit of a hack, but it is very lightweight and accomplishes the use case.
The only "cost" is that the job template now contains info about which output files AiiDA is interested in.

@sphuber