Add ImmigrantJobProcess

[DropD]

Description

Sometimes we need to import a previously run calculation into AiiDA. The way to do this used to be ImmigrantCalculations, e.g. PwImmigrantCalculation in the QE-plugin. They would be JobCalculations with extra functions to simulate having been submitted and calculated. The daemon would then automatically pick up such a calculation and retrieve and parse it.

With the new workflow engine this no longer works, instead we need a JobProcess subclass which can take simulate the steps up to retrieval. This also gives us the opportunity to split out the generalizable tasks and provide them for all plugins to use.

Goal: Reenable Immigrant type calculations and make them easy to implement for plugin developers.

Implementation Sketch

ImmigrantJobProcess(JobProcess):

    @override
    def run(self):
        """Simulate submitting and computing a job calculation, preparing it for retrieval."""
        from aiida.common.datastructures import calc_states
        _ = super(ImmigrantJobProcess, self).run()

        self.calc._set_state(calc_states.SUBMITTING)
        import_path = self.inputs.import_from_path
        self.calc._set_attr('remote_workdir', import_path)
        remotedata = RemoteData(computer=self.calc.get_computer, remote_path=import_path)
        remotedata.add_link_from(self.calc, label='remote_folder', link_type=LinkType.CREATE)
        remotedata.store()

        self.calc.set_state(calc_states.COMPUTED)
        return plumpy.Wait(msg='Waiting to retrieve', data=RETRIEVE_COMMAND)

    ## This hook has to be called in the appropriate place in JobProcess
    def spec_hook(self, spec):
        spec.input('import_from_path', valid_type=basestring)

Projected Usage

Plugin developers might choose to partially wrap the following in a convenience function (or method of their calculation).

proc_cls = ImmigrantJobProcess.build(CalculationFactory('my_plugin.calc'))
builder = proc_cls.get_builder()
remote_folder = '/absolute/path/on/the/remote/machine'
construct_immigrants_input(computer, remote_folder)  # Up to the plugin dev to implement
submit(proc_cls, **builder)

[chrisjsewell]

I would definitely +1 this. For myself (and I suspect others), wanting to adopt aiida, its only really feasible if I can migrate my previous computations.