Introduce calculate() function

[jan-janssen]

In the previous interface a new class could be implemented in pyiron by defining two functions write_input() and collect_output(). The example class would look like this:

from pyiron_base import GenericJob

class NewJobClass(GenericJob):
    def __init__(self, project, job_name):
        super().__init__(project, job_name)

    def write_input(self):
        ...

    def collect_output(self):
        ...

The new interface replaces the write_input() function with an get_input_parameter_dict() function which returns the files_to_write and the files_to_copy as a dictionary. In addition, the collect_output() function is placed outside the job class. Both of these changes are designed to support a more functional approach in pyiron_base. Only the get_input_parameter_dict() and get_output_parameter_dict() functions are now part of the new job class. The get_input_parameter_dict() function takes the input defined by the user on different properties like job.input.structure or functions like job.set_kpoints() and merges everything into a dictionary with input file content. This dictionary of input file content allows expert users to overwrite the pyiron generated input with their own input, if the get_input_parameter_dict() function defined in the new job class supports this. In the same way the get_output_parameter_dict() function allows the developer to provide additional parameters to the collect_output() function:

from pyiron_base import GenericJob

def collect_output(working_directory, output_parameter_1, output_parameter_2):
    ...
    return output_dict

class NewJobClass(GenericJob):
    def __init__(self, project, job_name):
        super().__init__(project, job_name)
        self._job_with_calculate_function = True
        self._collect_output_funct = collect_output

    def get_input_parameter_dict(self):
        ...
        return {
            "files_to_write": {"test.py": "if __name__ == '__main__':\n    print('Hello')\n"}, 
            "files_to_copy": {"my_bashrc": "~/.bashrc"}
        }

    def get_output_parameter_dict(self):
        return {"output_parameter_1": ..., "output_parameter_2": ...}

For those users who only want define the write_input() and collect_output() function, pyiron_base still supports the wrap_executable() function:

def write_input(input_dict, working_directory="."):
    ...

def collect_output(working_directory="."):
    ...
    return output_dict

job = pr.wrap_executable(
    job_name="Cat_Job_energy_1_0",
    write_input_funct=write_input,
    collect_output_funct=collect_output,
    input_dict={"energy": 1.0},
    executable_str="cat input_file > output_file",
    execute_job=True,
)

The limitations of the wrap_executable() function are: MPI parallelism is not supported and the input_dictionary is directly communicated to the write_input() function. So it is not possible to have a generic interface and an expert user interface with this approach. Internally the wrap_executable() already uses the new job class interface. In the same way the collect_output() function only accepts the working_directory as input parameter and it is currently not possible to provide additional parameters for the parsing of the output.

[jan-janssen]

I start by switching the ExecutableContainerJob to using the new calculate() function. Once this pull request is merged and the new pyiron_base version is released the same changes can be applied to LAMMPS and VASP in pyiron/pyiron_atomistics#1446 and pyiron/pyiron_atomistics#1447

[pmrv]

It's a bit annoying that this now duplicates a lot of the logic present in GenericJob.run_static and execute_job_with_external_executable. Do you see an option to unify this again (in the future)?

[jan-janssen]

It's a bit annoying that this now duplicates a lot of the logic present in GenericJob.run_static and execute_job_with_external_executable. Do you see an option to unify this again (in the future)?

I updated the run_static() function to either call execute_job_with_calculate_function() or execute_job_with_external_executable().

[samwaseda]

Except for my minor comment I'm ok with this.

[pmrv]

I don't this will actually work for PythonTemplateJob, because the CalculateCaller is now expected to run a script and won't touch run_static (which is what implementors are supposed to overload), right?

[jan-janssen]

As run_static() is defined in PythonTemplateJob it overwrites the run_static() in GenericJob.

[pmrv]

Like I wrote, I don't think this will work with the TemplateJob, but it also doesn't need to be unified now. Keeping _python_only_job around and checking for both in _check_if_input_should_be_written should be sufficient.

[pmrv]

Correct, I missed that it's overloaded anyway in case of TemplateJob.