Task executer: Handle task artefacts

[jackschofield23]

This has been split from #52

This ticket focuses on the Task Executer side. Ticket 52 is reserved for the Workflow Executer section.

Copied from #52

Workflow tasks use artefacts in two ways - as input, and as output. Artefacts can be files or directories, either the original payload that came from the Informatics Gateway or files that were generated by tasks in the workflow.

For more information about artefacts and how they're supposed to be used see the documentation: https://github.com/Project-MONAI/monai-deploy-workflow-manager/blob/develop/guidelines/mwm-workflow-spec.md#artifacts.

We need to support artefacts in the following ways:

• The task executer will then generate temporary credentials (not in scope for this ticket, see Generate temporary credentials (Links to #52) #112 ) and provide these to the task plug-in
• The task executer will receive output artefacts from the task plug-in and save their path in the task update status event

Acceptance criteria

• GIVEN a task definition contains input artefacts WHEN the task is dispatched THEN the artefact values should be converted to a file path and included in the task dispatch event
• GIVEN the task definition contains an input artefact WHEN the artefact value references a non-existent artefact THEN it shouldn't be included in the artefact map (note: in the future we'll add the notion of mandatory artefacts, see Mandatory versus optional artefacts #99)
• GIVEN a task update event contains an output artefacts map WHEN the task is updated THEN the output artefacts and their paths should be stored in MongoDB

Implementation notes

1. Artefact values can be one of the following:
   1.1. context.input should be converted to the folder where the original payload has been stored. In the case of a DICOM input, that is the study folder.
   1.2. context.executions.TASK_ID.artifacts.ARTIFACT_NAME should be converted to the path provided by the task plug-in for this artefact name in the output artefact map it provided when TASK_ID was wrapped up. These are "registered artefacts".
   1.3 context.executions.TASK_ID.output_dir should be translated to the output directory of the referenced task
2. See Update task status in the workflow instance #95 for more information on the task status update event. It is as part of handling the task status update that output artefacts should be registered.

File structure

The following structure should be used within MinIO:

BUCKET
- {{PAYLOAD_ID}}
-- dcm
-- fhr
-- workflows
--- {{WORKFLOW_INSTANCE_ID_1}}
---- {{execution_id_1}}
----- Files generated by execution 1
--- {{WORKFLOW_INSTANCE_ID_2}}

[RemakingEden]

Int tests:

1. Send task dispatch message with artefact
2. Receive task update event with same artefact

Data:

• Correct artefact (Happy path)
• Null

[mocsharp]

@RemakingEden can we close this as complete?