[not for merge] Demo for Logan of Transformation Catalog style behaviour #3270
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Logan had some questions that made me think of VDS style Transformation Catalogs: attributes of tasks that are tied neither purely to an app, app invocation or site/executor.
This PR is a quick (at core, 1 line) demo of what that might look like in Parsl: after a task is submitted and some initial processing has happened, a user plugin gets to screw with the task definition as it wants. This single point is deliberately not "per-site" or "per-app" etc - that's specifically the functionality this is trying to avoid.
Instead, the user supplied
tc_logan
can make its own judgements about what should be done, and how. It can mutate the task record however it wants, in the same way as task retry handlers can, but at a different point in task processing.Basically anything that's happened already in
dfk.submit
you should not mess with.The supplied demo test case shows adding in a parameter to a function that is a function of both the site name and app name - in the traditional Transformation Catalog, that might be a look up in a 2d-style site/app table.
Type of change