Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[not for merge] Demo for Logan of Transformation Catalog style behaviour #3270

Closed
wants to merge 3 commits into from

Conversation

benclifford
Copy link
Collaborator

Description

Logan had some questions that made me think of VDS style Transformation Catalogs: attributes of tasks that are tied neither purely to an app, app invocation or site/executor.

This PR is a quick (at core, 1 line) demo of what that might look like in Parsl: after a task is submitted and some initial processing has happened, a user plugin gets to screw with the task definition as it wants. This single point is deliberately not "per-site" or "per-app" etc - that's specifically the functionality this is trying to avoid.

Instead, the user supplied tc_logan can make its own judgements about what should be done, and how. It can mutate the task record however it wants, in the same way as task retry handlers can, but at a different point in task processing.

Basically anything that's happened already in dfk.submit you should not mess with.

The supplied demo test case shows adding in a parameter to a function that is a function of both the site name and app name - in the traditional Transformation Catalog, that might be a look up in a 2d-style site/app table.

Type of change

  • demo

@benclifford benclifford changed the title Demo for Logan of Transformation Catalog style behaviour [not for merge] Demo for Logan of Transformation Catalog style behaviour Apr 9, 2024
@benclifford
Copy link
Collaborator Author

closing this - @WardLT has had a chance to look and imagine ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant