Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow TaskGraph and Task to be pickled #22

Open
richpsharp opened this issue Mar 7, 2020 · 2 comments
Open

Allow TaskGraph and Task to be pickled #22

richpsharp opened this issue Mar 7, 2020 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@richpsharp
Copy link
Collaborator

richpsharp commented Mar 7, 2020

This will require implementing the __reduce__ method for these objects: https://docs.python.org/3/library/pickle.html#object.__reduce__

Note this feasible since TaskGraph can share the same database.

The tricky issue, if addressed, will be how to lock or detect duplicate submitted tasks in separate TaskGraph objects.

Currently TaskGraph stores some global state, it may be worth offloading that state exclusively to its database.

@richpsharp richpsharp added this to the 0.10 milestone Mar 7, 2020
@richpsharp richpsharp self-assigned this Mar 7, 2020
@richpsharp richpsharp added the enhancement New feature or request label Mar 7, 2020
@phargogh
Copy link
Member

phargogh commented Mar 8, 2020

Interesting! Out of curiosity, what's the use case for this? Allowing TaskGraph objects to be returned from a Task?

@richpsharp
Copy link
Collaborator Author

Two things:

  1. allow Tasks to be passed to add_task as arguments so the Task can call get at runtime and,
  2. allow TaskGraphs to be passed to Tasks so add_task could be called at runtime as well.

@phargogh phargogh removed this from the 0.10 milestone Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants