-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add way to store data returned from task #419
Comments
As of today, the task result is logged, but not saved because this would mean the cleaning of finished tasks, which up until now was just a matter of maintenance, would become a matter of functionality too (you mustn't delete a finished task until its result was read, otherwise it's lost) Your tasks are expected to have a side effect. But in your project, it's perfectly ok to define your own decorator that will store tasks results (and subsequently take care of the deletion) on your terms. It might be interesting to expose a simpler way to define a "middleware" though. |
Thanks for the explanation @ewjoachim I'll look at righting some decorators for storing returned values into a custom table. |
We might do something more straightforward, but until then: https://procrastinate.readthedocs.io/en/stable/howto/middleware.html |
Cf #370 : maybe we could revisit this decision some day. It may depend on whether there are volunteers who want to take a shot at implementing this. |
@ewjoachim I quite like the idea of a decorator/middleware and associated helper to retrieve results. So maybe a simple API like:
I'm guessing a task should fail if storage fails, but am not 100% sure about this. |
Do you mean you'd like Procrastinate to implement such a decorator, or do you like the idea of being able to do this on your own in your app outside of the scope of procrastinate itself ?
I guess this is application-dependent :) Because storage specifics can be very custom. I think that's partly why I'm divided regarding having Procrastinate do this by itself. |
Just wanted to add that I misunderstood what Procrastinate can/can't do and after implementing Procrastinate in my app, I'm probably rolling back to (pre-existing) celery implementation with trunk due to its ability to wait until the task completes and then yield the result back to the caller. In our case, we don't want jobs just "logging" that they finish, but, rather providing an ID back that tells us where to go to get the result of that particular job. |
Ok, sorry to see you go. If we end up steering toward this direction, I'll make sure to drop you a line :) |
Thanks! |
For example when a job created from a task such as:
finishes successfully, then the "Hello World" should be able to be looked up using the job id.
The text was updated successfully, but these errors were encountered: