Implementing task subscription filtering #11805
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chrisguidry
force-pushed
the
task-subscription-filtering
branch
from
b7b2038
to
88e9415
Compare
✅ Deploy Preview for prefect-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
zzstoatzz
reviewed
Feb 2, 2024
| ) | ||
|
|
||
| if not task_runs: | ||
| winner, *placers = finishers |
zzstoatzz
approved these changes
Feb 2, 2024
zzstoatzz
reviewed
Feb 2, 2024
| @@ -127,9 +122,7 @@ async def test_scheduled_tasks_are_enqueued_server_side( | |||
| task_run: TaskRun = await foo_task_with_result_storage.submit(42) | |||
There was a problem hiding this comment.
we shouldn't have to await this but that's because the _create_autonomous_task_run thing has the sync_compatible decorator on it which doesn't know both that this is an async context and that this task is sync - have a fix coming here
The goal here is for each subscriber to be able to filter to a subset of the tasks they are interested in (i.e. which tasks a task server is serving). This required routing task runs into queues and monitoring multiple queues for tasks. This was a good opportunity to refactor some of the mechanics here to capture the idea that a `TaskQueue` is really two queues (a "regular" queue and a high-priority retry queue). This led me down a path of tidying up the implementation to remove globals and encapsulate more of the behavior.
chrisguidry
force-pushed
the
task-subscription-filtering
branch
from
ff23c80
to
9c43e26
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The goal here is for each subscriber to be able to filter to a subset of the
tasks they are interested in (i.e. which tasks a task server is serving). This
required routing task runs into queues and monitoring multiple queues for
tasks. This was a good opportunity to refactor some of the mechanics here to
capture the idea that a
TaskQueueis really two queues (a "regular" queue anda high-priority retry queue). This led me down a path of tidying up the
implementation to remove globals and encapsulate more of the behavior.