Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce time it takes to notice a task is complete #424

Open
MattMcL4475 opened this issue Sep 27, 2023 · 0 comments · May be fixed by microsoft/CromwellOnAzure#709
Open

Reduce time it takes to notice a task is complete #424

MattMcL4475 opened this issue Sep 27, 2023 · 0 comments · May be fixed by microsoft/CromwellOnAzure#709
Assignees
Labels
enhancement New feature or request Performance Enable users can run task as cheap and as fast as possible TES Priority: P1 Groomed to a Priority 1 issue
Milestone

Comments

@MattMcL4475
Copy link
Collaborator

  1. TES will no longer ask Batch for complete information on each task each time. Instead,
  2. The controller will set each accepted task to QUEUED as it does today. That'll mean it is not assigned to a pool, yet.
  3. BatchScheduler will set QUEUED tasks to INITIALIZING when assigning them to a pool. CloudTasks will be set to immediately expire when complete.
  4. Scheduler/BatchScheduler will check the batch account blob for the current status of each task. When it's complete, it'll be thus processed.
  5. A per-pool query for failed tasks will be performed for failed tasks that will be thus processed.
  6. All the rest of the "error handling"/"hand holding" will be removed, including support for AutoPools.

In the future, #4 can be updated to cover other states, like "running", etc.

Task information beyond state will only be obtained when needed, not every loop. This will greatly diminish the number of calls to Batch, and greatly lessen the work TES is doing

@BMurri BMurri added the enhancement New feature or request label Sep 27, 2023
@BMurri BMurri added this to the next milestone Sep 27, 2023
@ngambani ngambani added Performance Enable users can run task as cheap and as fast as possible tobegroomed Add this label while creating new issues to get issues prioritized on the backlog labels Oct 5, 2023
@ngambani ngambani removed the tobegroomed Add this label while creating new issues to get issues prioritized on the backlog label Oct 26, 2023
@MattMcL4475 MattMcL4475 added the TES Priority: P1 Groomed to a Priority 1 issue label Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Performance Enable users can run task as cheap and as fast as possible TES Priority: P1 Groomed to a Priority 1 issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants