Skip to content

Comments

don't eval task output with llm unless the task lacks problem matchers#295814

Merged
meganrogge merged 2 commits intomainfrom
merogge/task-speed
Feb 17, 2026
Merged

don't eval task output with llm unless the task lacks problem matchers#295814
meganrogge merged 2 commits intomainfrom
merogge/task-speed

Conversation

@meganrogge
Copy link
Collaborator

@meganrogge meganrogge commented Feb 17, 2026

fix #266292

Skip the redundant ~10s LLM error assessment _assessOutputForErrors for tasks that have problem matchers.

Only pass taskProblemPollFn when the task has problem matchers; pass undefined otherwise. Skip _assessOutputForErrors when a _pollFn is provided, since it already supplies structured error info.

Tasks with problem matchers get fast structured errors. Tasks without problem matchers now correctly fall through to the LLM assessment instead of throwing.

Copilot AI review requested due to automatic review settings February 17, 2026 18:07
@meganrogge meganrogge self-assigned this Feb 17, 2026
@meganrogge meganrogge added this to the February 2026 milestone Feb 17, 2026
@meganrogge meganrogge enabled auto-merge (squash) February 17, 2026 18:08
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the performance of task output monitoring by eliminating redundant LLM calls for tasks that have problem matchers configured. The change addresses issue #266292 which identified that get_task_output was making two LLM requests that added 1.5-2+ seconds to tool execution time.

Changes:

  • Skip LLM-based error assessment (_assessOutputForErrors) when a custom polling function is provided, as it already supplies structured error information
  • Only pass taskProblemPollFn to OutputMonitor for tasks that have problem matchers configured, allowing tasks without matchers to use LLM assessment
  • Prevent taskProblemPollFn from being called and throwing for tasks without problem matchers

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/monitoring/outputMonitor.ts Modified _handleIdleState to skip LLM error assessment when a custom poll function is provided
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/taskHelpers.ts Added problem matcher detection to conditionally pass taskProblemPollFn only for tasks with configured matchers

@meganrogge meganrogge merged commit 6076010 into main Feb 17, 2026
24 checks passed
@meganrogge meganrogge deleted the merogge/task-speed branch February 17, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speed up get_task_output?

2 participants