don't eval task output with llm unless the task lacks problem matchers#295814
Merged
meganrogge merged 2 commits intomainfrom Feb 17, 2026
Merged
don't eval task output with llm unless the task lacks problem matchers#295814meganrogge merged 2 commits intomainfrom
meganrogge merged 2 commits intomainfrom
Conversation
Tyriar
approved these changes
Feb 17, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes the performance of task output monitoring by eliminating redundant LLM calls for tasks that have problem matchers configured. The change addresses issue #266292 which identified that get_task_output was making two LLM requests that added 1.5-2+ seconds to tool execution time.
Changes:
- Skip LLM-based error assessment (
_assessOutputForErrors) when a custom polling function is provided, as it already supplies structured error information - Only pass
taskProblemPollFntoOutputMonitorfor tasks that have problem matchers configured, allowing tasks without matchers to use LLM assessment - Prevent
taskProblemPollFnfrom being called and throwing for tasks without problem matchers
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/tools/monitoring/outputMonitor.ts | Modified _handleIdleState to skip LLM error assessment when a custom poll function is provided |
| src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/taskHelpers.ts | Added problem matcher detection to conditionally pass taskProblemPollFn only for tasks with configured matchers |
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/taskHelpers.ts
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix #266292
Skip the redundant ~10s LLM error assessment
_assessOutputForErrorsfor tasks that have problem matchers.Only pass
taskProblemPollFnwhen the task has problem matchers; passundefinedotherwise. Skip_assessOutputForErrorswhen a_pollFnis provided, since it already supplies structured error info.Tasks with problem matchers get fast structured errors. Tasks without problem matchers now correctly fall through to the LLM assessment instead of throwing.