Skip to content

feat(augment): multi-agent improvements, limit removal, and routing fix#2600

Merged
pkliczewski merged 4 commits intoredhat-developer:mainfrom
rrbanda:feat/augment-all-improvements
Mar 25, 2026
Merged

feat(augment): multi-agent improvements, limit removal, and routing fix#2600
pkliczewski merged 4 commits intoredhat-developer:mainfrom
rrbanda:feat/augment-all-improvements

Conversation

@rrbanda
Copy link
Copy Markdown
Contributor

@rrbanda rrbanda commented Mar 25, 2026

Summary

Follow-up to #2555 (augment workspace). Adds multi-agent orchestration improvements, removes artificial limits, and fixes multi-turn agent routing.

  • Multi-agent orchestration: ADK-based orchestrator with HITL approval mirroring, tool resolver, conversation state management, agent handoffs
  • Remove all truncation/limits: Eliminates MAX_TOOL_OUTPUT_CHARS, MAX_MCP_PROXY_RESPONSE_BYTES, MAX_RESPONSE_CHAIN_DEPTH, MAX_CONTINUATION_ITERATIONS, MAX_MESSAGE_CONTENT_LENGTH, maxToolOutputChars, maxResponseChars, and related validation upper bounds
  • Routing fix: Upgrade ADK to v0.1.12, restore resumeState / createContinuationState so follow-up messages resume from the active agent instead of always restarting from the router
  • Frontend enhancements: ToggleSwitch component, selectMenuProps, vector store UI improvements
  • Code quality: McpAuthService double-map-lookup fix, TfIdfEmbedder sort simplification
  • Documentation: CONFIG_REFERENCE.md, TROUBLESHOOTING.md
  • Tests: Comprehensive coverage for all new features (162 suites, 2644 tests passing)

Key changes

Agent routing continuity (fixes router-always-called-first issue)

  • AdkOrchestrator.ts: Uses createContinuationState() to save which agent handled each conversation turn, passes resumeState to ADK run()/runStream() so the next message goes directly to the active agent
  • Upgraded @augment-adk/augment-adk from 0.1.11 to 0.1.12

Limit removals (per design decision)

  • Removed all hard-coded truncation constants and per-server/per-agent output char limits
  • Kept MAX_AUTO_REAPPROVALS (Llama Stack bug workaround), auth constants, timeouts, and cache limits

Test plan

  • tsc --noEmit passes
  • All 162 test suites pass (2644 tests)
  • resumeState tests verify agent continuity across turns
  • Deploy and verify routing fix against live LlamaStack instance

@rrbanda rrbanda requested a review from pkliczewski as a code owner March 25, 2026 02:23
@rhdh-qodo-merge
Copy link
Copy Markdown

Review Summary by Qodo

Multi-agent orchestration, agent routing continuity fix, and comprehensive test coverage with limit removals

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• **Multi-agent orchestration improvements**: ADK-based orchestrator with HITL approval mirroring,
  tool resolver, conversation state management, and agent handoffs
• **Agent routing continuity fix**: Uses createContinuationState() to save active agent per turn
  and resumeState to route follow-up messages directly to the active agent instead of always
  restarting from the router (upgraded ADK to v0.1.12)
• **Removal of artificial limits**: Eliminates all hard-coded truncation constants
  (MAX_TOOL_OUTPUT_CHARS, MAX_MCP_PROXY_RESPONSE_BYTES, MAX_RESPONSE_CHAIN_DEPTH,
  MAX_CONTINUATION_ITERATIONS, MAX_MESSAGE_CONTENT_LENGTH, etc.)
• **Comprehensive test coverage**: 162 test suites with 2644 passing tests covering admin routes,
  multi-agent orchestration, tool execution, configuration merging, message processing, and all HTTP
  endpoints
• **New HTTP client**: ResponsesApiClient with streaming, retry logic, and TLS verification
  support
• **Configuration enhancements**: ConfigLoader with validation, defaults, and support for complex
  nested configs including multi-agent setups
• **Code quality improvements**: McpAuthService double-map-lookup fix, simplified sorting in
  TfIdfEmbedder
• **ESLint configuration**: Added workspace-level ESLint setup for consistent linting
Diagram
flowchart LR
  A["ADK Orchestrator<br/>with resumeState"] -- "saves active agent" --> B["Continuation State"]
  B -- "routes to active agent" --> C["Follow-up Messages"]
  D["Remove Limits<br/>MAX_TOOL_OUTPUT_CHARS<br/>MAX_MCP_PROXY_RESPONSE_BYTES<br/>etc."] --> E["Unrestricted Output"]
  F["ResponsesApiClient<br/>Streaming & Retry"] --> G["Tool Execution"]
  H["ConfigLoader<br/>with Validation"] --> I["Multi-agent Config"]
  J["162 Test Suites<br/>2644 Tests"] --> K["Comprehensive Coverage"]
Loading

Grey Divider

File Changes

1. workspaces/augment/plugins/augment-backend/src/routes/adminRoutes.test.ts 🧪 Tests +1698/-0

Admin routes comprehensive test coverage with 1698 test cases

• Comprehensive test suite for admin routes with 1698 lines covering permission enforcement, config
 CRUD operations, and document management
• Tests for RAG features including vector store management, document upload/delete, and search
 functionality
• Validation tests for configuration keys, MCP server setup, and SSRF protection
• Integration tests for branding merge, cache invalidation, and runtime config propagation

workspaces/augment/plugins/augment-backend/src/routes/adminRoutes.test.ts


2. workspaces/augment/plugins/augment-backend/src/providers/llamastack/__tests__/multiAgent.e2e.test.ts 🧪 Tests +1451/-0

Multi-agent orchestration end-to-end integration tests

• End-to-end integration tests for multi-agent orchestration with mock and real LlamaStack server
 support
• Tests cover single-agent chat, multi-agent handoffs, streaming responses, and agent-as-tool
 scenarios
• Mock server implementation simulating LlamaStack Responses API with deterministic behavior for CI
 testing
• Validates agent routing continuity, reasoning events, handoff metadata, and error handling across
 agent boundaries

workspaces/augment/plugins/augment-backend/src/providers/llamastack/tests/multiAgent.e2e.test.ts


3. workspaces/augment/plugins/augment-backend/src/providers/responses-api/client/ResponsesApiClient.ts ✨ Enhancement +454/-0

Responses API HTTP client with streaming and retry support

• HTTP client for OpenAI Responses API with support for JSON requests, FormData uploads, and SSE
 streaming
• Implements retry logic for transient failures with exponential backoff
• Handles TLS certificate verification with configurable skipTlsVerify for enterprise environments
• Includes structured error handling with ResponsesApiError for validation, tool type, and context
 overflow detection

workspaces/augment/plugins/augment-backend/src/providers/responses-api/client/ResponsesApiClient.ts


View more (111)
4. workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.test.ts 🧪 Tests +1191/-0

AdkOrchestrator comprehensive test coverage with agent continuity

• Comprehensive test suite (1191 lines) for AdkOrchestrator covering chat, streaming, tool
 discovery, and multi-turn agent continuity
• Tests HITL approval mirroring, tool caching with generation sync, and concurrent request isolation
• Validates resumeState mechanism for agent routing continuity across conversation turns
• Tests error handling, stream abortion, and tool execution with backend executors

workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.test.ts


5. workspaces/augment/plugins/augment-backend/src/services/RuntimeConfigResolver.test.ts 🧪 Tests +1089/-0

RuntimeConfigResolver configuration merging and caching tests

• Extensive test suite (1089 lines) for RuntimeConfigResolver covering YAML baseline and DB
 override merging
• Tests caching behavior, graceful degradation on DB failures, and concurrent resolve calls
• Validates type safety, vectorStoreConfig completeness, and cache snapshot isolation
• Tests production parameters (maxToolCalls, temperature, guardrails) and MCP server merging logic

workspaces/augment/plugins/augment-backend/src/services/RuntimeConfigResolver.test.ts


6. workspaces/augment/plugins/augment-backend/src/providers/responses-api/tools/BackendToolExecutor.test.ts 🧪 Tests +1016/-0

BackendToolExecutor MCP tool discovery and execution tests

• Large test suite (1016 lines) for BackendToolExecutor covering MCP tool discovery and execution
• Tests tool prefixing/unprefixing, caching with TTL, and concurrent discovery deduplication
• Validates SSRF guard, suffix-match fallback for unprefixed tool names, and partial re-discovery
 preservation
• Tests session expiry auto-reconnect, atomic swap correctness, and client lifecycle management

workspaces/augment/plugins/augment-backend/src/providers/responses-api/tools/BackendToolExecutor.test.ts


7. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ToolsBuilder.test.ts 🧪 Tests +278/-0

ToolsBuilder tool composition and configuration tests

• Test suite (278 lines) for buildTools function covering tool composition from multiple sources
• Tests file_search (RAG), MCP servers, custom functions, web_search, and code_interpreter tools
• Validates tool filtering, header injection, and approval configuration
• Tests combinations of all tool types and proper handling of missing dependencies

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ToolsBuilder.test.ts


8. workspaces/augment/.eslintrc.js ⚙️ Configuration changes +1/-0

ESLint configuration for augment workspace

• New ESLint configuration file that extends the root project's ESLint configuration
• Enables consistent linting rules across the augment workspace

workspaces/augment/.eslintrc.js


9. workspaces/augment/plugins/augment-backend/src/router.test.ts 🧪 Tests +977/-0

Complete router endpoint test coverage with auth and sessions

• Comprehensive test suite for the router with 977 lines covering all HTTP endpoints
• Tests for health, status, documents, branding, conversations, chat, sync, safety, evaluation,
 workflows, and quick-actions endpoints
• Permission and authentication tests including 403/401 error scenarios and security mode handling
• Session management tests covering CRUD operations, message retrieval, and cascade deletion

workspaces/augment/plugins/augment-backend/src/router.test.ts


10. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ResponsesApiProvider.test.ts 🧪 Tests +993/-0

ResponsesApiProvider capability tests with stream event handling

• 993-line test suite for ResponsesApiProvider covering initialization and delegation to
 orchestrator
• Tests for chat, streaming, conversations, RAG, safety, and evaluation capabilities
• Stream event normalization tests including HITL approval events and backend tool execution
• Model testing functionality with connection validation and inference verification

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ResponsesApiProvider.test.ts


11. workspaces/augment/plugins/augment-backend/src/providers/llamastack/config/ConfigLoader.ts ✨ Enhancement +891/-0

Comprehensive configuration loader with validation and defaults

• 891-line configuration loader for parsing app-config.yaml with validation and error handling
• Loads Llama Stack settings, security config, documents, MCP servers, workflows, and multi-agent
 configurations
• Supports both single and array-based vectorStoreIds with backward compatibility
• Parses complex nested configs for tool choice, reasoning, tool scoping, and hybrid search with
 sensible defaults

workspaces/augment/plugins/augment-backend/src/providers/llamastack/config/ConfigLoader.ts


12. workspaces/augment/plugins/augment-backend/src/providers/llamastack/MessageProcessor.test.ts 🧪 Tests +619/-0

Message processing utility tests with reasoning and handoff support

• 619-line test suite for message processing utilities including content extraction and conversation
 item handling
• Tests for extractContentFromItem, extractUserInputFromRaw, getInputText, and
 processConversationItems functions
• Covers reasoning extraction, tool call grouping, RAG source attachment, and multi-agent handoff
 detection
• Tests for system prompt filtering, orphaned item handling, and edge cases like empty messages

workspaces/augment/plugins/augment-backend/src/providers/llamastack/MessageProcessor.test.ts


13. workspaces/augment/plugins/augment-backend/.eslintrc.js ⚙️ Configuration changes +2/-0

ESLint configuration setup

• Minimal ESLint configuration file using Backstage CLI factory pattern

workspaces/augment/plugins/augment-backend/.eslintrc.js


14. workspaces/augment/.changeset/augment-initial-release.md Additional files +7/-0

...

workspaces/augment/.changeset/augment-initial-release.md


15. workspaces/augment/.changeset/config.json Additional files +14/-0

...

workspaces/augment/.changeset/config.json


16. workspaces/augment/.dockerignore Additional files +8/-0

...

workspaces/augment/.dockerignore


17. workspaces/augment/.eslintignore Additional files +1/-0

...

workspaces/augment/.eslintignore


18. workspaces/augment/.prettierignore Additional files +9/-0

...

workspaces/augment/.prettierignore


19. workspaces/augment/README.md Additional files +80/-8

...

workspaces/augment/README.md


20. workspaces/augment/app-config.yaml Additional files +99/-0

...

workspaces/augment/app-config.yaml


21. workspaces/augment/backstage.json Additional files +1/-0

...

workspaces/augment/backstage.json


22. workspaces/augment/bcp.json Additional files +3/-0

...

workspaces/augment/bcp.json


23. workspaces/augment/catalog-info.yaml Additional files +9/-0

...

workspaces/augment/catalog-info.yaml


24. workspaces/augment/package.json Additional files +64/-9

...

workspaces/augment/package.json


25. workspaces/augment/packages/app/.eslintrc.js Additional files +1/-0

...

workspaces/augment/packages/app/.eslintrc.js


26. workspaces/augment/packages/app/knip-report.md Additional files +13/-0

...

workspaces/augment/packages/app/knip-report.md


27. workspaces/augment/packages/app/package.json Additional files +77/-0

...

workspaces/augment/packages/app/package.json


28. workspaces/augment/packages/app/public/index.html Additional files +60/-0

...

workspaces/augment/packages/app/public/index.html


29. workspaces/augment/packages/app/public/manifest.json Additional files +15/-0

...

workspaces/augment/packages/app/public/manifest.json


30. workspaces/augment/packages/app/public/robots.txt Additional files +2/-0

...

workspaces/augment/packages/app/public/robots.txt


31. workspaces/augment/packages/app/src/App.tsx Additional files +118/-0

...

workspaces/augment/packages/app/src/App.tsx


32. workspaces/augment/packages/app/src/apis.ts Additional files +34/-0

...

workspaces/augment/packages/app/src/apis.ts


33. workspaces/augment/packages/app/src/components/Root/LogoFull.tsx Additional files +51/-0

...

workspaces/augment/packages/app/src/components/Root/LogoFull.tsx


34. workspaces/augment/packages/app/src/components/Root/LogoIcon.tsx Additional files +51/-0

...

workspaces/augment/packages/app/src/components/Root/LogoIcon.tsx


35. workspaces/augment/packages/app/src/components/Root/Root.tsx Additional files +109/-0

...

workspaces/augment/packages/app/src/components/Root/Root.tsx


36. workspaces/augment/packages/app/src/components/Root/index.ts Additional files +16/-0

...

workspaces/augment/packages/app/src/components/Root/index.ts


37. workspaces/augment/packages/app/src/components/catalog/EntityPage.tsx Additional files +404/-0

...

workspaces/augment/packages/app/src/components/catalog/EntityPage.tsx


38. workspaces/augment/packages/app/src/components/search/SearchPage.tsx Additional files +138/-0

...

workspaces/augment/packages/app/src/components/search/SearchPage.tsx


39. workspaces/augment/packages/app/src/index.tsx Additional files +20/-0

...

workspaces/augment/packages/app/src/index.tsx


40. workspaces/augment/packages/backend/.eslintrc.js Additional files +1/-0

...

workspaces/augment/packages/backend/.eslintrc.js


41. workspaces/augment/packages/backend/knip-report.md Additional files +15/-0

...

workspaces/augment/packages/backend/knip-report.md


42. workspaces/augment/packages/backend/package.json Additional files +54/-0

...

workspaces/augment/packages/backend/package.json


43. workspaces/augment/packages/backend/src/index.ts Additional files +45/-0

...

workspaces/augment/packages/backend/src/index.ts


44. workspaces/augment/plugins/augment-backend/README.md Additional files +242/-0

...

workspaces/augment/plugins/augment-backend/README.md


45. workspaces/augment/plugins/augment-backend/config.d.ts Additional files +1043/-0

...

workspaces/augment/plugins/augment-backend/config.d.ts


46. workspaces/augment/plugins/augment-backend/dev/index.ts Additional files +38/-0

...

workspaces/augment/plugins/augment-backend/dev/index.ts


47. workspaces/augment/plugins/augment-backend/knip-report.md Additional files +9/-0

...

workspaces/augment/plugins/augment-backend/knip-report.md


48. workspaces/augment/plugins/augment-backend/package.json Additional files +82/-0

...

workspaces/augment/plugins/augment-backend/package.json


49. workspaces/augment/plugins/augment-backend/report.api.md Additional files +1031/-0

...

workspaces/augment/plugins/augment-backend/report.api.md


50. workspaces/augment/plugins/augment-backend/src/constants.ts Additional files +186/-0

...

workspaces/augment/plugins/augment-backend/src/constants.ts


51. workspaces/augment/plugins/augment-backend/src/extensions.ts Additional files +78/-0

...

workspaces/augment/plugins/augment-backend/src/extensions.ts


52. workspaces/augment/plugins/augment-backend/src/index.ts Additional files +134/-0

...

workspaces/augment/plugins/augment-backend/src/index.ts


53. workspaces/augment/plugins/augment-backend/src/middleware/security.test.ts Additional files +521/-0

...

workspaces/augment/plugins/augment-backend/src/middleware/security.test.ts


54. workspaces/augment/plugins/augment-backend/src/middleware/security.ts Additional files +200/-0

...

workspaces/augment/plugins/augment-backend/src/middleware/security.ts


55. workspaces/augment/plugins/augment-backend/src/parsers/chatRequestParsers.test.ts Additional files +332/-0

...

workspaces/augment/plugins/augment-backend/src/parsers/chatRequestParsers.test.ts


56. workspaces/augment/plugins/augment-backend/src/parsers/chatRequestParsers.ts Additional files +145/-0

...

workspaces/augment/plugins/augment-backend/src/parsers/chatRequestParsers.ts


57. workspaces/augment/plugins/augment-backend/src/plugin.ts Additional files +278/-0

...

workspaces/augment/plugins/augment-backend/src/plugin.ts


58. workspaces/augment/plugins/augment-backend/src/providers/ProviderManager.test.ts Additional files +151/-0

...

workspaces/augment/plugins/augment-backend/src/providers/ProviderManager.test.ts


59. workspaces/augment/plugins/augment-backend/src/providers/ProviderManager.ts Additional files +127/-0

...

workspaces/augment/plugins/augment-backend/src/providers/ProviderManager.ts


60. workspaces/augment/plugins/augment-backend/src/providers/factory.ts Additional files +129/-0

...

workspaces/augment/plugins/augment-backend/src/providers/factory.ts


61. workspaces/augment/plugins/augment-backend/src/providers/index.ts Additional files +47/-0

...

workspaces/augment/plugins/augment-backend/src/providers/index.ts


62. workspaces/augment/plugins/augment-backend/src/providers/llamastack/AgentGraphManager.test.ts Additional files +280/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/AgentGraphManager.test.ts


63. workspaces/augment/plugins/augment-backend/src/providers/llamastack/AgentGraphManager.ts Additional files +204/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/AgentGraphManager.ts


64. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalContinuationExecutor.test.ts Additional files +509/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalContinuationExecutor.test.ts


65. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalContinuationExecutor.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalContinuationExecutor.ts


66. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalHandler.test.ts Additional files +224/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalHandler.test.ts


67. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalHandler.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ApprovalHandler.ts


68. workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendApprovalHandler.test.ts Additional files +230/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendApprovalHandler.test.ts


69. workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendApprovalHandler.ts Additional files +389/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendApprovalHandler.ts


70. workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendApprovalStore.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendApprovalStore.ts


71. workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendToolEvents.test.ts Additional files +102/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendToolEvents.test.ts


72. workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendToolExecutor.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/BackendToolExecutor.ts


73. workspaces/augment/plugins/augment-backend/src/providers/llamastack/BrandingConfigLoader.ts Additional files +17/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/BrandingConfigLoader.ts


74. workspaces/augment/plugins/augment-backend/src/providers/llamastack/CapabilityBuilders.test.ts Additional files +830/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/CapabilityBuilders.test.ts


75. workspaces/augment/plugins/augment-backend/src/providers/llamastack/CapabilityBuilders.ts Additional files +373/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/CapabilityBuilders.ts


76. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ChatDepsBuilder.test.ts Additional files +245/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ChatDepsBuilder.test.ts


77. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ChatDepsBuilder.ts Additional files +127/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ChatDepsBuilder.ts


78. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ClientManager.test.ts Additional files +221/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ClientManager.test.ts


79. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ClientManager.ts Additional files +149/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ClientManager.ts


80. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigLoader.test.ts Additional files +413/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigLoader.test.ts


81. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigLoader.ts Additional files +17/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigLoader.ts


82. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigResolutionService.test.ts Additional files +206/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigResolutionService.test.ts


83. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigResolutionService.ts Additional files +17/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConfigResolutionService.ts


84. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationFacade.test.ts Additional files +426/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationFacade.test.ts


85. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationFacade.ts Additional files +16/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationFacade.ts


86. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationHelpers.test.ts Additional files +236/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationHelpers.test.ts


87. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationHelpers.ts Additional files +16/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationHelpers.ts


88. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationRegistry.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationRegistry.ts


89. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationService.test.ts Additional files +597/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationService.test.ts


90. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationService.ts Additional files +16/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ConversationService.ts


91. workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentLister.test.ts Additional files +388/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentLister.test.ts


92. workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentLister.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentLister.ts


93. workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentSyncService.test.ts Additional files +661/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentSyncService.test.ts


94. workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentSyncService.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentSyncService.ts


95. workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentUploader.test.ts Additional files +341/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentUploader.test.ts


96. workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentUploader.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/DocumentUploader.ts


97. workspaces/augment/plugins/augment-backend/src/providers/llamastack/EvaluationService.test.ts Additional files +536/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/EvaluationService.test.ts


98. workspaces/augment/plugins/augment-backend/src/providers/llamastack/EvaluationService.ts Additional files +17/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/EvaluationService.ts


99. workspaces/augment/plugins/augment-backend/src/providers/llamastack/InputItemsNormalizer.test.ts Additional files +101/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/InputItemsNormalizer.test.ts


100. workspaces/augment/plugins/augment-backend/src/providers/llamastack/InputItemsNormalizer.ts Additional files +16/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/InputItemsNormalizer.ts


101. workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackApiError.test.ts Additional files +143/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackApiError.test.ts


102. workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackOrchestrator.test.ts Additional files +203/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackOrchestrator.test.ts


103. workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackOrchestrator.ts Additional files +20/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackOrchestrator.ts


104. workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackProvider.test.ts Additional files +887/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackProvider.test.ts


105. workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackProvider.ts Additional files +20/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/LlamaStackProvider.ts


106. workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpAuthService.test.ts Additional files +405/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpAuthService.test.ts


107. workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpAuthService.ts Additional files +17/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpAuthService.ts


108. workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpConfigLoader.ts Additional files +17/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpConfigLoader.ts


109. workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpToolBuilder.ts Additional files +18/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/McpToolBuilder.ts


110. workspaces/augment/plugins/augment-backend/src/providers/llamastack/MessageProcessor.ts Additional files +16/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/MessageProcessor.ts


111. workspaces/augment/plugins/augment-backend/src/providers/llamastack/ModerationService.ts Additional files +141/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/ModerationService.ts


112. workspaces/augment/plugins/augment-backend/src/providers/llamastack/OrchestratorInitializer.test.ts Additional files +307/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/OrchestratorInitializer.test.ts


113. workspaces/augment/plugins/augment-backend/src/providers/llamastack/OrchestratorInitializer.ts Additional files +234/-0

...

workspaces/augment/plugins/augment-backend/src/providers/llamastack/OrchestratorInitializer.ts


114. Additional files not shown Additional files +0/-0

...

Additional files not shown


Grey Divider

Qodo Logo

@rhdh-qodo-merge
Copy link
Copy Markdown

rhdh-qodo-merge Bot commented Mar 25, 2026

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (0) 📐 Spec deviations (0)

Grey Divider


Action required

1. RBAC flag ignored 🐞 Bug ⛨ Security
Description
requirePluginAccess always calls permissions.authorize() even when permission.enabled is
false, so installs that disable RBAC (or rely on the default) will still go through permission
evaluation instead of a non-RBAC mechanism.
Code

workspaces/augment/plugins/augment-backend/src/middleware/security.ts[R85-92]

+    try {
+      const credentials = await httpAuth.credentials(req);
+
+      const decision = (
+        await permissions.authorize([{ permission: augmentAccessPermission }], {
+          credentials,
+        })
+      )[0];
Evidence
createRouter computes permissionsEnabled from config and passes it into
createSecurityMiddleware, but requirePluginAccess does not branch on it and always authorizes
augment.access via the permissions service. This makes the permission.enabled toggle ineffective
for plugin access checks (only admin checks respect it).

workspaces/augment/plugins/augment-backend/src/router.ts[142-176]
workspaces/augment/plugins/augment-backend/src/middleware/security.ts[43-112]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`requirePluginAccess` currently always performs a Backstage RBAC permission evaluation (`permissions.authorize`) regardless of the `permission.enabled` flag.

### Issue Context
`permissionsEnabled` is already computed in the router and passed into `createSecurityMiddleware`, but only `checkIsAdmin` uses it. Plugin access should either:
- Short-circuit with a clear error when permissions are disabled but security mode requires RBAC, or
- Provide an explicit non-RBAC fallback mechanism (if intended).

### Fix Focus Areas
- workspaces/augment/plugins/augment-backend/src/middleware/security.ts[74-130]
- workspaces/augment/plugins/augment-backend/src/router.ts[142-176]

### Expected change
Gate the RBAC call behind `permissionsEnabled` (and define the intended behavior when it’s false, e.g. return 503 with guidance, or fall back to a config allowlist if supported).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Backend tool cache stale 🐞 Bug ⛯ Reliability
Description
Backend MCP tool discovery caching is keyed only by server IDs, so changing a server URL/headers
while keeping the same ID can keep the old registry/clients active until TTL expiry, routing tool
calls to the wrong endpoint.
Code

workspaces/augment/plugins/augment-backend/src/providers/responses-api/tools/BackendToolExecutor.ts[R276-316]

+  private static serverCacheKey(servers: MCPServerConfig[]): string {
+    return servers
+      .map(s => s.id)
+      .sort((a, b) => a.localeCompare(b))
+      .join(',');
+  }
+
+  /**
+   * Returns cached tools if the cache is still valid (within TTL) and
+   * was populated for the same set of servers.
+   * Otherwise triggers a fresh `discoverTools()` call.
+   * Concurrent callers share a single in-flight request only when they
+   * request the same server set.
+   */
+  async ensureToolsDiscovered(
+    servers: MCPServerConfig[],
+  ): Promise<ResponsesApiFunctionTool[]> {
+    const key = BackendToolExecutor.serverCacheKey(servers);
+    if (
+      this.cachedTools &&
+      this.cachedServerKey === key &&
+      this.isWithinTtl()
+    ) {
+      return this.cachedTools;
+    }
+    if (this.inflightDiscovery !== null && this.inflightServerKey === key) {
+      return this.inflightDiscovery;
+    }
+    this.inflightServerKey = key;
+    this.inflightDiscovery = this.discoverTools(servers)
+      .then(tools => {
+        this.cachedTools = tools;
+        this.cachedServerKey = key;
+        this.lastDiscoveryTimestamp = Date.now();
+        return tools;
+      })
+      .finally(() => {
+        this.inflightDiscovery = null;
+        this.inflightServerKey = '';
+      });
+    return this.inflightDiscovery;
Evidence
BackendToolExecutor.serverCacheKey uses only servers.map(s => s.id) and ensureToolsDiscovered
returns cached results as long as the server IDs set matches and TTL is valid. ChatDepsBuilder
invalidates the executor cache only when the *IDs* change, not when URLs/headers change. Admin UI
supports editing MCP server url for a given id, so updates may not take effect for up to
BACKEND_TOOL_DISCOVERY_TTL_MS and may keep calling the old serverUrl stored in the registry.

workspaces/augment/plugins/augment-backend/src/providers/responses-api/tools/BackendToolExecutor.ts[276-317]
workspaces/augment/plugins/augment-backend/src/constants.ts[154-158]
workspaces/augment/plugins/augment-backend/src/providers/llamastack/ChatDepsBuilder.ts[76-99]
workspaces/augment/plugins/augment-backend/src/routes/adminConfigRoutes.ts[118-166]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
Backend MCP tool discovery caching does not invalidate when MCP server connection details change (URL/headers) but IDs remain constant.

### Issue Context
`BackendToolExecutor.ensureToolsDiscovered` uses a server cache key derived only from server IDs. The tool registry stores `serverUrl` per tool; if an admin edits the server URL for the same ID, the executor may keep using the old URL until TTL expires. `ChatDepsBuilder` invalidation logic also compares only IDs.

### Fix Focus Areas
- workspaces/augment/plugins/augment-backend/src/providers/responses-api/tools/BackendToolExecutor.ts[276-317]
- workspaces/augment/plugins/augment-backend/src/providers/llamastack/ChatDepsBuilder.ts[76-99]

### Suggested fix
Include a stable hash of `{id,url,headers,type}` (and possibly `authRef/oauth/serviceAccount`) in both:
- `BackendToolExecutor.serverCacheKey`
- `ChatDepsBuilder`’s `lastServerIds` tracking (rename accordingly)

So URL/headers edits trigger `invalidateCache()` immediately.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Deps built for default agent 🐞 Bug ✓ Correctness
Description
AdkOrchestrator.buildRunOptions always calls buildDepsForAgent(defaultAgent), so when resuming
from a non-default active agent the per-agent dependencies/config (if any) will not be built for the
actual starting agent.
Code

workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[R265-279]

+    const defaultAgent = snapshot.agents.get(snapshot.defaultAgentKey);
+    if (!defaultAgent) {
+      throw new Error(
+        `[AdkOrchestrator] Default agent "${snapshot.defaultAgentKey}" not found`,
+      );
+    }
+
+    const deps = await buildDepsForAgent(defaultAgent);
+    const model = new BackstageModelAdapter(this.chatService, deps.client);
+    const adkConfig = toAdkEffectiveConfig(deps.config);
+    const adkAgents = this.snapshotToAdkAgents(snapshot);
+    const adkMcpServers = deps.mcpServers.map(toAdkMcpServerConfig);
+    const functionTools = await this.discoverBackendTools(deps);
+
+    const toolResolver = this.buildToolResolver(functionTools, deps);
Evidence
Both chat() and chatStream() may set agentRef.key from resumeState.currentAgentKey, but
buildRunOptions() ignores that and always selects snapshot.defaultAgentKey to pass into
buildDepsForAgent. The BuildDepsForAgent type is explicitly “per-agent ChatDeps”, so this
mismatch can yield incorrect deps (e.g., agent-specific config/clients/tooling) when a conversation
resumes mid-graph.

workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[257-296]
workspaces/augment/plugins/augment-backend/src/providers/responses-api/agents/agentGraph.ts[54-56]
workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[101-132]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
When resuming from a continuation state, the orchestrator chooses the correct `currentAgentKey` for routing, but still builds `ChatDeps` using the default agent.

### Issue Context
`BuildDepsForAgent` is intended to produce per-agent dependencies. Using the default agent can be wrong when the run is resumed from a different agent.

### Fix Focus Areas
- workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[101-132]
- workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[257-296]

### Suggested fix
In `buildRunOptions`, select the agent to pass into `buildDepsForAgent` based on:
- `resumeState?.currentAgentKey` (if present and exists in snapshot), otherwise
- `snapshot.defaultAgentKey`.

This aligns deps building with actual run entry agent.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

Follow-up to redhat-developer#2555. Adds multi-agent orchestration improvements,
removes artificial limits, and fixes multi-turn agent routing.

Multi-agent orchestration:
- ADK-based orchestrator with HITL approval mirroring, tool resolver,
  conversation state management, and agent handoffs
- VectorStoreFacade lazy init and single-flight improvements
- ResponsesApiService previous_response_id fix

Limit removals (per design decision):
- Remove MAX_TOOL_OUTPUT_CHARS, MAX_MCP_PROXY_RESPONSE_BYTES,
  MAX_RESPONSE_CHAIN_DEPTH, MAX_CONTINUATION_ITERATIONS,
  MAX_MESSAGE_CONTENT_LENGTH, and all truncation logic
- Remove maxToolOutputChars and maxResponseChars config
- Keep MAX_AUTO_REAPPROVALS (Llama Stack bug workaround),
  auth constants, timeouts, and cache limits

Routing fix:
- Upgrade @augment-adk/augment-adk from 0.1.11 to 0.1.12
- Restore createContinuationState and resumeState so follow-up
  messages resume from the active agent instead of restarting
  from the router

Frontend enhancements:
- ToggleSwitch component, selectMenuProps helper
- Vector store UI and admin panel improvements

Code quality:
- McpAuthService double-map-lookup fix
- TfIdfEmbedder sort simplification

Documentation:
- CONFIG_REFERENCE.md, TROUBLESHOOTING.md

Tests:
- Comprehensive coverage (162 suites, 2644 tests passing)
- 6 resumeState continuity tests
@rrbanda rrbanda force-pushed the feat/augment-all-improvements branch from 066e242 to d42b83b Compare March 25, 2026 02:26
@rhdh-gh-app
Copy link
Copy Markdown

rhdh-gh-app Bot commented Mar 25, 2026

Missing Changesets

The following package(s) are changed by this PR but do not have a changeset:

  • @red-hat-developer-hub/backstage-plugin-augment-backend
  • @red-hat-developer-hub/backstage-plugin-augment-common
  • @red-hat-developer-hub/backstage-plugin-augment

See CONTRIBUTING.md for more information about how to add changesets.

Changed Packages

Package Name Package Path Changeset Bump Current Version
@red-hat-developer-hub/backstage-plugin-augment-backend workspaces/augment/plugins/augment-backend none v0.1.0
@red-hat-developer-hub/backstage-plugin-augment-common workspaces/augment/plugins/augment-common none v0.1.0
@red-hat-developer-hub/backstage-plugin-augment workspaces/augment/plugins/augment none v0.1.0

- Use localeCompare for string sorts (TfIdfEmbedder, BackendToolExecutor.test)
- Fix Promise-in-boolean conditionals with !== undefined (McpAuthService, VectorStoreFacade)
- Remove unnecessary non-null assertions (AdkOrchestrator, VectorStoreFacade)
- Reduce cognitive complexity by extracting helpers:
  - AdkOrchestrator: resolveAgentContinuity(), mirrorPendingApprovals()
  - chatRoutes: resolveConversationId()
  - BackendToolExecutor: processDiscoveryResults(), preserveFailedServerTools(), etc.
- Move makeMultiAgentSnapshot to outer describe scope (AdkOrchestrator.test)
- Mark ToggleSwitch props as readonly
Comment on lines 265 to +279
const defaultAgent = snapshot.agents.get(snapshot.defaultAgentKey);
if (!defaultAgent) {
throw new Error(
`[AdkOrchestrator] Default agent "${snapshot.defaultAgentKey}" not found`,
);
}

const deps = await buildDepsForAgent(defaultAgent);
const model = new BackstageModelAdapter(this.chatService, deps.client);
const adkConfig = toAdkEffectiveConfig(deps.config);
const adkAgents = this.snapshotToAdkAgents(snapshot);
const adkMcpServers = deps.mcpServers.map(toAdkMcpServerConfig);
const functionTools = await this.discoverBackendTools(deps);

const toolResolver = this.buildToolResolver(functionTools, deps);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Deps built for default agent 🐞 Bug ✓ Correctness

AdkOrchestrator.buildRunOptions always calls buildDepsForAgent(defaultAgent), so when resuming
from a non-default active agent the per-agent dependencies/config (if any) will not be built for the
actual starting agent.
Agent Prompt
### Issue description
When resuming from a continuation state, the orchestrator chooses the correct `currentAgentKey` for routing, but still builds `ChatDeps` using the default agent.

### Issue Context
`BuildDepsForAgent` is intended to produce per-agent dependencies. Using the default agent can be wrong when the run is resumed from a different agent.

### Fix Focus Areas
- workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[101-132]
- workspaces/augment/plugins/augment-backend/src/providers/llamastack/adk-adapters/AdkOrchestrator.ts[257-296]

### Suggested fix
In `buildRunOptions`, select the agent to pass into `buildDepsForAgent` based on:
- `resumeState?.currentAgentKey` (if present and exists in snapshot), otherwise
- `snapshot.defaultAgentKey`.

This aligns deps building with actual run entry agent.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

rrbanda added 2 commits March 24, 2026 22:48
- Remove maxResponseChars from MCPServerConfig in report.api.md
  to match the source type (field was removed in limit-removal changes)
- Move makeMultiAgentSnapshot to module scope (outside describe block)
  to satisfy SonarCloud's "no functions in nested scope" rule
…spots

Replace http:// with https:// in test fixture data (mock baseUrl and
serverUrl values) to clear SonarCloud security hotspot warnings about
clear-text protocol usage.
@sonarqubecloud
Copy link
Copy Markdown

@pkliczewski pkliczewski merged commit b31900f into redhat-developer:main Mar 25, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants