feat: ChatModelAgentMiddleware, agent callbacks and language setting#809
Merged
shentongmartin merged 65 commits intomainfrom Mar 9, 2026
Merged
feat: ChatModelAgentMiddleware, agent callbacks and language setting#809shentongmartin merged 65 commits intomainfrom
shentongmartin merged 65 commits intomainfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #809 +/- ##
==========================================
+ Coverage 80.49% 81.61% +1.12%
==========================================
Files 129 146 +17
Lines 13050 15932 +2882
==========================================
+ Hits 10504 13003 +2499
- Misses 1745 1987 +242
- Partials 801 942 +141 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
shentongmartin
force-pushed
the
alpha/08
branch
from
cc8c005 to
e5dfcea
Compare
…722) * feat: add enhanced tool support with multimodal output capabilities and improve message formatting This commit introduces enhanced tool interfaces that support structured multimodal outputs, enabling tools to return rich content beyond simple text responses. Key Changes: 1. New Enhanced Tool Interfaces: - Added EnhancedInvokableTool and EnhancedStreamableTool interfaces for multimodal tool execution - Both interfaces use ToolCallInfo as input and return ToolResult for structured output 2. ToolResult Schema: - Introduced ToolResult type to represent multimodal tool outputs - Supports multiple content types: text, image, audio, video, and file - Added ToolOutputPart with Index field for streaming chunk merging - Implemented ToMessageInputParts() for seamless model integration 3. ToolsNode Enhancements: - Extended ToolsNode to support both legacy and enhanced tool types - Added automatic conversion between invokable and streamable endpoints - Implemented middleware support for enhanced tools - Enhanced interrupt and rerun mechanism to handle ToolResult 4. React Agent Integration: - Introduce enhancedToolResultSender and enhancedStreamToolResultSender types - Support sending *schema.ToolResult with multimodal content (images, audio, video, files) - Implement EnhancedInvokable and EnhancedStreamable middleware in tool result collector 5. Message.String() Enhancement: - Add formatting support for UserInputMultiContent, AssistantGenMultiContent, and MultiContent - Implement formatInputPart, formatOutputPart, and formatChatMessagePart helper functions - Create mediaPartFormatter interface with wrapper types for unified media formatting 6. User Input Multi-Content Concatenation: - Implement concatUserMultiContent function for merging MessageInputPart slices - Support text and base64 audio merging with proper MIME type handling - Integrate into ConcatMessages function 7. Callback System: - Added CallbackInput and CallbackOutput types for tool callbacks - Implemented conversion functions for different callback input/output types 8. Comprehensive Test Coverage: - Added tests for enhanced invokable and streamable tools - Added TestMessageString with 14 test cases covering various message types Impact: - Enables tools to return rich multimodal content (images, audio, video, files) - Provides foundation for more sophisticated tool implementations - Maintains full backward compatibility with existing tool ecosystem
…string]bool (#737) - Update ToolsConfig.ReturnDirectly type - Update ChatModelAgentContext.ReturnDirectly type - Update internal types: chatModelAgentExecCtx, execContext, reactConfig - Update all related function signatures - Update test files with new type syntax Change-Id: I7d819f1c44da91b76cf9a9f867a88008068153b8
…AfterModelRewriteState (#745)
…nd reduction packages (#749)
Change-Id: I61cd3709e78b5e1ef1fe169572913e7e35946d56
* feat(adk): tool reduction middleware * refactor(adk): move ancient reduction middleware to internal package * chore(adk): reduction mw add i18n, rename TokenCounter * chore(adk): add reduction mw comments * chore(adk): refactor reduction mw config field, tool stream copy
…to value types and added extensive test suite for backend_inmemory GrepRaw implementation (#750) * feat(adk): enhance GrepRequest with value-type parameters and comprehensive test coverage Enhanced GrepRequest API by converting pointer parameters to value types and added extensive test suite for backend_inmemory GrepRaw implementation. Key Improvements: 1. GrepRequest Parameter Enhancement (backend.go): - Converted AfterLines, BeforeLines, ContextLines, HeadLimit, Offset from *int to int - Simplified parameter usage: direct value assignment instead of pointer references - Added clear semantics: values <= 0 indicate unset/default state - Improved API ergonomics and reduced boilerplate code 2. Backend_inmemory GrepRaw Support (backend_inmemory.go): - Updated parameter handling logic to support value-based parameters - Enhanced applyContext function with proper value validation (> 0 checks) - Improved applyPagination to handle int parameters correctly - Maintained backward compatibility while simplifying implementation 3. Middleware Integration (middlewares/filesystem/filesystem.go): - Added seamless pointer-to-value conversion for parameter passing - Implemented proper default value handling for HeadLimit and Offset - Ensured compatibility between middleware and backend layers 4. Comprehensive Test Suite (backend_inmemory_test.go): - Added 40+ table-driven test scenarios for GrepRaw functionality - Implemented concurrent safety tests (50 goroutines) - Added edge case tests (long lines, unicode, special characters) - Created detailed content validation tests - Added 4 performance benchmark tests - Included complete test coverage documentation Test Coverage: - All output modes: 100% (files_with_matches, content, count) - Parameter combinations: comprehensive coverage - Context lines: all combinations tested - Pagination: HeadLimit + Offset scenarios - Error handling: invalid patterns, boundary conditions - Concurrency: thread-safety verified - Performance: benchmarked common use cases Impact: - Simplified API: ContextLines: 1 instead of &contextLines - Better usability: no pointer variable creation needed - Enhanced reliability: comprehensive test coverage added - Improved maintainability: clear parameter semantics Files Changed: - adk/filesystem/backend.go: parameter type definitions - adk/filesystem/backend_inmemory.go: GrepRaw implementation - adk/filesystem/backend_inmemory_test.go: comprehensive test suite - adk/middlewares/filesystem/filesystem.go: middleware integration - adk/middlewares/filesystem/filesystem_test.go: test updates - adk/middlewares/filesystem/prompt.go: minor adjustments * feat(adk): add jsonschema descriptions to filesystem tool arguments Add detailed JSON Schema descriptions to all filesystem tool argument structs for better AI model understanding and tool invocation. Changes: - readFileArgs: Added descriptions for file_path, offset, and limit fields - writeFileArgs: Added descriptions for file_path and content fields - editFileArgs: Added descriptions for file_path, old_string, new_string, and replace_all fields with default value annotation - globArgs: Added descriptions for pattern and path fields - grepArgs: Added comprehensive descriptions for all 14 fields including pattern, path, glob, output_mode (with enum values), context options (-A/-B/-C), line numbers, case sensitivity, file type, pagination (head_limit/offset), and multiline mode Also updated tool descriptions in prompt.go: - WriteFileToolDesc: Improved usage guidelines with clearer instructions - GlobToolDesc: Simplified description focusing on key capabilities This enhancement enables LLM-based agents to better understand tool parameters and generate more accurate tool calls.
… filesystem.Backend interface (#753)
* refactor(adk): refactor tool reduction middleware * refactor(adk): flatten reducton middleware configs, remove redundant parts
… Python glob and ls command (#767)
…722) * feat: add enhanced tool support with multimodal output capabilities and improve message formatting This commit introduces enhanced tool interfaces that support structured multimodal outputs, enabling tools to return rich content beyond simple text responses. Key Changes: 1. New Enhanced Tool Interfaces: - Added EnhancedInvokableTool and EnhancedStreamableTool interfaces for multimodal tool execution - Both interfaces use ToolCallInfo as input and return ToolResult for structured output 2. ToolResult Schema: - Introduced ToolResult type to represent multimodal tool outputs - Supports multiple content types: text, image, audio, video, and file - Added ToolOutputPart with Index field for streaming chunk merging - Implemented ToMessageInputParts() for seamless model integration 3. ToolsNode Enhancements: - Extended ToolsNode to support both legacy and enhanced tool types - Added automatic conversion between invokable and streamable endpoints - Implemented middleware support for enhanced tools - Enhanced interrupt and rerun mechanism to handle ToolResult 4. React Agent Integration: - Introduce enhancedToolResultSender and enhancedStreamToolResultSender types - Support sending *schema.ToolResult with multimodal content (images, audio, video, files) - Implement EnhancedInvokable and EnhancedStreamable middleware in tool result collector 5. Message.String() Enhancement: - Add formatting support for UserInputMultiContent, AssistantGenMultiContent, and MultiContent - Implement formatInputPart, formatOutputPart, and formatChatMessagePart helper functions - Create mediaPartFormatter interface with wrapper types for unified media formatting 6. User Input Multi-Content Concatenation: - Implement concatUserMultiContent function for merging MessageInputPart slices - Support text and base64 audio merging with proper MIME type handling - Integrate into ConcatMessages function 7. Callback System: - Added CallbackInput and CallbackOutput types for tool callbacks - Implemented conversion functions for different callback input/output types 8. Comprehensive Test Coverage: - Added tests for enhanced invokable and streamable tools - Added TestMessageString with 14 test cases covering various message types Impact: - Enables tools to return rich multimodal content (images, audio, video, files) - Provides foundation for more sophisticated tool implementations - Maintains full backward compatibility with existing tool ecosystem
) Added the ability to customize tool names for all filesystem middleware tools, allowing users to override default tool names with their own naming conventions. Main changes: - Added CustomXXXToolName fields to Config and MiddlewareConfig structs for all 7 tools (ls, read_file, write_file, edit_file, glob, grep, execute) - Implemented selectToolName() helper function to handle custom name selection with fallback to defaults - Updated all tool creation functions (newLsTool, newReadFileTool, newWriteFileTool, newEditFileTool, newGlobTool, newGrepTool, newExecuteTool, newStreamingExecuteTool) to accept name parameter - Modified getFilesystemTools() to pass custom tool names to each tool creation function - Updated NewMiddleware() to forward custom tool name configurations Implementation details: - Custom tool names are optional pointer fields, maintaining backward compatibility - When CustomXXXToolName is nil, default constants from ToolNameXXX are used - selectToolName() provides centralized logic for name resolution - All 7 filesystem tools now support name customization: ls, read_file, write_file, edit_file, glob, grep, and execute Benefits: - Enables alignment with organization-specific naming conventions - Allows shorter or more intuitive names for specific use cases - Prevents naming conflicts with other tools in the system - Maintains full backward compatibility with existing code
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
* refactor(adk): refactor filesystem middleware tool configuration with custom tools and disable flags - Add support for custom tool implementations via ToolConfig - Add Disabled flag to allow disabling individual tools - Change Name and Desc fields from *string to string for better usability - Merge tool configuration logic with legacy descriptor fields - Improve code organization and field ordering in ToolConfig~ * refactor: optimize code structure, add edge case tests and improve documentation - Extract validateConfigCore to eliminate duplicate validation logic - Refactor getFilesystemTools using toolSpec pattern to reduce repetition - Add comprehensive edge case tests for ToolConfig: * Empty Desc with nil legacyDesc * CustomTool with Disabled flag * Multiple conflicting configurations * Nil config fallback scenarios * Empty Name/Desc defaults - Clarify CustomTool comment to mention Backend-associated implementation * docs: add documentation comments for toolSpec and createToolFromSpec - Add comprehensive comment for toolSpec structure explaining its purpose - Add detailed comment for createToolFromSpec function describing its workflow - Improve code readability and maintainability through better documentation * refactor: change Desc field back to *string pointer type - Change ToolConfig.Desc from string to *string for better optionality - Update mergeToolConfigWithDesc to handle nil pointer checks - Update createToolFromSpec to convert *string to string when calling createFunc - Update all related tests to use pointer types for Desc field - Keep Name field as string type (non-pointer) * refactor: remove ExecuteToolConfig configuration - Remove ExecuteToolConfig from Config and MiddlewareConfig structs - Execute tool is now automatically created based on Shell/StreamingShell availability - Remove ExecuteToolConfig from validation logic - Remove ExecuteToolConfig from toolSpecs array - Add standalone execute tool creation logic after toolSpecs loop - Remove related test cases - Simplify execute tool configuration by using default name and description
feat(adk): optimize the verification logic
…ng shell tests (#842) Remove the default system prompt injection logic from NewMiddleware and New, keeping only custom prompt support. Delete unused ToolsSystemPrompt and ExecuteToolsSystemPrompt constants from prompt.go. Add comprehensive test coverage for streaming shell execution, validation, and edge cases.
…llocations (#850) - Change Backend.Read return type from string to FileContent struct for extensibility - Optimize InMemoryBackend.Read to use strings.IndexByte scanning instead of strings.Split/Join, eliminating unnecessary allocations for large files - Add fast path for the common case (no offset, content within limit) - Fix default limit from 200 to 2000 to match ReadRequest documentation - Fix offset to be strictly 1-based as documented in ReadRequest - Move line number formatting from backend to middleware layer (separation of concerns) - Add comprehensive Read edge case tests
hi-pender
reviewed
Mar 9, 2026
Change-Id: Ia71dc2f76902414750772b8be0a413c7f7d1d2db
hi-pender
approved these changes
Mar 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
merge alpha/08 to main