Feature: Enhanced Tool Management, New Agent Tools, and LLM Retry Options

[szmania]

This pull request introduces several significant enhancements to Aider-CE, focusing on tool management, agent capabilities, and robust LLM interactions.

Key Changes:

1. Enhanced Tool Management (/tools* commands)

A new suite of commands has been introduced to manage custom tools directly from the chat interface:

• /tools: List available standard and custom tools.
• /tools-create: Create a new custom tool from a description.
• /tools-load: Load a tool from a file or by name.
• /tools-unload: Unload a tool.
• /tools-mv: Move a tool between local and global scopes.
• /tools-edit: Add a tool's source file to the chat for editing.
• /tools-rm: Delete a custom tool.
• /tools-reload: Reload all custom tools.

Other improvements to tool management include:

• Fuzzy Search: Implemented fuzzy matching for tool names in management commands for easier lookup.
• Improved UX: Added completions for tool commands and flexible tool editing.
• Robustness: Improved tool lookup logic, normalized tool names, and added support for local vs. global tool identification.
• Dependency Isolation: Custom tools now have their dependencies installed into isolated virtual environments.

Tool Scopes

Tools in Aider-CE are categorized into three scopes:

• Standard Tools: These are the built-in tools that come with Aider-CE.
• Custom (Local) Tools: These are project-specific tools that you can create. They are stored in the .aider/tools/ directory within your project's repository and are only available for that project.
• Custom (Global) Tools: These are tools that you want to be available across all your projects. They are stored in the ~/.aider/tools/ directory in your home folder.

2. New Agent Tools

Two powerful new tools have been added to the AgentCoder's registry:

• create_tool: Allows the agent to dynamically create new Python tools to extend its own capabilities.
• context_manager: Enables the agent to proactively manage its context window by partially clearing chat history (with keep_percent support) and removing unneeded files.

3. LLM Retry Options

Added new command-line arguments to args.py to improve reliability during intermittent API issues:

• --retry-timeout: Set the timeout for retrying LLM calls.
• --retry-backoff-factor: Configure the exponential backoff factor for retries.
• --retry-on-unavailable: Enable automatic retries on service unavailable errors.

4. Other Improvements

• MCP Enhancements: Improved MCP error handling with detailed logs and server-specific log files.
• Git Integration: Overhauled gitignore rules to prevent committing ignored files and added support for --add-gitignore-files.
• TUI Fixes: Various fixes for the Textual TUI, including improved exit handling and input lag reduction.
• Session Management: Added fzf support for session commands and improved session name completions.

This PR compiles changes from over 100 commits since tag v0.90.6, providing a more robust and extensible foundation for Aider-CE.

[dwash96]

So I am not going to merge this as is (it's simply way to much code, features, and enmeshment) and there are a few issues that I have with it and a few things here that I like for subsequent, more-focused PRs.

Tool Calls/Tool Creator

Issues/Thoughts

I don't think tool creation is an omnipresent need/primary action of the system and as such should not have the space devoted to it in the system prompt that it has. I also think the file loader logic is very much overcomplicated. I am already working on a generic file based plugin import system with this as the core:

https://medium.com/@david.bonn.2010/dynamic-loading-of-python-code-2617c04e5f3f

using importlib.util as my north star on allowing user-defined files into the system. Once this is added, creating a tool will be more about creating something that properly extends BaseTool and after @gopar's changes in:

#361

are finished, reloading the MCP server with new tool definitions to support tool commands will be much more straightforward.

Recommendations

The tool command API makes sense and should be refactored to work with the generic plugin loader I'll add in the next version and @gopar's changes when he finishes them. We should definitely keep these commands:

/tools: List available standard and user-defined tools.
/tools-load: Load a tool from a file or by name.
/tools-remove: Unload a tool.
/tools-edit: Add a user-defined tool's source file to the chat for editing.
/tools-reload: Reloads/re-registers all user-defined custom tools.

Context Management

Issues/Thoughts

So, first, we actually already have a ContextManager tool now as of 0.92.0:
https://github.com/dwash96/aider-ce/blob/main/cecli/tools/context_manager.py

For the addition/removal of in context files.

I also don't really think agent controlled, percentage-based context compaction will work since, a decent amount of the time, the agent will remove context it actually needs and then need to re-find it all again.

Recommendations

Review the existing context compaction logic (e.g. what happens when --enable-context-compaction is on. It currently summarizes the chat context when the user/assistant message flow is 80% the total size of the model's context window. View compact_context_if_needed() in base_coder.py for more details on that.

The opportunity for extension is adding a file content size check variant to this logic. Compaction (really truncation in this instance) is probably best done by injecting a user message in the stream or modifying the final reminders message (view format_chat_chunks() for details on that) that explicitly asks the LLM to remove unnecessary files from the chat before proceeding, if the file content is large and letting it use the current context manager's remove tool on its own to accomplish that. There should probably be a "cooldown" between when we can inject these messages so it doesn't catch the model in a loop of trying to do it, say every 10 turns. We ultimately shouldn't need to devote system prompt space for this.

Retry Settings

Recommendations

I generally like these changes but they are likely to need work to merge cleanly with @chrisnestrud's changes to models.py with the ModelProviderManager. They may be orthogonal but I wouldn't be surprised if they aren't. My only real issue is that it adds to the command line sprawl. I would prefer new, related settings be implemented much closer to how agent-config and tui-config are, as nested json/yaml options. Refer to:

https://github.com/dwash96/aider-ce/blob/main/cecli/website/docs/config/tui.md#complete-configuration-example

and lookup convert_yaml_to_json_string() in main.py to see how these types of options are handled. Decoding nested configs into dicts-with-defaults as necessary is my preferred strategy of managing setting complexity so we'd end up with something like this:

retries:
  timeout: <number>
  backoff-factor: <number>
  retry-on-unavailable: <boolean>

And this nested structure can be extended more or less indefinitely as long as we set defaults for all options when not present at parse time. Really, the compaction logic can almost certainly also be placed into unified structure like this. Eventually, this will make settings documentation much more straightforward as all of these blocks will be independently configurable

[szmania]

So I am not going to merge this as is (it's simply way to much code, features, and enmeshment) and there are a few issues that I have with it and a few things here that I like for subsequent, more-focused PRs.

Tool Calls/Tool Creator

Issues/Thoughts

I don't think tool creation is an omnipresent need/primary action of the system and as such should not have the space devoted to it in the system prompt that it has. I also think the file loader logic is very much overcomplicated. I am already working on a generic file based plugin import system with this as the core:

https://medium.com/@david.bonn.2010/dynamic-loading-of-python-code-2617c04e5f3f

using importlib.util as my north star on allowing user-defined files into the system. Once this is added, creating a tool will be more about creating something that properly extends BaseTool and after @gopar's changes in:

#361

are finished, reloading the MCP server with new tool definitions to support tool commands will be much more straightforward.

Recommendations

The tool command API makes sense and should be refactored to work with the generic plugin loader I'll add in the next version and @gopar's changes when he finishes them. We should definitely keep these commands:

/tools: List available standard and user-defined tools. /tools-load: Load a tool from a file or by name. /tools-remove: Unload a tool. /tools-edit: Add a user-defined tool's source file to the chat for editing. /tools-reload: Reloads/re-registers all user-defined custom tools.

Context Management

Issues/Thoughts

So, first, we actually already have a ContextManager tool now as of 0.92.0: https://github.com/dwash96/aider-ce/blob/main/cecli/tools/context_manager.py

For the addition/removal of in context files.

I also don't really think agent controlled, percentage-based context compaction will work since, a decent amount of the time, the agent will remove context it actually needs and then need to re-find it all again.

Recommendations

Review the existing context compaction logic (e.g. what happens when --enable-context-compaction is on. It currently summarizes the chat context when the user/assistant message flow is 80% the total size of the model's context window. View compact_context_if_needed() in base_coder.py for more details on that.

The opportunity for extension is adding a file content size check variant to this logic. Compaction (really truncation in this instance) is probably best done by injecting a user message in the stream or modifying the final reminders message (view format_chat_chunks() for details on that) that explicitly asks the LLM to remove unnecessary files from the chat before proceeding, if the file content is large and letting it use the current context manager's remove tool on its own to accomplish that. There should probably be a "cooldown" between when we can inject these messages so it doesn't catch the model in a loop of trying to do it, say every 10 turns. We ultimately shouldn't need to devote system prompt space for this.

Retry Settings

Recommendations

I generally like these changes but they are likely to need work to merge cleanly with @chrisnestrud's changes to models.py with the ModelProviderManager. They may be orthogonal but I wouldn't be surprised if they aren't. My only real issue is that it adds to the command line sprawl. I would prefer new, related settings be implemented much closer to how agent-config and tui-config are, as nested json/yaml options. Refer to:

https://github.com/dwash96/aider-ce/blob/main/cecli/website/docs/config/tui.md#complete-configuration-example

and lookup convert_yaml_to_json_string() in main.py to see how these types of options are handled. Decoding nested configs into dicts-with-defaults as necessary is my preferred strategy of managing setting complexity so we'd end up with something like this:

retries:
  timeout: <number>
  backoff-factor: <number>
  retry-on-unavailable: <boolean>

And this nested structure can be extended more or less indefinitely as long as we set defaults for all options when not present at parse time. Really, the compaction logic can almost certainly also be placed into unified structure like this. Eventually, this will make settings documentation much more straightforward as all of these blocks will be independently configurable

How do you envision users to create new tools in this plugin loader? The tool command API makes sense and should be refactored to work with the generic plugin loader I'll add in the next version and @gopar's changes when he finishes them. We should definitely keep these commands:

[szmania]

So I am not going to merge this as is (it's simply way to much code, features, and enmeshment) and there are a few issues that I have with it and a few things here that I like for subsequent, more-focused PRs.

Tool Calls/Tool Creator

Issues/Thoughts

I don't think tool creation is an omnipresent need/primary action of the system and as such should not have the space devoted to it in the system prompt that it has. I also think the file loader logic is very much overcomplicated. I am already working on a generic file based plugin import system with this as the core:
https://medium.com/@david.bonn.2010/dynamic-loading-of-python-code-2617c04e5f3f
using importlib.util as my north star on allowing user-defined files into the system. Once this is added, creating a tool will be more about creating something that properly extends BaseTool and after @gopar's changes in:
#361
are finished, reloading the MCP server with new tool definitions to support tool commands will be much more straightforward.

Recommendations

The tool command API makes sense and should be refactored to work with the generic plugin loader I'll add in the next version and @gopar's changes when he finishes them. We should definitely keep these commands:
/tools: List available standard and user-defined tools. /tools-load: Load a tool from a file or by name. /tools-remove: Unload a tool. /tools-edit: Add a user-defined tool's source file to the chat for editing. /tools-reload: Reloads/re-registers all user-defined custom tools.

Context Management

Issues/Thoughts

So, first, we actually already have a ContextManager tool now as of 0.92.0: https://github.com/dwash96/aider-ce/blob/main/cecli/tools/context_manager.py
For the addition/removal of in context files.
I also don't really think agent controlled, percentage-based context compaction will work since, a decent amount of the time, the agent will remove context it actually needs and then need to re-find it all again.

Recommendations

Review the existing context compaction logic (e.g. what happens when --enable-context-compaction is on. It currently summarizes the chat context when the user/assistant message flow is 80% the total size of the model's context window. View compact_context_if_needed() in base_coder.py for more details on that.
The opportunity for extension is adding a file content size check variant to this logic. Compaction (really truncation in this instance) is probably best done by injecting a user message in the stream or modifying the final reminders message (view format_chat_chunks() for details on that) that explicitly asks the LLM to remove unnecessary files from the chat before proceeding, if the file content is large and letting it use the current context manager's remove tool on its own to accomplish that. There should probably be a "cooldown" between when we can inject these messages so it doesn't catch the model in a loop of trying to do it, say every 10 turns. We ultimately shouldn't need to devote system prompt space for this.

Retry Settings

Recommendations

I generally like these changes but they are likely to need work to merge cleanly with @chrisnestrud's changes to models.py with the ModelProviderManager. They may be orthogonal but I wouldn't be surprised if they aren't. My only real issue is that it adds to the command line sprawl. I would prefer new, related settings be implemented much closer to how agent-config and tui-config are, as nested json/yaml options. Refer to:
https://github.com/dwash96/aider-ce/blob/main/cecli/website/docs/config/tui.md#complete-configuration-example
and lookup convert_yaml_to_json_string() in main.py to see how these types of options are handled. Decoding nested configs into dicts-with-defaults as necessary is my preferred strategy of managing setting complexity so we'd end up with something like this:

retries:
  timeout: <number>
  backoff-factor: <number>
  retry-on-unavailable: <boolean>

And this nested structure can be extended more or less indefinitely as long as we set defaults for all options when not present at parse time. Really, the compaction logic can almost certainly also be placed into unified structure like this. Eventually, this will make settings documentation much more straightforward as all of these blocks will be independently configurable

How do you envision users to create new tools in this plugin loader? The tool command API makes sense and should be refactored to work with the generic plugin loader I'll add in the next version and @gopar's changes when he finishes them. We should definitely keep these commands:

Also curious how you will handle external dependencies for custom tools with this new plugin loader. Say a new tool requires duckduckgo search or something.

[dwash96]

To answer your questions:
https://github.com/dwash96/aider-ce/blob/v0.93.0/cecli/website/docs/config/agent-mode.md#creating-custom-tools

Based on:
https://github.com/dwash96/aider-ce/blob/v0.93.0/cecli/helpers/plugin_manager.py
https://github.com/dwash96/aider-ce/blob/v0.93.0/cecli/tools/utils/registry.py#L55

A generalized file based importer than can be made to read from files and directories and can directly insert tools into a centralized registry of tools. I see creating custom tools being the full responsibility of the user. LLMs can absolutely be used at their discretion to create something that complies with the base class and can be placed really where ever the user wants it to be. If a user needs specific external dependencies, I am going to assume they are python advanced enough to configure PYTHONPATH so that they have the relevant modules available in their system path or use importlib directly the way that I am here.

Per my previous comment, adding the commands that allow for the management/addition/removal of these things at run time is a good next step

[szmania]

Ok i'll start breaking out the things from this PR into smaller PRs that are not blocked by pre-requisites.