Skip to content

Optimization - #346

Merged
Zochory merged 6 commits into
codebase-optimizationfrom
experimental
Nov 12, 2025
Merged

Optimization#346
Zochory merged 6 commits into
codebase-optimizationfrom
experimental

Conversation

@Zochory

@Zochory Zochory commented Nov 12, 2025

Copy link
Copy Markdown
Member

This pull request primarily improves documentation and project configuration, with a focus on developer experience, architectural clarity, and lifecycle extensibility for agents. It introduces a comprehensive onboarding and workflow guide for WARP users, documents new extensibility points (like agent lifecycle hooks and progress emitters), and clarifies architectural and validation patterns. It also removes deprecated configuration and CI files.

Documentation improvements:

  • Added a new WARP.md file providing a complete onboarding and workflow guide for contributors using WARP, including stack overview, common commands, architecture, extension points, and conventions.
  • Expanded docs/AGENTS.md with a section on Agent Lifecycle Hooks, documenting the AgentLifecycle protocol with warmup() and teardown() methods, including a full example implementation.
  • Updated docs/Project_Architecture_Blueprint.md to:
    • Document the new ProgressEmitter interface for custom progress reporting, including usage examples. [1] [2] [3] [4]
    • Clarify the role of Pydantic models in workflow YAML validation, and update architectural component descriptions for accuracy.
    • Reference the OpenTelemetry integration guide for observability setup.

Configuration and workflow:

  • Added a $schema directive to config/workflows.yaml for YAML language server support, improving IDE validation and developer experience.

Developer workflow and CI:

  • Removed the deprecated .github/workflows/codex-autofix.yml workflow, which previously attempted to auto-fix CI failures using Codex.

Project hygiene:

  • Removed the unused .cursor/mcp.json configuration file.
  • Minor cleanup in README.md (removed a blank line and trailing whitespace). [1] [2]

Other documentation fixes:

  • Corrected the health check route reference in docs/api/developer-guide.md to point to api/routers/health.py instead of the outdated path.## Description

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code refactoring
  • Performance improvement
  • Test updates
  • Configuration change

Related Issues

Fixes #

Changes Made

  • (describe your changes here)

Testing

  • All existing tests pass (uv run pytest)
  • Added new tests for new functionality
  • Configuration validation passes (uv run python tests/test_config.py)
  • Linting passes (uv run ruff check .)
  • Formatting is correct (uv run black .)
  • Type checking passes (uv run mypy .)

Test Commands Run

# Commands run to validate changes
uv run pytest
uv run python tests/test_config.py
uv run ruff check .
uv run black .

Documentation

  • Updated relevant documentation
  • Added docstrings to new functions/classes
  • Updated README if needed
  • Updated CHANGELOG if needed

Screenshots (if applicable)

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Additional Notes

This pull request introduces several documentation improvements and new developer guides to clarify architectural concepts, agent lifecycle management, and frontend-backend integration. The changes expand on how to extend and operate the system, including new features for agent lifecycle hooks, dynamic agent composition, and progress reporting.

Key documentation and architecture updates:

Developer & Operator Documentation

  • Added a comprehensive WARP.md guide for developers using WARP, detailing the stack, common commands, architecture, and extension points for both backend and frontend.
  • Updated the root README.md to remove the architecture banner image and minor formatting adjustments. [1] [2]
  • Added a schema directive to config/workflows.yaml for better YAML validation tooling support.

Architecture & Extensibility

  • Expanded the architecture blueprint (docs/Project_Architecture_Blueprint.md) to document the new ProgressEmitter interface for pluggable progress reporting, including example usage and integration points in the orchestration layer. [1] [2] [3] [4]
  • Clarified the use of Pydantic models for workflow config validation and improved documentation around error handling, validation, and OpenTelemetry integration. [1] [2]

Agent Lifecycle & Extension

  • Added a new section to docs/AGENTS.md describing agent lifecycle hooks (warmup() and teardown()), including a code example for managing stateful resources within agents.

API & Frontend Integration

  • Updated the frontend-backend integration guide to document dynamic agent composition via the agents parameter in chat requests, and described new SSE event types for incremental reasoning updates (reasoning.delta, reasoning.completed). [1] [2]
  • Clarified streaming feature flags and their effect on event emission and frontend behavior.
  • Fixed a reference in the API developer guide to the correct health check route file.

@Zochory Zochory added this to the v0.5.6 milestone Nov 12, 2025
@github-project-automation github-project-automation Bot moved this to Backlog in AgenticFleet Nov 12, 2025
@github-actions

Copy link
Copy Markdown
Contributor

🤖 Hi @Zochory, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

@github-actions github-actions Bot added dependencies Dependency updates area: config Configuration system type: documentation Improvements or additions to documentation type: test Test-related changes python Python-related changes labels Nov 12, 2025
@coderabbitai

coderabbitai Bot commented Nov 12, 2025

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch experimental

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Zochory
Zochory requested a review from Copilot November 12, 2025 17:13
@Zochory
Zochory marked this pull request as ready for review November 12, 2025 17:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request introduces significant improvements to documentation, project architecture, and extensibility for the AgenticFleet system. It focuses on enhancing developer experience through comprehensive guides, introducing agent lifecycle hooks, adding caching and checkpointing capabilities, and modernizing the codebase with typed configurations.

Key Changes:

  • Added WARP.md developer onboarding guide with comprehensive workflow documentation
  • Introduced agent lifecycle hooks (warmup() and teardown()) via the AgentLifecycle protocol
  • Implemented Redis-based caching with LangCache integration for workflow responses
  • Added workflow checkpointing support via WorkflowCheckpointService
  • Migrated from dataclass to Pydantic models for workflow configuration validation
  • Introduced OpenTelemetry integration for observability
  • Enhanced frontend with conversation sidebar, message history loading, and SSE stream cancellation
  • Added health check endpoints with framework version validation

Reviewed Changes

Copilot reviewed 78 out of 81 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/validate_test_improvements.py Updated test execution to use asyncio.run(), added type ignores, improved mock safety checks, and converted timestamps to datetime objects
tests/test_workflow_factory.py Added WorkflowConfig import, helper function for YAML writing, and serialization round-trip test
tests/test_response_aggregator.py Refactored to test reasoning delta streaming with simplified helper functions and event parsing
tests/test_framework_health.py New comprehensive test suite for framework health checks and package version validation
tests/test_event_bridge.py Added test for AgentRunUpdateEvent reasoning delta conversion
tests/test_agent_lifecycle.py New test suite for agent lifecycle hooks (warmup/teardown) with async support
src/frontend/ Added conversation management, sidebar UI, message history loading, and stream cancellation
src/agentic_fleet/workflow/magentic_workflow.py Extensive refactoring with caching, checkpointing, telemetry, and structured request handling
src/agentic_fleet/workflow/events.py Added reasoning delta event conversion and backward compatibility stubs
src/agentic_fleet/utils/telemetry.py New optional OpenTelemetry integration module
src/agentic_fleet/utils/redis_cache.py New Redis cache manager with LangCache integration
src/agentic_fleet/models/workflow_config.py New Pydantic models replacing dataclasses for typed configuration
src/agentic_fleet/agents/coordinator.py Added lifecycle hook execution with async/sync support
src/agentic_fleet/agents/base.py New AgentLifecycle protocol definition
src/agentic_fleet/api/routers/health.py New health check endpoints with Redis rate limiting
src/agentic_fleet/framework/health.py New framework health validation with package version checks
Documentation files Comprehensive updates to architecture, configuration, and API integration guides

Comment thread src/frontend/src/stores/chatStore.ts Outdated
},

createNewConversation: async () => {
const state = get();

Copilot AI Nov 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused variable state.

Copilot uses AI. Check for mistakes.
Comment thread src/agentic_fleet/api/app.py Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>
@github-project-automation github-project-automation Bot moved this from Backlog to In review in AgenticFleet Nov 12, 2025
@Zochory

Zochory commented Nov 12, 2025

Copy link
Copy Markdown
Member Author

@copilot code review

Copilot AI commented Nov 12, 2025

Copy link
Copy Markdown
Contributor

@Zochory I've opened a new pull request, #347, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan

* fix: remove unused variable and useless try/catch in chatStore

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>
@Zochory
Zochory merged commit 82a2e84 into codebase-optimization Nov 12, 2025
2 checks passed
@Zochory
Zochory deleted the experimental branch November 12, 2025 17:36
@github-project-automation github-project-automation Bot moved this from In review to Done in AgenticFleet Nov 12, 2025
Zochory added a commit that referenced this pull request Nov 15, 2025
* up (#343)

* Delete .github/workflows/codex-autofix.yml

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Delete src/agentic_fleet/ui/assets directory

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Refactor tests: Replace hardcoded values with constants and improve patterns (#341)

* Initial plan

* Apply test improvements: refactor hardcoded values and improve test patterns

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

* Fix import order and formatting in test files

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

* Fix code review issues: REPO variable order and async fixture

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

* Fix remaining code review issues: variable name and comment clarity

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

* Update tools/scripts/setup-pypi-environment.sh

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

---------

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>
Co-authored-by: Zachary BENSALEM <zachary@qredence.ai>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Delete tools/scripts/setup-pypi-environment.sh

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Delete playwright.config.ts

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Delete playwright.service.config.ts

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Delete test_sse_performance.py

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

---------

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* clean-1

* clean-2

* refactor: optimize code structure and improve readability in test files

* Refactor code structure for improved readability and maintainability

* Optimization (#346)

* refactor: optimize code structure and improve readability in test files

* Refactor code structure for improved readability and maintainability

* refactor: remove obsolete workflow API and related components

* Refactor code structure for improved readability and maintainability

* Update src/agentic_fleet/api/app.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* fix: remove unused variable and useless try/catch in chatStore (#347)

* Initial plan

* fix: remove unused variable and useless try/catch in chatStore

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

---------

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>

* Refactor code structure and improve workflow configuration (#348)

* refactor: optimize code structure and improve readability in test files

* Refactor code structure for improved readability and maintainability

* refactor: remove obsolete workflow API and related components

* Refactor code structure for improved readability and maintainability

* Update src/agentic_fleet/api/app.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* feat: add fast path model configuration and improve logging in workflow routes

---------

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactor code structure for improved readability and maintainability

* Remove unnecessary future annotations from prompt templates

* Refactor agent modules to remove unnecessary future annotations and streamline imports

* Remove unnecessary future annotations from model files

* Remove obsolete test files to streamline the test suite and improve maintainability

* Remove implementation status and testing specification documents as part of project restructuring

* Remove unused REDIS_URL variable from environment configuration

* Refactor SSE handling in chat module and add SSE parser utility

* Refactor: Remove unnecessary future annotations and clean up imports

- Removed `from __future__ import annotations` from multiple files where it was not needed.
- Updated health check package version requirements to use minimum versions.
- Replaced `pkg_resources` with `importlib.metadata` for package version checks.
- Cleaned up Redis integration code, marking it as removed and providing clear runtime errors for legacy imports.
- Simplified various repository and service classes by using string annotations for type hints.
- Removed backward compatibility stubs in workflow events.
- Adjusted logging and error handling for improved clarity and maintainability.

* Refactor: Remove obsolete model files to streamline codebase

* Refactor: Remove persistence layer files to streamline codebase

* Refactor: Remove health check module to streamline codebase

* Refactor: Remove approvals and conversations modules to streamline codebase

* Remove deprecated API components and exception handling

- Deleted the centralized exception handling module to streamline error management.
- Removed API models and response schemas related to the Responses API.
- Eliminated workflow management routes and services to simplify the workflow handling process.
- Cleaned up health check endpoints and their associated logic.
- Removed unused routers and service files to enhance code maintainability.

* Refactor: Remove workflow-related modules to streamline codebase

* Remove deprecated utility modules and workflow configuration files

- Deleted `factory.py`, `logging.py`, `logging_sanitize.py`, `message_classifier.py`, `performance.py`, `redis_cache.py`, `telemetry.py` from utils as they are no longer needed.
- Removed `workflows.schema.json` and `workflows.yaml` files, which contained the packaged default multi-workflow configuration.

* remove old plan

* Remove outdated documentation files: persistence-guide.md, prompt-kit-integration.md, and responsive-design-implementation.md

* Delete src/agentic_fleet directory

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* clean

* feat: Major refactor - reorganize into src/agentic_fleet package structure

- Moved all core code into src/agentic_fleet/ package
- Created modular CLI with command-based structure in cli/commands/
- Added DSPy workflow integration with supervisor, signatures, and compilation
- Implemented quality assessment, progress tracking, and handoff management
- Added FastAPI REST API in api/app.py
- Created comprehensive evaluation framework with metrics
- Added tool registry system for dynamic tool discovery
- Implemented async compilation and caching utilities
- Added telemetry, tracing, and logging infrastructure
- Created self-improvement and history analysis scripts
- Updated all agents to use base agent class
- Reorganized workflows into execution, routing, quality, and fleet modules
- Added type definitions and configuration schema
- Created browser and Tavily search tool integrations
- Added comprehensive examples and evaluation tasks

* Refactor code structure and remove redundant code blocks for improved readability and maintainability

* Update Jupyter notebooks to reflect environment changes and code formatting improvements

- Set execution_count to null for initial code cells in `agent_as_workflow.ipynb` and `azure_responses_client.ipynb`
- Adjusted import statements to ensure proper line endings in `agent_as_workflow.ipynb` and `azure_responses_client.ipynb`
- Updated kernel display name and version to ".venv (3.13.9)" in `agent_as_workflow.ipynb`, `azure_responses_client.ipynb`, `magentic_fleet.ipynb`, and `openai_responses_example.ipynb`
- Changed header formatting from "##" to "#" in `magentic_fleet.ipynb` for consistency

* fix: Correct prompt template descriptions and add future annotations for type hints

* refactor: Remove unused future annotations from prompt templates

* Add comprehensive tests for routing helpers and supervisor workflow

- Introduced tests for routing helper functions including normalization and edge case detection.
- Added enhanced tests for the supervisor workflow, covering fallback mechanisms, task handling, and agent execution.
- Implemented integration tests for tool registry population and capabilities inference during workflow initialization.
- Created comprehensive tests for supervisor workflow streaming, validation, and task management.
- Ensured proper handling of agent failures and task validation scenarios.

* Fix security and type safety issues from PR #349 automated review (#351)

* Initial plan

* Fix security issue and code quality issues from PR #349 review

- Remove API key from log message in tavily_mcp_tool.py (security fix)
- Add missing TYPE_CHECKING imports in workflows/__init__.py
- Add missing TYPE_CHECKING import in workflows/fleet/__init__.py
- Fix incorrect parameter names in builder.py (id -> executor_id)

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

* Potential fix for pull request finding 'Wrong name for an argument in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Remove Gemini workflows and enhance CLI functionality

- Deleted the following GitHub Actions workflows:
  - .github/workflows/gemini-review.yml
  - .github/workflows/gemini-scheduled-triage.yml
  - .github/workflows/gemini-triage.yml

- Added a new option to the run command in the CLI to optimize for latency with the `--fast` flag.
- Adjusted the workflow initialization in the WorkflowRunner class to ensure consistent configuration usage.
- Improved confidence display in routing history by handling non-numeric values gracefully.
- Updated the title display for completed messages from agents to ensure clarity in agent identification.

* Implement feature X to enhance user experience and optimize performance

* Refactor create_supervisor_workflow to accept optional config parameter for enhanced flexibility

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Enhance CLI and workflows with new features and optimizations, including fast mode for latency, backward compatibility for console imports, and improved agent normalization in DSPySupervisor.

* Potential fix for pull request finding 'Explicit export is not defined'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Fix inconsistent project naming in documentation (#352)

* Initial plan

* Fix inconsistent project naming in documentation

Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Zochory <60674042+Zochory@users.noreply.github.com>
Co-authored-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

* Potential fix for pull request finding 'Wrong number of arguments in a class instantiation'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>

---------

Signed-off-by: Zachary BENSALEM <zachary@qredence.ai>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: config Configuration system dependencies Dependency updates python Python-related changes type: documentation Improvements or additions to documentation type: test Test-related changes

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants