Skip to content

refactor: modularize agent toolsets and improve documentation#6

Merged
l50 merged 9 commits intomainfrom
jayson/cap-776-test-ares-agent-integration-with-grafana-via-mcp
Dec 18, 2025
Merged

refactor: modularize agent toolsets and improve documentation#6
l50 merged 9 commits intomainfrom
jayson/cap-776-test-ares-agent-integration-with-grafana-via-mcp

Conversation

@l50
Copy link
Copy Markdown
Contributor

@l50 l50 commented Dec 16, 2025

Key Changes:

  • Refactored all agent toolsets into modular files under src/tools/
  • Improved type annotations, docstrings, and comments for maintainability
  • Updated agent and core to use new toolset structure
  • Enhanced README and added new docs for Taskfile and Grafana MCP usage

Added:

  • Modular toolsets:
    • src/tools/observability.py: Loki and Prometheus tools with detailed docstrings
    • src/tools/grafana.py: Grafana alerting tools and MCP connection logic
    • src/tools/investigation.py: Investigation state management and question engines
    • src/tools/mitre.py: MITRE ATT&CK lookup tools
    • src/tools/actions.py: Investigation completion and escalation actions
    • src/tools/__init__.py: Toolset exports for clean imports
  • Core agent factory:
    • src/core/create.py and src/core/__init__.py: Centralized agent creation logic
  • Developer and usage documentation:
    • docs/taskfile_usage.md: Taskfile command usage and troubleshooting
    • docs/grafana_mcp_usage.md: Guide for Grafana MCP integration and query patterns
    • tests/README.md: Test coverage for MCP and Windows attack queries

Changed:

  • Refactored src/agent.py to use create_investigation_agent from src/core
  • Updated agent toolset injection to support MCP tools and new modular imports
  • Rewrote all toolset docstrings for clarity and standardization
  • Enhanced type hints throughout models, engines, mitre, and report modules
  • Improved README with quick start, variable tables, and detailed CLI/taskfile usage
  • Expanded and updated .gitignore for project-specific directories
  • Upgraded pyproject.toml:
    • Renamed project to ares
    • Reorganized dependency groups and extras
    • Modernized build and lint/test config for better type checking and formatting
  • Improved script and test annotations for mypy/pytest compatibility

Removed:

  • Deleted monolithic src/tools.py in favor of modular toolset files
  • Removed redundant tool and hook definitions in favor of new structure
  • Eliminated unused imports and legacy argument fields from agent and main

l50 added 3 commits December 15, 2025 18:27
…omation

**Added:**

- Example environment variable file for agent configuration, including API keys,
  data source URLs, and investigation settings - `.env.example`
- Documentation for Grafana MCP integration and advanced log query guides for
  investigations - `docs/grafana_mcp_usage.md`
- Detailed Taskfile usage guide for development and production workflows -
  `docs/taskfile_usage.md`
- Test script and README for Windows attack indicator queries using Grafana MCP -
  `tests/test_grafana_mcp_windows.py` and `tests/README.md`
- Python project lockfile for reproducible builds - `uv.lock`

**Changed:**

- Major rewrite of `README.md` for concise overview, usage, and configuration,
  focusing on quick start, key features, and platform integration
- Taskfile expanded with ares-specific tasks, including agent run, investigation,
  configuration checks, report management, and MITRE testing
- `pyproject.toml` now describes the actual "ares" project, with correct
  dependencies, dev tools, documentation extras, build settings, and CLI entry
  point
- Refactored agent, tools, and main entrypoint for:
    - Support of enhanced Grafana MCP toolset for log/label/stat discovery
    - Defaulting all timestamps to timezone-aware UTC
    - Improved evidence and question deduplication logic
    - Propagation of Dreadnode platform organization/workspace config
    - Taskfile variable-based runtime configuration and secrets loading
    - Consistent duration formatting and report timestamps
    - Lint/type check and coverage configuration to match new source layout
- Small improvements to error handling and typing in scripts and hooks

**Removed:**

- Old project template metadata from `pyproject.toml`
- Outdated or redundant project setup and poetry config
- Unused error messages and print statements from scripts for cleaner automation output
**Changed:**

- Added explicit return type annotations to functions and methods to improve type
  clarity across `.py` files
- Added type annotations for class definitions where relevant, e.g., `# type:
  ignore[misc]`, to suppress type checker complaints about generic base classes
- Added `# type: ignore[untyped-decorator]` to decorator usages (e.g.,
  `@dn.tool_method`, `@pytest.fixture`, `@app.command`) to suppress mypy errors
  on untyped decorators
- Improved typing for pytest fixtures and test functions, specifying return types
  and argument types
- Removed unused imports (e.g., `import sys` in `generate_readme.py`)
- Updated `main()` functions to have explicit `-> None` return type and, where
  appropriate, call `main()` directly instead of `sys.exit(main())` for
  consistency with new type hints
…ated package

**Added:**

- Introduced `src/core/` package with `create_investigation_agent` factory for agent
  instantiation and configuration
- Added modular `src/tools/` package:
    - `actions.py` for investigation completion and escalation actions
    - `grafana.py` for Grafana and MCP tool integration
    - `investigation.py` for state management and question engine tools
    - `mitre.py` for MITRE ATT&CK lookup tools
    - `observability.py` for Loki and Prometheus query tools
- Added `tests/test_mcp_integration.py` for validating MCP connection and tool
  listing

**Changed:**

- Refactored `src/agent.py` to delegate agent creation to `core.create_investigation_agent`
  and utilize new toolset imports
- Updated all imports in `src/agent.py`, `src/main.py`, and related modules to use
  new `src/tools` and `src/core` structure
- Updated `README.md` and documentation to emphasize MCP-based querying and
  service account token usage
- Enhanced docstrings and type annotations across models, engines, mitre, and
  report modules for clarity and API discoverability
- Improved agent initialization workflow to prefer MCP tools, fall back gracefully,
  and provide clearer logging around tool availability

**Removed:**

- Removed legacy monolithic `src/tools.py`, splitting its logic into specialized
  modules under `src/tools/`
- Eliminated direct references to Loki and Prometheus URLs from configuration and
  documentation, reflecting the move to MCP-based data source abstraction
@linear
Copy link
Copy Markdown

linear Bot commented Dec 16, 2025

CAP-776 Test Ares Agent Integration with Grafana via MCP

Description:
Validate the end-to-end integration between the Ares blue team agent and Grafana using the Grafana MCP server. This includes confirming connectivity, data retrieval, and workflow functionality to ensure the agent supports observability-driven security investigations.


Objective:

Ensure the Ares agent can reliably connect to Grafana through MCP, query data sources (Loki, Prometheus), and utilize observability data within investigation workflows. Document all findings, issues, and recommendations for future improvements.


Scope of Work:

  • Install and configure Grafana MCP server following docs/topics/grafana-mcp-setup.md
  • Set up a Grafana service account with required permissions
  • Configure Ares agent with MCP credentials and endpoints
  • Verify MCP server connectivity to Grafana instance
  • Test Ares agent connectivity to MCP and successful authentication
  • Validate querying of Grafana data sources (Loki, Prometheus) via MCP
  • Test retrieval of dashboards, panels, and observability metrics
  • Run sample investigation workflows using Grafana-sourced data
  • Test MITRE ATT&CK correlation and timeline generation features
  • Document test scenarios, results, issues, and configuration notes
  • Update documentation and file bug reports or enhancement requests as needed

Dependencies:

  • Running Grafana instance (cloud or self-hosted)
  • Grafana MCP server installed and configured
  • Sample data sources in Grafana (Loki, Prometheus)
  • Ares agent environment with all dependencies installed
  • Access to docs/topics/grafana-mcp-setup.md and src/tools.py

Acceptance Criteria:

  1. Ares agent successfully authenticates and connects to Grafana via MCP.
  2. Agent can query and retrieve data from Loki and Prometheus through MCP.
  3. Investigation workflows display and utilize Grafana observability data.
  4. Authentication, authorization, and error handling are verified and documented.
  5. All test scenarios, configuration steps, and issues are documented in detail.
  6. Any discovered bugs or limitations are filed as separate tickets.
  7. Documentation is updated with new findings or usage examples as required.

Additional Notes:

  • Test both Grafana Cloud and self-hosted environments if possible.
  • Cover various query types (LogQL, PromQL) to ensure broad compatibility.
  • Record performance notes and any rate limiting encountered.
  • Capture relevant screenshots or logs for inclusion in documentation.
  • Reference repository: https://github.com/dreadnode/ares

@dreadnode-renovate-bot dreadnode-renovate-bot Bot added area/docs Changes made to project documentation area/python area/pre-commit Changes made to pre-commit hooks labels Dec 16, 2025
…thon 3.12 support

**Added:**

- Added ruff linter configuration for `examples/**/*.py` to allow printing and
  datetime without timezone in `pyproject.toml`

**Changed:**

- Changed GitHub Actions CI to test Python 3.10, 3.11, and 3.12 instead of 3.9
- Updated `Taskfile.yaml` to rename the Windows Grafana test task to
  `ares:example:grafana-windows` and run the script from the `examples` dir
- Improved documentation in `README.md`, `docs/topics/grafana-mcp-setup.md`,
  and updated example commands and explanations for clarity and readability
- Updated Markdown tables in `README.md` for consistent formatting
- Moved and rewrote `tests/README.md` as `examples/README.md` to describe
  usage of examples instead of tests

**Removed:**

- Removed `tests/test_grafana_mcp_windows.py` and replaced with
  `examples/grafana_mcp_windows_example.py`
- Removed references to running Windows Grafana tests from the `tests` directory
@dreadnode-renovate-bot dreadnode-renovate-bot Bot added area/examples area/github Changes made to GitHub Actions workflows labels Dec 16, 2025
l50 added 5 commits December 16, 2025 15:48
**Removed:**

- Deleted GitHub Actions workflow that automated PR description updates using
  rigging and OpenAI API integration (rigging_pr_description.yaml)
- Removed automation for generating and posting AI-generated PR summaries on
  pull request creation
…tion script

**Added:**

- Added `jinja2` and conditional `tomli` to dev dependencies in `pyproject.toml` for
  template rendering and toml parsing on Python <3.11
- Added `jinja2` and conditional `tomli` to dev dependencies in `uv.lock` for
  consistent environment management

**Changed:**

- Updated `generate_readme.py` to import `tomllib` on Python >=3.11 and fall back to
  `tomli` for earlier versions, ensuring compatibility
- Modified dry-run mode in `generate_readme.py` to print the generated content for
  better feedback
- Adjusted `uv.lock` to pin `exceptiongroup`'s `typing-extensions` dependency only on
  Python <3.11 for accurate compatibility
…eprecated pytest option

**Changed:**

- Loosened version constraint for `pytest` by removing the upper bound in both
  `pyproject.toml` and `uv.lock` to allow future updates without manual changes
- Updated `pytest-asyncio` minimum version to `0.24.0` and removed upper bound
  in both `pyproject.toml` and `uv.lock` for broader compatibility
- Removed deprecated `asyncio_default_fixture_loop_scope` option from
  pytest configuration in `pyproject.toml` to align with newer pytest-asyncio
  versions and prevent warnings/errors
**Changed:**

- Include pytest-asyncio in the list of Python test dependencies installed in the
  CI workflow to support testing of async code
**Changed:**

- Updated the test matrix to include Python 3.13 for CI runs in
  `.github/workflows/tests.yaml` to ensure compatibility with the latest Python
  version
@l50 l50 merged commit 04c7072 into main Dec 18, 2025
8 checks passed
@l50 l50 deleted the jayson/cap-776-test-ares-agent-integration-with-grafana-via-mcp branch December 18, 2025 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Changes made to project documentation area/github Changes made to GitHub Actions workflows area/pre-commit Changes made to pre-commit hooks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant