Skip to content

feat: add evidence validation and adaptive query limits for blue investigations#27

Merged
l50 merged 1 commit intomainfrom
jayson/cap-831-implement-adaptive-query-limits-for-blue-agent
Jan 11, 2026
Merged

feat: add evidence validation and adaptive query limits for blue investigations#27
l50 merged 1 commit intomainfrom
jayson/cap-831-implement-adaptive-query-limits-for-blue-agent

Conversation

@l50
Copy link
Copy Markdown
Contributor

@l50 l50 commented Jan 11, 2026

Key Changes:

  • Introduced evidence validation against recent query results to improve provenance
  • Implemented adaptive query limits with bonus queries for productive investigations
  • Added tools for parallel LogQL queries and efficient pattern combination
  • Enhanced Grafana integration with investigation lifecycle annotations

Added:

  • Evidence validation module that:
    • Stores recent query results and extracted IOCs for provenance checks
    • Validates evidence values and adjusts confidence based on validation
    • Provides a tool for auto-extracted IOCs to guide evidence recording
  • Support for parallel LogQL queries via execute_parallel_queries tool, enabling
    efficient batch querying for independent investigations
  • Tool combine_query_patterns to merge multiple regex patterns into a single
    LogQL query for better efficiency
  • Grafana annotation posting for investigation start, completion, timeout, and
    failure events, including summary details

Changed:

  • Blue agent query management:
    • Reworked query limit logic to use adaptive, stage-based budgets with bonus
      queries for finding evidence and reaching higher Pyramid of Pain levels
    • Only queries returning results count against the investigation's query budget
    • All query attempts (successful and failed) are tracked for observability
  • Evidence recording flow:
    • Automatically validates evidence against stored query results and marks as
      validated/unvalidated
    • Reduces confidence score for unvalidated evidence
    • Records provenance by linking evidence to source query ID
  • Investigation tools:
    • record_evidence now returns validation status and logs validation metrics
    • New tool get_suggested_evidence exposes auto-extracted IOCs from queries
  • Updated system instructions to strongly encourage use of parallel and combined
    queries for efficiency, with concrete examples

Removed:

  • No major removals; existing sequential query patterns are now discouraged in
    favor of new parallel and combined query tools

…otations

**Added:**

- Implemented evidence validation against recent query results, including provenance
  tracking, IOC extraction, and confidence adjustment - `src/ares/core/evidence_validation.py`
- Introduced automatic posting of investigation started, completed, timeout, and failed
  annotations to Grafana for investigation lifecycle observability
- Added `get_suggested_evidence` tool to suggest IOCs extracted from queries for
  improved evidence recording accuracy
- Provided `execute_parallel_queries` and `combine_query_patterns` tools for efficient
  and parallelized log querying in investigations
- Enhanced `Evidence` model with `source_query_id` and `validated` fields for traceability

**Changed:**

- Updated blue agent investigation orchestrator to post Grafana annotations at start,
  completion, timeout, and failure of investigations
- Refactored query tracking to count only successful (result-producing) queries against
  adaptive query limits and not penalize failed/empty queries
- Made query limits adaptive: increased limits with investigation progress, bonus
  queries for finding evidence or reaching higher Pyramid of Pain levels, and a hard cap
- Updated evidence recording to validate values, adjust confidence if unvalidated, and
  log provenance
- Improved system instructions to document parallel and combined query strategies and
  explain new evidence validation and IOC suggestion capabilities
- Updated default query limit variable names in `Taskfile.yaml` for blue and red agents

**Removed:**

- Removed old badge section in `README.md` in favor of up-to-date project badges
- Deprecated redundant agent thread import in factories for blue and red agents
@linear
Copy link
Copy Markdown

linear Bot commented Jan 11, 2026

CAP-831 Implement Adaptive Query Limits for Blue Agent

Description:
Update the Ares Blue Team SOC Investigation Agent to use adaptive query limits based on investigation progress and results. This will replace static query limits with a dynamic system that rewards productive investigations and aligns query allowance with investigation stages.


Objective:

Enable the Blue Agent to dynamically adjust its query limits, improving investigation efficiency by granting bonus queries for productive actions and aligning query allowances to investigation phases, while preventing excessive querying.


Scope of Work:

  • Refactor query limit logic in src/ares/core/factories/blue_factory.py
  • Implement bonus query allocation (+2) for evidence found and pyramid level 4+ reached, with a maximum cap of 15 queries
  • Modify query counting to exclude queries that return zero results
  • Introduce staged query limits per investigation phase as defined in QUERY_LIMITS_BY_STAGE
  • Update or add tests to cover adaptive query limit behaviors
  • Update documentation/comments to reflect new adaptive query logic

Acceptance Criteria:

  1. Query limit increases by +2 when evidence is found, up to a maximum of 15 queries per investigation
  2. Query limit increases by +2 when the agent reaches pyramid level 4 or higher, respecting the 15-query cap
  3. Only queries that return data are counted against the limit; queries with zero results do not decrement the remaining queries
  4. Query limits are enforced according to investigation phase: 5 for triage, +3 for causation, +3 for lateral, and 0 for synthesis
  5. Automated tests validate all adaptive query limit scenarios and caps
  6. Code and documentation are updated to reflect the new logic

Additional Notes:

  • See src/ares/core/factories/blue_factory.py lines 38-39 for current static limits
  • Consider edge cases where multiple bonus conditions are met simultaneously
  • Ensure logic is easily extensible for future adjustments

@dreadnode-renovate-bot dreadnode-renovate-bot Bot added area/taskfiles area/templates Changes made to warpgate template configurations labels Jan 11, 2026
@l50 l50 merged commit 39a7c1b into main Jan 11, 2026
8 checks passed
@l50 l50 deleted the jayson/cap-831-implement-adaptive-query-limits-for-blue-agent branch January 11, 2026 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/templates Changes made to warpgate template configurations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant