Skip to content

v2.5.0 Validation

Choose a tag to compare

View all tags
@gmelli gmelli released this 04 Oct 19:47
· 199 commits to main since this release
v2.5.0
7fafe21

AGET v2.5.0 "Validation" Release Notes

Release Date: 2025-10-06
Version: v2.5.0
Codename: "Validation"
Status: ✅ Production Ready

Overview

AGET v2.5.0 "Validation" introduces contract-based validation framework for agent identity and version consistency. This release establishes automated validation patterns that prevent common migration issues (version drift, identity conflation) while maintaining backward compatibility with v2.4.0 agents.

Migration Scope: 3 agents + template (proof of concept for 17-agent fleet)

What's New

1. Contract Test Framework ⭐

Wake Protocol Contract Tests (4 tests):

  • test_wake_protocol_reports_agent_name - Validates agent identity reporting
  • test_wake_protocol_reports_version - Validates version format (X.Y.Z)
  • test_wake_protocol_reports_capabilities - Validates capability structure (flexible: dict or list)
  • test_wake_protocol_reports_domain - Validates domain specification

Identity Contract Tests (3 tests):

  • test_identity_consistency_version_json_vs_manifest - Prevents version drift (L28)
  • test_identity_no_conflation_with_directory_name - Validates identity = location
  • test_identity_persistence_across_invocations - Separates identity from operational state

Purpose: Automated validation of agent compliance with AGET framework standards

2. Flexible Capability Validation

Problem Solved: Agents use different capability representations (dict, list, none)

Solution: Contract tests now accept both formats:

# Before (rigid):
assert isinstance(capabilities, dict)

# After (flexible):
assert isinstance(capabilities, (dict, list))

Impact: Tests resilient to implementation choices, prevents brittleness

3. Version Drift Prevention (L28 Enhancement)

Automated Guards:

  • Contract test validates version.json = agent_manifest.yaml
  • Fails immediately on version mismatch
  • Applied to all migrated agents

Result: Zero version drift detected in v2.5 migration (3 agents validated)

4. Template Generalization

Template vs Agent Distinction:

  • Templates: agent_name optional (populated on instantiation)
  • Agents: agent_name required (identity = location)

Contract tests handle both cases:

if "agent_name" in data:
    assert agent_name == directory_name

5. Session Metadata Validation (v1.0 Standard)

Validated Components:

  • Agent identity reporting (wake protocol)
  • Version consistency (identity contract)
  • Session state separation (operational vs identity fields)

Breaking Changes

⚠️ Contract Tests Required

Impact: All agents must pass 7 contract tests to be v2.5 compliant

Migration Path:

  1. Copy test_wake_contract.py and test_identity_contract.py to tests/
  2. Run: python3 -m pytest tests/test_wake_contract.py tests/test_identity_contract.py -v
  3. Fix any failures (typically version inconsistencies)
  4. Update version.json and agent_manifest.yaml to v2.5.0

Compatibility: v2.4.0 agents continue to work, but won't have contract test validation

⚠️ Version Consistency Enforced

Impact: version.json and agent_manifest.yaml must show same version

Before v2.5: No automated check (drift possible)
After v2.5: Contract test fails on mismatch

Fix: Update both files atomically during version promotion (L28 protocol)

Agents Migrated

Production Agents (3)

  1. my-github-AGET v2.5.0 - Coordinator agent (exemplar tier)
  2. my-OpenAI-DeepResearch-aget v2.5.0 - Research agent (foundation tier)
  3. my-spotify-analyst-aget v2.5.0 - Analytics agent (exemplar tier, 89% coverage)

Framework

  1. aget-cli-agent-template v2.5.0 - Template with generalized contract tests

Fleet Status: 3 of 17 active agents migrated (14 pending future rollout)

Validation Results

Contract Test Coverage

  • Total tests: 21 (7 tests × 3 agents)
  • Passing: 21/21 (100%)
  • Version drift: 0 instances detected
  • Time to execute: <0.2s per agent

Application Test Coverage

  • my-github-AGET: 37 tests passing
  • my-OpenAI-DeepResearch-aget: 0 tests (research agent, no application tests)
  • my-spotify-analyst-aget: 66 tests passing (89% code coverage)
  • Total: 103 application + 21 contract = 124 tests passing

Manual Validation

  • Wake protocol: All agents correctly report v2.5.0 identity ✅
  • Session tracking: Functional across all agents ✅
  • Version consistency: version.json = agent_manifest.yaml ✅

Known Issues

1. Workspace Observer Test Failure (my-github-AGET)

Status: Pre-existing from v2.4, deferred to backlog
Impact: 2 tests fail in test_coordinator_features.py
Workaround: Tests pass with workspace observer disabled
Fix: Scheduled for v2.5.1 patch

2. Coverage Warnings (my-spotify-analyst-aget)

Status: Expected behavior
Impact: Contract tests don't cover application code (validate metadata files)
Workaround: Run application tests separately for coverage
Fix: Not needed (by design)

Performance

Migration Time

  • Gate 1 (Canary): 125 min - Process establishment
  • Gate 3.1 (Fleet Agent #1): 25 min - Proven pattern
  • Gate 3.2 (Fleet Agent #2): 27 min - Proven pattern
  • Average (Fleet): 26 min per agent (5x faster than canary)

Efficiency Gains

  • Process investment: 125 min establishing pattern
  • Savings per agent: 99 min (125 - 26)
  • Net savings (3 agents): 73 min
  • Projected savings (14 remaining): 23.1 hours for full fleet

Resource Usage

  • Contract tests: <0.2s execution time
  • Disk space: ~10KB per agent (2 test files + README)
  • Memory: Negligible (metadata validation only)

Upgrade Guide

For Agent Maintainers

Step 1: Add Contract Tests (5 min)

# Copy from template
cp ~/github/aget-cli-agent-template/tests/test_wake_contract.py YOUR_AGENT/tests/
cp ~/github/aget-cli-agent-template/tests/test_identity_contract.py YOUR_AGENT/tests/

# Run tests
cd YOUR_AGENT
python3 -m pytest tests/test_wake_contract.py tests/test_identity_contract.py -v

Step 2: Fix Any Failures (5-10 min)

  • Version mismatch: Update version.json and agent_manifest.yaml to same version
  • Agent name mismatch: Ensure agent_name matches directory name
  • Capabilities type: Both dict and list are valid (no fix needed)

Step 3: Version Promotion (5 min)

# Update version.json
jq '.aget_version = "2.5.0" | .migrated_to_v25 = "2025-10-06"' .aget/version.json > tmp.json
mv tmp.json .aget/version.json

# Update agent_manifest.yaml
# Change: version: "2.4.0" → version: "2.5.0"
# Change: aget_compliance: "v2.4.0" → aget_compliance: "v2.5.0"

# Add changelog entry (see template for format)

Step 4: Validate (2 min)

# Verify version consistency
python3 -m pytest tests/test_identity_contract.py::test_identity_consistency_version_json_vs_manifest -v

# Verify all contract tests pass
python3 -m pytest tests/test_wake_contract.py tests/test_identity_contract.py -v

Total Time: 15-20 min per agent (after v2.5 template available)

For Template Users

New Agent Creation:

# Clone template
git clone https://github.com/gmelli/aget-cli-agent-template.git my-new-agent
cd my-new-agent

# Contract tests included by default (tests/ directory)
# Instantiate with agent name
jq '.agent_name = "my-new-agent"' .aget/version.json > tmp.json
mv tmp.json .aget/version.json

# Verify
python3 -m pytest tests/test_wake_contract.py tests/test_identity_contract.py -v

Deprecations

None. v2.5.0 is fully backward compatible with v2.4.0 agents.

Security

Version Drift Prevention

  • Automated validation prevents version.json ≠ agent_manifest.yaml
  • Reduces risk of inconsistent agent behavior
  • Guards against stale version reporting

Identity Validation

  • Enforces agent_name = directory_name (identity = location)
  • Prevents identity conflation (#76)
  • Separates identity from operational state

Credits

Contributors

  • my-supervisor-AGET: Migration orchestration, gate execution, documentation
  • User (gmelli): Advisory at boundaries, process critique, quality assurance

Learnings Applied

  • L28: Version Promotion Protocol (prevents drift)
  • L80: Validate Before Building (check existing systems first)
  • L82: Success as Stop Condition (sufficient success ≠ maximum effort)
  • L87: Advisory at Boundaries (external review prevents tunnel vision)

New Learnings Captured

  • L88: Contract tests validate behavior, not implementation details
  • L89: Process investment ROI - establish once, execute many
  • L90: Spot-check validation (67% sampling) for fleet quality
  • L91: Advisory at boundaries prevents tunnel vision (validated)

Roadmap

v2.5.1 (Patch) - Estimated: 1-2 weeks

  • Fix workspace observer test failure (my-github-AGET)
  • Update template tests to be flexible by default
  • Create TECHNICAL_DEBT.md for debt tracking
  • Create gate completion checklist

v2.5.x (Minor Patches) - Estimated: 1-2 months

  • Enhance commit message template
  • Automated version promotion (bump_agent_version.py)
  • Full fleet rollout documentation

v2.6.0 (Major) - Estimated: Q1 2026

  • Full fleet rollout (14 remaining agents)
  • Parallel migration tooling
  • Coordination capabilities
  • Handoff protocol enhancements

Support

Documentation

Getting Help

Conclusion

AGET v2.5.0 "Validation" establishes contract-based validation as a foundation for fleet-scale agent management. The migration process proven across 3 diverse agents (coordinator, research, analytics) demonstrates framework maturity and readiness for full fleet rollout.

Next Steps:

  1. Push v2.5.0 to remote repositories
  2. Create GitHub release
  3. Plan v2.5.1 patch for remaining issues
  4. Defer full fleet rollout to v2.6

Status: ✅ PRODUCTION READY - Ship it! 🚀

Release notes v2.5.0
Released: 2025-10-06
Migration time: 4.5 hours (257 minutes)

Assets 2
Loading