[cli-tools-test] Daily CLI Tools Exploratory Testing Report — 2026-05-02

[github-actions[bot]]

Summary

All three CLI tools (audit, logs, compile) are functioning correctly. Testing was performed against 207 workflows across multiple phases.

Metric	Value
Workflows compiled	207
Compilation success rate	100%
Audit runs tested	4
Log filter scenarios tested	6
Critical bugs found	0
Minor issues found	3

Phase 2: logs Tool ✅

• Basic log download (last 24h, 3 runs): completed in 19.5s ✅
• Engine filter (claude): returned 3 matching runs ✅
• Workflow name filter (contribution-check): returned 3 matching runs ✅
• Non-existent workflow: graceful error with helpful suggestions ✅
• Old date range (-365d): returns empty result cleanly ✅
• Minor: Future date (2099-01-01) returns empty with no explanatory message

Phase 3: audit Tool ✅

• Audit successful run (§25245017074): complete report with metrics, findings, behavior fingerprint ✅
• Audit claude-engine run (§25244792701): includes cost analysis ($1.85), token usage ✅
• Audit cached run: returns in ~0.15s ✅
• Invalid run ID (99999999999): clear error with suggestions ✅

Phase 4: compile Tool ✅

• Compile all 207 workflows: 100% success rate ✅
• Compile individual workflows (archie, ai-moderator): ✅
• Compile with --actionlint: detects docker unavailability (expected in CI) ✅
• Invalid workflow name: clear error with suggestions ✅
• Incremental compile: lock files unchanged when source unchanged ✅

Performance

Operation	Time
status	~3.2s
logs (3 runs, -1d)	~19.5s
audit (cached)	~0.15s
compile (single)	<1s
compile (207 workflows)	~30s

Minor Issues

1. Parameter naming: CLI docs show --workflow-name and --run-id but actual params are --workflows and --run_id_or_url. Agents need --help first.
2. Future date empty with no message: logs --start-date 2099-01-01 returns empty without explanation.
3. Actionlint docker warning: Running --actionlint in CI produces docker_unavailable noise.

Log Storage

43 run directories / 331MB / 8,960 files in /tmp/gh-aw/aw-mcp/logs/

References: §25245213380

Generated by Daily CLI Tools Exploratory Tester · ● 1.5M · ◷

• expires on May 5, 2026, 6:03 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-05T06:03:03.622Z.

Closed by Workflow