[Pelis Agent Factory Advisor] Pelis Agent Factory Advisor — Agentic Workflow Opportunities (2026-06-16)

[github-actions[bot]]

📊 Executive Summary

gh-aw-firewall has a mature agentic workflow estate (42 agentic + ~20 traditional workflows) with excellent smoke test, security, token analysis, and code quality coverage. The top gaps for a security/firewall tool are: no Docker base image CVE monitoring, dangerously low coverage on the two most critical TypeScript files (cli.ts 0%, docker-manager.ts 18%), and no health aggregation layer for the 7+ smoke test flavors running every 12h.

---

📋 Workflow Inventory

Workflow	Purpose	Trigger	Assessment
build-test	Build + test suite	label_command	✅ Active
ci-cd-gaps-assessment	CI/CD gap analysis	daily	✅ Meta-workflow
ci-doctor	Analyze CI failures	workflow_run	✅ Active
claude/copilot-token-* (×4)	Token analytics + optimization	daily/workflow_run	✅ Good
cli-flag-consistency-checker	CLI flag vs docs parity	weekly	✅ Good
config-consistency-auditor	Config file consistency	weekday daily	✅ Good
contribution-check	PR vs CONTRIBUTING.md	label_command	✅ Active
dependency-security-monitor	npm CVE monitoring	daily	⚠️ No Docker image scan
doc-maintainer	Documentation review	daily, 7-day gate	✅ Good
duplicate-code-detector	Code duplication scan	daily	✅ Active
export-audit	API surface / unused exports	weekly	✅ Good
firewall-issue-dispatcher	Sync gh-aw issues → here	every 12h	✅ Unique
issue-duplication-detector	Duplicate issue detection	issues event	✅ Good
issue-monster	Batch issue → Copilot agent	issues + hourly	✅ Active
plan	/plan slash command	slash_command	✅ On-demand
red-team-benchmark	Adversarial exfil testing	weekly	🔒 Flagship
refactoring-scanner	Refactoring opportunities	daily	✅ Active
schema-sync	Schema synchronization	weekday daily	✅ Active
secret-digger-* (×3)	Container secret probing	dispatch	🔒 Red-team ×3
security-guard	PR security posture review	label_command	🔒 Active
security-review	Daily threat modeling	daily	🔒 Flagship
smoke-* (×9 flavors)	Engine + feature validation	every 12h	✅ Thorough, no aggregator
test-coverage-improver	Write missing unit tests	schedule	⚠️ Critical files uncovered
test-coverage-reporter	Coverage trend report	daily + push	⚠️ Reporting only
test-hard-cap-ai-credits	AI credits hard cap	dispatch	✅ Active
update-release-notes	Release notes from diff	release	✅ Good

---

🚀 Recommendations

P0 — High Impact · Low Effort

1. 🐳 Docker Base Image CVE Monitor

What: Weekly scan of ubuntu/squid and ubuntu:22.04 base images for CVEs using docker scout or Trivy; create issues for HIGH/CRITICAL findings.

Why: dependency-security-monitor covers npm packages but ignores the Docker images that form the actual sandbox boundary. A compromised base image directly undermines the firewall's isolation guarantee — the most glaring security gap for a tool whose core value is network isolation.

How: Mirror dependency-security-monitor structure; agent runs docker scout cves ghcr.io/github/gh-aw-firewall/agent:latest and squid:latest, creates issues for HIGH/CRITICAL CVEs with base image pin suggestions.

Effort: Low — template available in agentics dependency-security-monitor.

---

2. 💰 Cost Tracker on PRs

What: Post per-run token cost summaries (from token-usage.jsonl) as PR comments when smoke/agentic CI completes.

Why: Token analytics infrastructure exists (two analyzers + two optimizers) but there is no per-PR cost visibility. The Agentics library has a Cost Tracker template exactly for this. Developers get immediate feedback on token costs.

How: workflow_run trigger after smoke test completion; parse token-usage.jsonl artifact; post concise cost comment on the triggering PR.

Effort: Low — Cost Tracker template available; infrastructure in place.

---

P1 — High Impact · Medium Effort

3. 🔍 Log Watcher for Smoke Tests

What: After each smoke test batch, scan logs for errors, retry loops, rate-limit messages, and token anomalies; post a health report.

Why: With 9 smoke flavors running every 12h, silent degradations (rate limits, model deprecations, flaky network) persist unnoticed. ci-doctor only triggers on outright failures, not degraded-but-passing runs. The community Log Watcher (AgentMeter/agentmeter-action) is a ready-made template.

Effort: Medium — correlating run IDs to PRs and parsing artifacts.

---

4. 🛡️ Firewall ACL Regression Detector

What: When squid-config.ts or containers/squid/** changes in a PR, automatically test that known-allowed domains still pass and known-blocked domains are still blocked.

Why: The core security contract of awf is the domain ACL. Today, ACL generation code changes have no integration-level validation. A refactor accidentally flipping ACL logic could silently allow exfiltration — the highest-severity failure mode for this tool.

How: PR trigger on squid-config file changes; agent starts AWF with test domain list, makes requests, validates pass/fail matrix, comments on PR.

Effort: Medium — requires an integration test harness using awf itself.

---

P2 — Medium Impact

5. 📈 Coverage Ratchet for Critical Files

What: Dedicated weekly workflow targeting cli.ts (0%) and docker-manager.ts (18%) specifically, with explicit per-file coverage targets (not just overall thresholds).

Why: The existing test-coverage-improver is generic and the thresholds sit just above the current watermark (38%). These two files orchestrate container lifecycle and network isolation setup — zero and near-zero coverage is a concrete security risk. The overall metric masks the critical gap.

Effort: Medium — requires writing tests for complex async/docker-exec code paths.

---

P3 — Nice to Have

• 😤 Grumpy Reviewer — opinionated architectural code review (template available)
• 💬 Discussion Task Miner — extract actionable issues from security-review discussions (template available)
• 🗺️ Weekly Repo Map — codebase structure + size trends (template available)

---

📈 Maturity Assessment

Dimension	Current	Target	Gap
Security Automation	4/5	5/5	Docker image CVE scan missing
Test Coverage	2/5	4/5	Critical files uncovered (cli.ts 0%)
Observability	3/5	4/5	No smoke test health aggregation
Cost Visibility	3/5	4/5	No per-PR cost feedback
Release Automation	4/5	5/5	Solid
PR Quality Gates	4/5	5/5	ACL regression test missing

Overall: 4/5 — One of the most complete agentic workflow setups observed. Gaps are specific and actionable, not foundational.

---

📝 Cache Note

Content hash 08e334c6d63f14b2829907c5645eb34832c83b2ac96cd79910daa928300cff73 — update pelis_docs_hash in cache-memory on next run. Track: Docker CVE monitor adoption, cli.ts coverage trend, ACL regression detector.

Generated by Pelis Agent Factory Advisor · ◷

• expires on Jun 23, 2026, 10:53 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-06-23T22:53:04.824Z.

Closed by Workflow