Agent Persona Exploration - 2026-04-27

[github-actions[bot]]

Overview

This research explores how the agentic-workflows custom agent responds to workflow creation requests from 5 distinct software worker personas. Analysis is based on deep review of the agent's instruction files (.github/aw/create-agentic-workflow.md, .github/aw/github-agentic-workflows.md) and real workflow examples in .github/workflows/*.md.

Note: The agentic-workflows agent type is not directly invocable as a task sub-agent in this environment. This analysis is based on static analysis of the agent's configuration, instruction files, and corpus of existing workflows.

---

Persona Overview

• Agent: agentic-workflows custom agent (Copilot Chat)
• Scenarios Tested: 4 representative scenarios across 5 personas
• Average Quality Score: 4.1/5.0

---

Key Findings

• 🎯 Trigger selection is consistently strong — the agent correctly maps PR scenarios to pull_request, scheduled reports to schedule: weekly/daily, and command-based tasks to slash_command
• 🔐 Security posture is well-enforced — instructions explicitly mandate read-only agent jobs with all writes routed through safe-outputs; this is reinforced with ❌ WRONG examples showing what not to do
• 🌐 GitHub MCP is the correct path for API access — documentation clearly warns against network: allowed: api.github.com and always steers toward tools: github: toolsets:
• ⚠️ Artifact/log access is an edge case — for scenarios requiring workflow run logs or coverage artifacts, the agent's tooling guidance is less prescriptive
• 📅 Fuzzy scheduling is the default recommendation — schedule: daily on weekdays or schedule: weekly over fixed crons

---

Top Patterns

1. Most common triggers: pull_request: [opened, synchronize], issues: [opened], schedule: daily/weekly, slash_command
2. Most recommended tools: github: toolsets: [default], cache-memory, mount-as-clis: true
3. Security practices: Read-only permissions: on agent job + safe-outputs: for all writes; lock-for-agent: true for concurrency

---

View Scenario Analysis (4 scenarios tested)

Scenario 1 — Backend Engineer: DB Schema Migration Review on PRs

Dimension	Score	Notes
Trigger appropriateness	5/5	pull_request: [opened, synchronize] — exact match
Tool selection	4/5	github: toolsets: [default] for PR diff; LLM handles SQL analysis natively
Security practices	5/5	Read-only permissions + add-comment safe output
Prompt clarity	4/5	Would generate clear instructions to check for DROP TABLE, DROP COLUMN, missing defaults
Completeness	4/5	May miss paths filter to scope to **/migrations/*.sql files

Average: 4.4/5 — ✅ Strong response expected

---

Scenario 2 — DevOps Engineer: Failed Deployment Incident Creation

Dimension	Score	Notes
Trigger appropriateness	4/5	workflow_run: types: [completed] correct; agent may not know exact workflow names
Tool selection	3/5	Needs github: toolsets: [actions] for log download — not always auto-inferred
Security practices	4/5	create-issue + add-comment safe outputs; permissions OK
Prompt clarity	3/5	Root cause categorization (OOM vs timeout vs config) is vague without examples
Completeness	3/5	Missing if: github.event.workflow_run.conclusion == 'failure' guard

Average: 3.4/5 — ⚠️ Adequate but needs improvement; workflow_run trigger is underspecified

---

Scenario 3 — QA Tester: Coverage Analysis on PRs

Dimension	Score	Notes
Trigger appropriateness	5/5	pull_request: [opened, synchronize] — exact match
Tool selection	3/5	Coverage data lives in artifacts; github: toolsets: [actions] needed but may be missed
Security practices	5/5	add-comment: max: 1 safe output; read-only permissions
Prompt clarity	4/5	Coverage delta table format is specific enough
Completeness	3/5	No guidance on how agent fetches coverage artifacts from CI run

Average: 4.0/5 — ⚠️ Good trigger/security but artifact access gap

---

Scenario 4 — Product Manager: Weekly Feature Digest

Dimension	Score	Notes
Trigger appropriateness	5/5	schedule: weekly with fuzzy scheduling — perfect fit
Tool selection	5/5	github: toolsets: [issues, pull_requests] — correct and minimal
Security practices	5/5	create-discussion safe output; read-only permissions
Prompt clarity	5/5	Grouping by label + markdown formatting is clearly actionable
Completeness	5/5	workflow_dispatch auto-added with fuzzy schedule; nothing missing

Average: 5.0/5 — ✅ Excellent response expected

---

View Areas for Improvement

Issue 1: workflow_run Trigger Guidance Is Thin

The create-agentic-workflow.md documentation has detailed coverage of issues, pull_request, schedule, and slash_command triggers but minimal guidance on workflow_run. For DevOps incident-automation scenarios, the agent may:

• Generate a trigger without the required if: conclusion == 'failure' guard
• Not know to specify workflows: [...] names
• Miss that github.event.workflow_run.head_sha is the correct ref for log fetching

Recommendation: Add a workflow_run trigger example to .github/aw/create-agentic-workflow.md.

Issue 2: Artifact/Coverage Data Access Is an Unsupported Pattern

QA coverage scenarios assume the agent can access CI-generated coverage artifacts. The GitHub MCP actions toolset exposes artifact metadata, but downloading artifact content requires additional steps. The agent documentation does not describe a pattern for this.

Recommendation: Document a steps: + bash pattern for downloading and parsing artifacts in .github/aw/github-agentic-workflows.md.

Issue 3: paths Filter on PR Triggers Not Always Suggested

For scenarios that should only fire when specific files change (e.g., **/migrations/*.sql, coverage/**), the agent may generate broad pull_request triggers without paths: filters, causing unnecessary runs.

Recommendation: Add a paths: filter best practice note to the trigger selection guidance section of .github/aw/create-agentic-workflow.md.

---

Recommendations

1. Add workflow_run trigger examples to .github/aw/create-agentic-workflow.md — include the if: conclusion == 'failure' guard, workflow name specification, and log-access patterns via github: toolsets: [actions]

2. Document artifact access pattern in .github/aw/github-agentic-workflows.md — a minimal steps: block showing how to download and expose CI artifacts (coverage reports, test results) before the agent runs

3. Promote paths: filters as a recommended practice for PR-triggered workflows — this reduces noisy runs and keeps workflows targeted; the create-agentic-workflow.md instruction file should prompt the agent to ask whether the workflow should be scoped to specific file patterns

---

Appendix: Scenario Coverage Matrix

Persona	Task	Trigger	Expected Quality
Backend Engineer	DB schema migration review	pull_request	✅ Strong (4.4/5)
Frontend Developer	Visual regression testing	pull_request + playwright	(not tested)
DevOps Engineer	Failed deployment incidents	workflow_run	⚠️ Adequate (3.4/5)
QA Tester	Coverage delta analysis	pull_request	⚠️ Adequate (4.0/5)
Product Manager	Weekly feature digest	schedule: weekly	✅ Excellent (5.0/5)

References:

• §24975769240

Generated by Agent Persona Explorer · ● 741.3K · ◷