feat(agentworkforce): integration-verifier persona + skills for end-to-end Relayfile integration verification

[khaliqgant]

Summary

Adds a dedicated AgentWorkforce persona — integration-verifier — that drives the same end-to-end verification methodology we proved against rw_fc7b534b during the May 2026 discovery-chain rollout (cloud#745 → #756 → #761 + #771, plus the gaps cloud#766, #775, #778, #780 we surfaced and filed honestly).

Invoked via:

agentworkforce agent integration-verifier

or scripted with a kickoff task description (e.g. "verify gmail + gcal full-fledged on rw_fc7b534b").

Why this exists

A merged PR is not a verified integration. Across the May 2026 effort the following gaps were only visible by actually exercising the end-to-end flow on a real workspace — never by reading PR descriptions:

• Discovery schemas materializing at the advertised paths but containing zero properties for some flat resources (silently masked by writeManagedFile byte-stable dedup) — cloud#778.
• Outbound writeback rejecting canonical edit paths the adapter itself emits — cloud#780.
• A control-plane endpoint silently 15–90s slow under realistic load, so the CLI's HTTP client times out before the cloud responds — cloud#766.
• Sync workers silently no-op'ing because of a plan/quota gate, while reporting backfilled:true.
• A Worker bundle transitively pulling pg from a sibling import, breaking deploys — cloud#762.

Each of these passed CI. Each was caught only by running the full flow. The persona locks in that the full flow gets run on every new integration.

What's in the PR (3 files, purely additive)

.agentworkforce/workforce/
├── personas/
│   └── integration-verifier.json
└── skills/
    ├── integration-verification-playbook/SKILL.md
    └── relayfile-architecture-cheatsheet/SKILL.md

integration-verifier.json

• harness: claude, model claude-opus-4-7, high reasoning, 1800s timeout.
• permissions.mode = bypassPermissions — auto-approve every tool call (the model-side discipline lives in the system prompt, not the harness allow-list).
• Three skills, loaded at the start of every task:
  1. @agent-relay/setting-up-relayfile (prpm — installed on session boot) — the canonical operator-facing setup walkthrough; authoritative for CLI mechanics.
  2. local/integration-verification-playbook — the seven-phase methodology + failure-pattern table + issue-filing template (live ledger).
  3. local/relayfile-architecture-cheatsheet — compressed architecture model so the agent doesn't re-derive it.
• claudeMdContent encodes operating principles: don't guess payloads, schema-validated > LAYOUT-documented > skip, outward-facing safety (test markers, throwaway targets), honest issue-filing, don't bypass real CI signals, no autonomous merges without explicit operator authorization.

integration-verification-playbook/SKILL.md

The seven-phase flow per provider × workspace:

1. Auth — token validity check, the cloud#766 /join-slow workaround (direct curl with --max-time 120 + persist to ~/.relayfile/credentials.json).
2. Connect — relayfile integration connect …, verify oauth.connected:true, connectionMatched:true, real currentConnectionId directly against the cloud status endpoint.
3. Sync — wait for initialSync.state:"complete", confirm record/index/alias trees exist (not just LAYOUT.md/_index.json).
4. Tree-shape — read /<provider>/LAYOUT.md, spot-check canonical records + alias subtrees match what's documented.
5. Discovery — POST /sync/refresh, read each writable resource's .schema.json + .create.example.json + .adapter.md; watch for the cloud#778 pattern (backfilled:true but properties:{} with stale lastEditedAt).
6. Outbound writeback — only with a schema-validated payload, only against a designated throwaway, with a clearly-marked [relayfile writeback test <ISO-utc>] marker; verify the op completed + the round-trip; watch for the cloud#780 pattern ("No <provider> writeback rule matched <path>").
7. Webhook-in — operator makes a provider-side change; verify the change propagates into the mount within ~60s.

Plus a recurring-failure-pattern table with the precedent issue for each class, the issue-filing template, and test-marker conventions. The file is explicitly the live ledger — new failure modes get appended.

relayfile-architecture-cheatsheet/SKILL.md

Compressed working-memory model so the verifier doesn't re-derive the architecture each session:

• Two services (api.relayfile.dev vs agentrelay.com/cloud) and two tokens (cloud-credentials vs workspace token).
• Mount daemon state files + the fix(mountsync): defense-in-depth against mount-root clobber data loss #164/feat(mountsync): structural mount-root invariants (long-term hardening) #165/fix(mountsync): converge large workspaces — bootstrap timeout, safe fast-path, resumable pull #166 invariants (clobber protection, two-phase tombstones, bootstrap convergence).
• WorkspaceDO / R2 / DO SQLite layout + the #730 OOM fix.
• The sync pipeline: nango-webhook-router → record-writer.ts writeBatchToRelayfile → materializeProviderContract and the single ADAPTERS registry.
• The discovery chain end-to-end and the honest coverage scope of #761 (flat resources only; placeholder-path sub-resources stay permissive-empty by design).
• The file-native writeback contract (canonical edit vs non-canonical create vs receipt pointer).
• The digest pipeline + the #771 internal-path filter.
• Worker import safety (B1) + Phase-0 acceptance-against-prod semantics (the only legitimate skip case).
• Quick-paths cheatsheet for every common curl call (token mint, tree read, file read, sync refresh, op status, integration status).
• Cross-links to every PR and rule mentioned.

Validation

• JSON validates against ../workforce/packages/persona-kit/schemas/persona.schema.json (jsonschema.validate ✓).
• permissions.mode: "bypassPermissions" is in $defs.PermissionMode.enum.
• All three skill source fields resolve: prpm skill installs at session boot; both local SKILL.md files exist at the declared paths.
• agentworkforce agent integration-verifier installs cleanly (no cp: SKILL.md: No such file or directory failures — the directory-per-skill layout matches what the CLI's plugin installer expects).

How it gets used

For a new integration verification (the immediate use case is Gmail + Google Calendar):

agentworkforce agent integration-verifier

Then provide the kickoff TASK_DESCRIPTION: which provider(s), which workspace id, smoke vs full-fledged scope. The agent loads all three skills, runs the seven phases, captures empirical evidence, files issues per the template for any real gap, and returns a verification matrix.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Rate limit exceeded

@khaliqgant has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 9 minutes and 31 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fcc3a04-730c-4d8e-a48c-5d94e52ee660

📥 Commits

Reviewing files that changed from the base of the PR and between 025e072 and 216a3d9.

📒 Files selected for processing (3)

• .agentworkforce/workforce/personas/integration-verifier.json
• .agentworkforce/workforce/skills/integration-verification-playbook/SKILL.md
• .agentworkforce/workforce/skills/relayfile-architecture-cheatsheet/SKILL.md

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch personas

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 2 additional findings.