Skip to content

feat: query /reflect before and after running the agent in harnesses#30028

Merged
pelikhan merged 2 commits intomainfrom
copilot/update-harness-query-reflect
May 3, 2026
Merged

feat: query /reflect before and after running the agent in harnesses#30028
pelikhan merged 2 commits intomainfrom
copilot/update-harness-query-reflect

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 3, 2026

The harnesses previously called fetchAWFReflect only after the agent run. If a harness exits unexpectedly before reaching the post-run fetch, the proxy state is never captured.

Changes

  • claude_harness.cjs / copilot_harness.cjs: Add a best-effort fetchAWFReflect call before the agent execution loop, immediately after startup validation. This writes the initial api-proxy /reflect snapshot to disk and logs it before any agent output, making proxy configuration available for diagnostics regardless of how the run ends.

  • pi_provider.cjs: Add an agent_end handler that calls fetchAWFReflect after the Pi agent session finishes. The extension already fetched /reflect on agent_start (pre-run); this adds the matching post-run fetch for parity with the other harnesses.

The post-run call in the copilot and claude harnesses is unchanged — it remains as the authoritative snapshot for the step summary.

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented May 3, 2026

@copilot also update the pi agent api-proxy provider extension

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 3, 2026

@copilot also update the pi agent api-proxy provider extension

Done in 91c864b. Added an agent_end handler to pi_provider.cjs that calls fetchAWFReflect after the agent session finishes, matching the pre/post-run pattern in the copilot and claude harnesses. The existing agent_start call (pre-run) is unchanged.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/github/gh-aw/contents/.github%2Fworkflows%2Faudit-workflows.md
    • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node --experimental-import-meta-resolve --require /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/suppress-warnings.cjs --conditions node --conditions development /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/dist/workers/forks.js --quiet k/gh-aw/node_mod--verify git comm�� -m (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git e/git conf�� user.email test@test.com git git-receive-packgit (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title feat: query /reflect before running the agent in harnesses feat: query /reflect before and after running the agent in harnesses May 3, 2026
Copilot AI requested a review from pelikhan May 3, 2026 23:42
@pelikhan pelikhan marked this pull request as ready for review May 3, 2026 23:43
Copilot AI review requested due to automatic review settings May 3, 2026 23:43
@pelikhan pelikhan merged commit 2a81990 into main May 3, 2026
9 of 11 checks passed
@pelikhan pelikhan deleted the copilot/update-harness-query-reflect branch May 3, 2026 23:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR is intended to improve observability around AWF proxy state by capturing /reflect snapshots before agent execution begins and, for Pi, after the session ends.

Changes:

  • Added a best-effort pre-run fetchAWFReflect call to the Copilot and Claude harnesses.
  • Added a post-run agent_end fetchAWFReflect hook to the Pi provider for parity with other harnesses.
  • Also includes unrelated documentation/catalog updates outside the stated /reflect harness scope.
Show a summary per file
File Description
docs/src/content/docs/reference/frontmatter-full.md Adds/removes frontmatter reference documentation entries unrelated to the harness /reflect change.
docs/src/content/docs/agent-factory-status.mdx Updates the agent/workflow status table with additional workflow entries and engine labels.
actions/setup/js/pi_provider.cjs Adds pre-run and post-run /reflect fetches via Pi SDK event handlers.
actions/setup/js/copilot_harness.cjs Adds a pre-run /reflect fetch before entering the Copilot retry loop.
actions/setup/js/claude_harness.cjs Adds a pre-run /reflect fetch before entering the Claude retry loop.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

actions/setup/js/pi_provider.cjs:117

  • The new agent_end /reflect fetch is not covered by tests. This file currently has no test companion even though nearby Pi extensions are covered (actions/setup/js/pi_steering_extension.test.cjs), so a regression in whether the handler is registered or awaited would go unnoticed.
  pi.on("agent_end", async () => {
    // Fetch AWF API proxy reflection data after the agent finishes for the post-run step summary.
    // This is best-effort: failures are logged but do not affect the agent exit code.
    await fetchAWFReflect({
      reflectUrl: AWF_API_PROXY_REFLECT_URL,
      outputPath: AWF_REFLECT_OUTPUT_PATH,
      timeoutMs: AWF_REFLECT_TIMEOUT_MS,
      modelsTimeoutMs: AWF_MODELS_URL_TIMEOUT_MS,
      logger: log,
  • Files reviewed: 5/5 changed files
  • Comments generated: 4

Comment on lines +1172 to +1180
# Named model alias definitions with ordered fallback lists, resolved recursively
# by AWF. Each key is an alias name (use empty string "" for the default policy).
# Each value is an ordered list of vendor/modelid glob patterns or other alias
# names to try in sequence. Entries defined here are merged on top of the builtin
# aliases; the main workflow file always wins over imported aliases. Builtin
# aliases include: sonnet, haiku, opus, gpt-5, gpt-5-mini, gpt-5-codex,
# gemini-flash, gemini-pro, small, mini, large, auto.
# (optional)
models:
| [Deployment Incident Monitor](https://github.com/github/gh-aw/blob/main/.github/workflows/deployment-incident-monitor.md) | copilot | [![Deployment Incident Monitor](https://github.com/github/gh-aw/actions/workflows/deployment-incident-monitor.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/deployment-incident-monitor.lock.yml) | - | - |
| [Design Decision Gate 🏗️](https://github.com/github/gh-aw/blob/main/.github/workflows/design-decision-gate.md) | claude | [![Design Decision Gate 🏗️](https://github.com/github/gh-aw/actions/workflows/design-decision-gate.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/design-decision-gate.lock.yml) | - | - |
| [Dev](https://github.com/github/gh-aw/blob/main/.github/workflows/dev.md) | copilot | [![Dev](https://github.com/github/gh-aw/actions/workflows/dev.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/dev.lock.yml) | `daily around 9:00` | - |
| [Dev](https://github.com/github/gh-aw/blob/main/.github/workflows/dev.md) | pi | [![Dev](https://github.com/github/gh-aw/actions/workflows/dev.lock.yml/badge.svg)](https://github.com/github/gh-aw/actions/workflows/dev.lock.yml) | `daily around 9:00` | - |
Comment on lines +314 to +316
// Fetch AWF API proxy reflection data before running the agent to capture initial proxy state.
// This is best-effort: failures are logged but do not affect the agent run.
await fetchAWFReflect({ logger: log });
Comment on lines +221 to +223
// Fetch AWF API proxy reflection data before running the agent to capture initial proxy state.
// This is best-effort: failures are logged but do not affect the agent run.
await fetchAWFReflect({ logger: log });
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants