Skip to content

Use API proxy event logs as the ambient context source of truth#37655

Merged
pelikhan merged 4 commits into
mainfrom
copilot/fix-ambient-context-analysis
Jun 7, 2026
Merged

Use API proxy event logs as the ambient context source of truth#37655
pelikhan merged 4 commits into
mainfrom
copilot/fix-ambient-context-analysis

Conversation

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

The ambient context optimizer was analyzing prompt.txt instead of the request text actually sent to the model, which led to false inline-agent findings and missed real compiler leakage. This updates the workflow guidance to treat API proxy event logs as authoritative, while keeping prompt.txt only as a diagnostic cross-check.

  • Request source selection

    • Prefer the canonical API proxy request log:
      • sandbox/firewall/logs/api-proxy-logs/event-logs.jsonl
    • Accept the legacy filename:
      • sandbox/firewall/logs/api-proxy-logs/events.jsonl
    • Preserve the matching sandbox/firewall-audit-logs/... fallback path.
    • Use Copilot session events.jsonl only as the fallback source for first user.message extraction.
  • prompt.txt semantics

    • Stop treating prompt.txt as ambient-context source of truth.
    • Keep it only for compiler/debug cross-checking against the API proxy request payload.
    • Explicitly call out inline agents or inline linters present only in prompt.txt as likely compilation bugs rather than workflow bloat.
  • Cross-validation metadata

    • Add guidance to capture:
      • request_input_tokens
      • prompt_chars
      • request_prompt_char_delta
    • Add per-request char/token ratio to make size analysis less dependent on raw character counts alone.
  • Analysis rubric

    • Extend deterministic analysis to count inline linters (## linter:) alongside inline agents and skills.
    • Clarify that the workflow should optimize the actual first request sent to the DLLM, not the compiled prompt artifact.

Example of the updated source-order rule:

1. use the first DLLM request payload from
   sandbox/firewall/logs/api-proxy-logs/event-logs.jsonl
2. fall back to sandbox/agent/logs/copilot-session-state/<session-id>/events.jsonl
3. treat prompt.txt only as a cross-check artifact

Copilot AI and others added 3 commits June 7, 2026 21:08
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix ambient context analysis to use API proxy event log Use API proxy event logs as the ambient context source of truth Jun 7, 2026
Copilot AI requested a review from pelikhan June 7, 2026 21:19
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Hey @Copilot 👋 — thanks for tackling the ambient context analysis fix! Switching the primary request source from prompt.txt to the API proxy event logs in daily-ambient-context-optimizer.md is a meaningful correctness improvement, and the complementary removal of --artifacts usage in agentics-maintenance.yml keeps both files in sync.

A couple of things to address before this is ready for review:

  • Complete the final checklist item — "Run targeted validation, secret scan, and final review checks" is still unchecked, and the PR is in draft/WIP state. Mark this done and promote to ready-for-review when validation passes.
  • No test changes in the diff — The checklist marks "Add or update focused tests for the corrected behavior" as done, but no test files appear in the diff. If tests live outside this diff (e.g., a separate commit or separate PR), please link to them. If test coverage for the new log-path logic isn't needed, a brief note in the PR body explaining why would help reviewers.

If you'd like a hand closing the loop, you can assign this prompt to your coding agent:

Complete the remaining checklist item for PR #37655 in github/gh-aw:

1. Run the targeted validation checklist:
   - Confirm the API proxy event log path `sandbox/firewall/logs/api-proxy-logs/event-logs.jsonl` (and fallback `sandbox/firewall-audit-logs/...`) resolves correctly in test runs.
   - Run a secret scan over the changed files to confirm no credentials or tokens are embedded.
   - Review the updated eligibility rules in daily-ambient-context-optimizer.md to ensure the fallback chain (API proxy → events.jsonl → prompt.txt cross-check) is logically consistent.
2. If test coverage for the new log-path resolution logic is warranted, add or update the relevant test files and include them in this PR.
3. Once all checklist items are checked, remove the [WIP] prefix from the PR title and mark the draft as ready for review.

Generated by ✅ Contribution Check · 466.7 AIC · ⌖ 13.2 AIC · ⊞ 24.3K ·

@pelikhan pelikhan marked this pull request as ready for review June 7, 2026 22:17
Copilot AI review requested due to automatic review settings June 7, 2026 22:17
@pelikhan pelikhan merged commit ed1f52f into main Jun 7, 2026
@pelikhan pelikhan deleted the copilot/fix-ambient-context-analysis branch June 7, 2026 22:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the “Daily Ambient Context Optimizer” workflow guidance to treat API proxy event logs as the authoritative source for the first DLLM request, and de-emphasizes prompt.txt to a secondary cross-check artifact to reduce false findings in ambient-context analysis.

Changes:

  • Updates sampling/extraction rules to prioritize API proxy event logs and use prompt.txt only for compilation-debug cross-checking.
  • Extends deterministic analysis guidance to include inline linter counting and request char/token ratio metadata.
  • Adjusts the maintenance workflow’s gh aw logs invocation by removing the --artifacts usage filter.
Show a summary per file
File Description
.github/workflows/daily-ambient-context-optimizer.md Revises first-request source-of-truth and analysis rubric/metadata for ambient-context optimization.
.github/workflows/agentics-maintenance.yml Changes the artifact download scope for the activity report logs step by removing --artifacts usage.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 3

Comment on lines +85 to +87
- preferred: the first DLLM request payload in the canonical `sandbox/firewall/logs/api-proxy-logs/event-logs.jsonl`, accepting the legacy `sandbox/firewall/logs/api-proxy-logs/events.jsonl` name too (including the matching `sandbox/firewall-audit-logs/...` fallback path when present)
- fallback: the first `user.message` event in `sandbox/agent/logs/copilot-session-state/<session-id>/events.jsonl`
- use `prompt.txt` only as a compilation-debug cross-check, never as the ambient-context source of truth
Comment on lines +99 to +101
1. the first DLLM request payload captured in the canonical API proxy event log `sandbox/firewall/logs/api-proxy-logs/event-logs.jsonl`, accepting the legacy `sandbox/firewall/logs/api-proxy-logs/events.jsonl` name too (or the same path under `sandbox/firewall-audit-logs/` when that artifact layout is present), because that is the text actually sent to the DLLM
2. otherwise, extract the first user-message payload from `sandbox/agent/logs/copilot-session-state/<session-id>/events.jsonl`
3. read `prompt.txt` only as a secondary compilation-debug artifact for cross-checking; do not use it as the primary request text
Comment on lines 418 to 425
run: |
${GH_AW_CMD_PREFIX} logs \
--repo "${{ github.repository }}" \
--start-date -1w \
--count 100 \
--artifacts usage \
--output ./.cache/gh-aw/activity-report-logs \
--format markdown \
> ./.cache/gh-aw/activity-report-logs/report.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[q] Fix ambient context analysis to use API proxy event log

3 participants