Improve portfolio yield telemetry ingestion and evidence classification

[Copilot]

✨ Enhancement

What does this improve?
This PR improves how portfolio-yield telemetry is ingested and interpreted so workflow evidence quality is based on clearer, more reliable signals.

• scripts/aw_yield_precompute.py
  • Normalizes telemetry into distinct dimensions (metrics, observed, validated, source)
  • Improves workflow-to-telemetry matching using repo-relative workflow paths to avoid ambiguous filename/stem matches
• scripts/aw_yield_postcompute.py
  • Updates portfolio evidence-quality outputs to use observed/validated coverage split
  • Surfaces clearer evidence classification in postcomputed reporting payloads
• .github/workflows/aw-portfolio-yield.md (+ compiled lock file)
  • Adds/updates telemetry snapshot ingestion via actions/github-script JavaScript
  • Stops pagination early once runs are outside the 90-day window to reduce API calls/runtime
  • Recompiled workflow artifact to keep source/lock in sync

Why is this valuable?
The changes reduce telemetry mis-association risk, improve confidence in evidence scoring, and make portfolio-level governance recommendations more trustworthy by distinguishing declared observability from observed and validated signals.

Implementation approach:

• Keep deterministic scoring/reporting paths intact while refining telemetry inputs and evidence classification.
• Apply targeted fixes from PR review feedback without broad behavioral changes outside telemetry ingestion/classification.
• Maintain workflow compilation parity after source workflow updates.

Testing

• Existing unit tests run for baseline validation.

• Targeted regression coverage includes telemetry path resolution behavior (including duplicate filename scenarios in different directories).

• Workflow compilation/validation run for aw-portfolio-yield to confirm lock-file consistency and valid generated workflow.

• Inspect the targeted review comment and affected code path

• Run existing lint/build/tests to establish baseline before changes

• Apply the minimal code change for the specific comment

• Run targeted tests for the changed area

• Reply to the review comment with the addressing commit hash

• Run final validation (code review + CodeQL)

[mnkiefer]

@copilot Recompile workflow

[Copilot]

@copilot Recompile workflow

Done in 2f43989.

[github-actions]

Hey @copilot-swe-agent 👋 — nice work improving the aw-portfolio-yield telemetry ingestion and evidence classification! The changes across the precompute/postcompute scripts and their tests look cohesive and on-topic.

One thing is holding this back from being review-ready:

• Missing description — the PR body currently contains only the agent's task checklist. Reviewers need a human-readable summary of what changed (e.g. what telemetry fields were added, how evidence classification was improved) and why (the motivation or linked issue). Please replace or supplement the checklist with a proper description.

If you'd like a hand, you can assign this prompt to your coding agent:

Update the PR body for PR #31483 in github/gh-aw to include a proper description.
Replace the task checklist with a markdown summary that covers:
1. What changed — summarise the key modifications to aw_yield_precompute.py, aw_yield_postcompute.py, and the aw-portfolio-yield workflow definition.
2. Why — the motivation for improving telemetry ingestion and evidence classification (link the originating issue if one exists).
3. How to test — brief notes on what the updated tests cover.
Keep the checklist below the description if it is still useful for tracking agent progress.

Generated by Contribution Check · ● 8.5M · ◷

[mnkiefer]

@copilot Update the PR body for PR #31483 in github/gh-aw to include a proper description.
Replace the task checklist with a markdown summary that covers:

1. What changed — summarise the key modifications to aw_yield_precompute.py, aw_yield_postcompute.py, and the aw-portfolio-yield workflow definition.
2. Why — the motivation for improving telemetry ingestion and evidence classification (link the originating issue if one exists).
3. How to test — brief notes on what the updated tests cover.
   Keep the checklist below the description if it is still useful for tracking agent progress.

[Copilot]

@copilot Update the PR body for PR #31483 in github/gh-aw to include a proper description.
Replace the task checklist with a markdown summary that covers:

1. What changed — summarise the key modif...

Updated the PR body. Current head is 2f43989.

[Copilot]

Pull request overview

Updates the portfolio yield workflow to ingest telemetry from actual GitHub Actions run data and refines evidence scoring/coverage to distinguish declared observability vs observed vs validated telemetry signals.

Changes:

• Extend precompute telemetry normalization to separate metrics, observed, validated, and source, and update workflow + portfolio evidence scoring and coverage metrics accordingly.
• Update postcompute to derive portfolio evidence quality from observed/validated coverage and surface the split coverage metrics in payloads/reports.
• Add a workflow pre-agent step to snapshot recent workflow run telemetry into JSON and pass it into precompute; regenerate the compiled workflow artifact.

Show a summary per file

File	Description
scripts/aw_yield_precompute.py	Normalizes telemetry entries into structured fields and updates evidence/coverage scoring logic.
scripts/aw_yield_postcompute.py	Computes derived evidence quality and reports declared/observed/validated telemetry coverage.
tests/test_aw_yield_precompute.py	Adds tests for telemetry normalization, evidence quality behavior, and split coverage metrics.
tests/test_aw_yield_postcompute.py	Updates fixtures/assertions to validate split coverage reporting and confidence rules.
.github/workflows/aw-portfolio-yield.md	Adds a pre-agent telemetry snapshot step and wires its output into precompute via env var.
.github/workflows/aw-portfolio-yield.lock.yml	Recompiled workflow to include the new telemetry snapshot step and env wiring.
.github/workflows/issue-monster.lock.yml	Minor comment-only change in the compiled workflow artifact.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 7/7 changed files
• Comments generated: 2

[mnkiefer]

@copilot Do not use python or bash but actions/github-script and javascript.

[Copilot]

@copilot Do not use python or bash but actions/github-script and javascript.

For this PR change, the telemetry collection/update logic is implemented in actions/github-script using JavaScript. No Python was introduced.