Skip to content

Strengthen daily max-credit test prompt with forced multi-turn large-file reads#37669

Merged
pelikhan merged 2 commits into
mainfrom
copilot/update-daily-max-credit-test-prompt
Jun 7, 2026
Merged

Strengthen daily max-credit test prompt with forced multi-turn large-file reads#37669
pelikhan merged 2 commits into
mainfrom
copilot/update-daily-max-credit-test-prompt

Conversation

Copilot AI commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

The daily max credit test was too lightweight and could complete without reliably crossing the per-run credit boundary. This change makes the prompt explicitly drive higher token/credit consumption so the max-ai-credits: 1 guardrail is exercised deterministically.

  • What changed

    • Updated /home/runner/work/gh-aw/gh-aw/.github/workflows/daily-max-ai-credits-test.md to require a minimum of 4 assistant turns.
    • Reworked the task to include multiple explicit large-file read jobs instead of a single noop-style call.
    • Tightened expected behavior language so the intended outcome is guardrail cutoff before all jobs finish.
  • Prompt contract updates

    • Turn 1: start signal via noop.
    • Turns 2–4: read and summarize these large files:
      • pkg/parser/schemas/main_workflow_schema.json
      • .github/workflows/daily-max-ai-credits-test.lock.yml
      • .github/workflows/daily-credit-limit-test.lock.yml
    • Preserve fallback instruction if the run unexpectedly does not hit the credit limit.
  • Illustrative prompt excerpt

    Use at least four separate assistant turns.
    
    Turn 2 (Job 1): Read `pkg/parser/schemas/main_workflow_schema.json`.
    Turn 3 (Job 2): Read `.github/workflows/daily-max-ai-credits-test.lock.yml`.
    Turn 4 (Job 3): Read `.github/workflows/daily-credit-limit-test.lock.yml`.
    
    Since max-ai-credits is 1, the firewall should cut off the agent before all turns complete.

Copilot AI and others added 2 commits June 7, 2026 22:44
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Force multi-turn large-file reads in daily max AI credits test prompt Strengthen daily max-credit test prompt with forced multi-turn large-file reads Jun 7, 2026
Copilot AI requested a review from pelikhan June 7, 2026 22:46
@pelikhan pelikhan marked this pull request as ready for review June 7, 2026 22:49
Copilot AI review requested due to automatic review settings June 7, 2026 22:49
@pelikhan pelikhan merged commit 8b223a9 into main Jun 7, 2026
@pelikhan pelikhan deleted the copilot/update-daily-max-credit-test-prompt branch June 7, 2026 22:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make the “Daily Max AI Credits Test” more reliably exceed the per-run max-ai-credits: 1 budget by forcing multiple assistant turns and multiple large-file read tasks, so the firewall cutoff behavior is exercised deterministically.

Changes:

  • Strengthened the daily-max-ai-credits-test prompt to require at least 4 assistant turns and explicit large-file reads across turns.
  • Updated the expected-outcome wording to emphasize a cutoff before all jobs complete.
  • Refreshed generated content in daily-ambient-context-optimizer.lock.yml (metadata and workflow description string).
Show a summary per file
File Description
.github/workflows/daily-max-ai-credits-test.md Updates the intentionally-failing prompt contract to drive higher credit/token usage via forced multi-turn large-file reads.
.github/workflows/daily-ambient-context-optimizer.lock.yml Updates generated lock metadata and workflow description text for the ambient context optimizer workflow.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment on lines +47 to 50
Use **at least four separate assistant turns**. Do not combine all work into one response.

This invocation consumes AI credits. Since the per-run budget is 1, the AWF firewall will
cut off the agent before or immediately after this call, causing the run to fail.
Turn 1: Call `noop` with the message: "Starting max-ai-credits multi-turn guardrail test."

Comment on lines 1418 to 1421
env:
WORKFLOW_NAME: "Daily Ambient Context Optimizer"
WORKFLOW_DESCRIPTION: "Samples recent agentic workflow runs, inspects the first DLLM request text, and recommends prompt, skill, and agent changes to shrink ambient context"
WORKFLOW_DESCRIPTION: "Samples recent agentic workflow runs, inspects the first DLLM request from API proxy event logs, and recommends prompt, skill, and agent changes to shrink ambient context"
HAS_PATCH: ${{ needs.agent.outputs.has_patch }}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants