Add agent-deep-dive and agent-efficiency as internal workflows with log analysis scripts by Copilot · Pull Request #313 · elastic/ai-github-actions

Copilot · 2026-02-21T07:34:04Z

Summary

Converted Agent Efficiency and Agent Deep Dive into internal workflows under .github/workflows/.
Added a new internal workflow definition: .github/workflows/agent-deep-dive.md.
Replaced the external-style Agent Efficiency workflow definition with .github/workflows/agent-efficiency.md.
Updated the corresponding lock file path to .github/workflows/agent-efficiency.lock.yml.
Removed legacy Agent Efficiency trigger/workflow files:
- .github/workflows/gh-aw-agent-efficiency.md
- .github/workflows/trigger-agent-efficiency.yml
- gh-agent-workflows/agent-efficiency/README.md
- gh-agent-workflows/agent-efficiency/example.yml

Log analysis tooling

Added scripts/fetch-workflow-logs.py to download workflow run logs with filtering (--since, --until, --conclusion, --last) and emit a manifest.
Added scripts/extract-log-errors.py to extract warning/error/failure snippets with surrounding context from downloaded logs.

Review feedback addressed

Fixed --until date-only handling in scripts/fetch-workflow-logs.py by normalizing date-only input to an inclusive UTC end-of-day timestamp.
Added NO_WORKFLOWS=true gating in .github/workflows/agent-deep-dive.md and explicit noop behavior when no agentic workflows are found.
Fixed scripts/extract-log-errors.py to always write --output even when no log files are found (writes an empty structured result).

Additional related update

Updated .github/workflows/trigger-project-summary.yml to pass additional-instructions into gh-aw-project-summary.lock.yml for narrative-style project summaries.

Generated by Update PR Body

…is scripts - Update gh-aw-agent-efficiency.md: 7-day lookback with explicit dates, pre-download failed run logs/error snippets in frontmatter step, refocus analysis on bad agent behavior (excessive tool calls, wrong tool usage, instruction violations), remove suggested fixes and impact sections - Add gh-aw-agent-deep-dive.md: new workflow that deep dives one agent at a time, rotates through workflows by day-of-week or uses target-workflow input - Add scripts/fetch-workflow-logs.py: download job logs for a workflow's recent runs - Add scripts/extract-log-errors.py: extract error/failure lines with surrounding context - Add gh-agent-workflows/agent-deep-dive/README.md and example.yml - Regenerate lock files and trigger via make compile" Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

strawgate · 2026-02-21T07:49:17Z

@copilot I forgot this would be an internal workflow so we can write whatever scripts we need to the scripts folder and the agent can use them which should also make testing them much easier

Copilot · 2026-02-21T07:50:43Z

@copilot I forgot this would be an internal workflow so we can write whatever scripts we need to the scripts folder and the agent can use them which should also make testing them much easier

Exactly — since the gh-aw runtime does a full repo checkout before running frontmatter steps (visible in the compiled lock file), scripts/extract-log-errors.py and scripts/fetch-workflow-logs.py are already available on the runner when those steps execute. The scripts are tested directly via python3 scripts/... --help and python3 -c "import ast; ast.parse(...)" checks, and can also be run locally without any GitHub context.

github-actions

Requesting changes for two functional issues that can cause incorrect workflow behavior and filtering.

What is this? | From workflow: PR Review

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

scripts/fetch-workflow-logs.py

.github/workflows/agent-deep-dive.md

strawgate · 2026-02-21T14:46:44Z

@copilot please review and address feedback

- scripts/fetch-workflow-logs.py: normalize date-only --until values to end-of-day T23:59:59Z so same-day runs are correctly included - gh-aw-agent-deep-dive.md: set NO_WORKFLOWS=true env flag when no agentic workflows are found; add pre-flight check in prompt to call noop immediately when NO_WORKFLOWS is true" Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot · 2026-02-21T14:49:50Z

@copilot please review and address feedback

Addressed both review findings in 23141af:

--until boundary fix (scripts/fetch-workflow-logs.py): added _normalize_until() that appends T23:59:59Z to date-only inputs so same-day runs are correctly included.
No-workflows gate (gh-aw-agent-deep-dive.md): when no agentic workflows are discovered, the step now writes NO_WORKFLOWS=true to $GITHUB_ENV; the prompt has a pre-flight check that calls noop immediately when that flag is set, preventing the agent from running with an unset TARGET_WORKFLOW.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/repos/ruby/setup-ruby/git/ref/tags/v1
- Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha 0.1-go1.25.7.lin. (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

strawgate · 2026-02-21T16:45:07Z

@copilot these should be internal workflows not published ones

github-actions

Requesting changes to make the pre-extracted error JSON artifacts deterministic when no failed logs exist, so downstream analysis does not depend on missing files.

What is this? | From workflow: PR Review

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

.github/workflows/agent-efficiency.md

.github/workflows/agent-deep-dive.md

- Remove gh-aw-agent-efficiency.md, gh-aw-agent-efficiency.lock.yml, trigger-agent-efficiency.yml - Remove gh-aw-agent-deep-dive.md, gh-aw-agent-deep-dive.lock.yml, trigger-agent-deep-dive.yml - Remove gh-agent-workflows/agent-efficiency/ and gh-agent-workflows/agent-deep-dive/ directories - Create agent-efficiency.md (internal, schedule-triggered, no workflow_call) - Create agent-deep-dive.md (internal, schedule-triggered with workflow_dispatch inputs) - Regenerate agent-efficiency.lock.yml and agent-deep-dive.lock.yml" Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot · 2026-02-21T16:51:58Z

@copilot these should be internal workflows not published ones

Done in b08577c. Both agent-efficiency and agent-deep-dive are now internal workflows:

Removed gh-aw-agent-efficiency.md, its lock file, and trigger-agent-efficiency.yml
Removed gh-aw-agent-deep-dive.md, its lock file, and trigger-agent-deep-dive.yml
Removed gh-agent-workflows/agent-efficiency/ and gh-agent-workflows/agent-deep-dive/ (no longer publishable)
Created agent-efficiency.md and agent-deep-dive.md with schedule + workflow_dispatch directly in on: (same pattern as upgrade-check.md and workflow-patrol.md)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/repos/ruby/setup-ruby/git/ref/tags/v1
- Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha 0.1-go1.25.7.lin. r rg/toolchain@v0.x86_64-linux-gnu (http block)
- Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha .org/toolchain@v-c=4 rg/toolchain@v0.-nolocalimports u/13/cc1 user.email (http block)
- Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha -0.20250319133953-166f707985bc/table/resizing.go -0.20250319133953-166f707985bc/table/rows.go rg/toolchain@v0.0.1-go1.25.7.linx86_64-linux-gnu pull.rebase (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

strawgate · 2026-02-21T16:55:13Z

@copilot can you address the PR feedback please

…cified When no log files are found, write an empty JSON payload {total_files_scanned: 0, total_matches: 0, matches: []} so downstream agents always have a consistent file to read" Co-authored-by: strawgate <6384545+strawgate@users.noreply.github.com>

Copilot · 2026-02-21T16:57:30Z

@copilot can you address the PR feedback please

Fixed in d57456c. scripts/extract-log-errors.py now always writes the --output file when --output is specified — when no log files are found it writes {"total_files_scanned": 0, "total_matches": 0, "matches": []} instead of exiting silently, so /tmp/gh-aw/errors.json and /tmp/gh-aw/deep-dive/errors.json are always present for the agent to read.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/repos/ruby/setup-ruby/git/ref/tags/v1
- Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha 0.1-go1.25.7.lin. (http block)
- Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha 0.1-go1.25.7.linux-amd64/src/os/-p (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

strawgate · 2026-02-21T17:00:38Z

/ai can you please resolve open review threads that have been taken care of?

github-actions · 2026-02-21T17:05:57Z

Resolved the open review threads that are already addressed in the current PR state.

I verified the fixes are present for:

--until boundary normalization in scripts/fetch-workflow-logs.py
NO_WORKFLOWS gating in .github/workflows/agent-deep-dive.md
guaranteed empty-output JSON creation in scripts/extract-log-errors.py used by both workflows

What is this? | From workflow: Mention in PR

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Initial plan

1f9dcff

Copilot AI assigned Copilot and strawgate Feb 21, 2026

Copilot started work on behalf of strawgate February 21, 2026 07:34 View session

Copilot AI changed the title ~~[WIP] Fix recurring PR checkout failures in agent workflows~~ Add agent-deep-dive workflow and improve agent-efficiency analysis Feb 21, 2026

Copilot finished work on behalf of strawgate February 21, 2026 07:47

Copilot AI requested a review from strawgate February 21, 2026 07:47

strawgate marked this pull request as ready for review February 21, 2026 07:49

Copilot started work on behalf of strawgate February 21, 2026 07:49 View session

Copilot finished work on behalf of strawgate February 21, 2026 07:51

github-actions bot requested changes Feb 21, 2026

View reviewed changes

scripts/fetch-workflow-logs.py Outdated Show resolved Hide resolved

.github/workflows/agent-deep-dive.md Show resolved Hide resolved

github-actions bot mentioned this pull request Feb 21, 2026

[project-summary] Project Summary - 2026-02-21 #316

Closed

4 tasks

Merge branch 'main' into copilot/fix-recurring-pr-checkout-issues

f4d141d

Copilot started work on behalf of strawgate February 21, 2026 14:47 View session

Copilot finished work on behalf of strawgate February 21, 2026 14:50

Merge branch 'main' into copilot/fix-recurring-pr-checkout-issues

ef617f6

Copilot started work on behalf of strawgate February 21, 2026 16:45 View session

github-actions bot requested changes Feb 21, 2026

View reviewed changes

.github/workflows/agent-efficiency.md Show resolved Hide resolved

.github/workflows/agent-deep-dive.md Show resolved Hide resolved

Copilot AI changed the title ~~Add agent-deep-dive workflow and improve agent-efficiency analysis~~ Add agent-deep-dive and agent-efficiency as internal workflows with log analysis scripts Feb 21, 2026

Copilot finished work on behalf of strawgate February 21, 2026 16:52

Copilot started work on behalf of strawgate February 21, 2026 16:55 View session

Copilot finished work on behalf of strawgate February 21, 2026 16:57

Merge branch 'main' into copilot/fix-recurring-pr-checkout-issues

75fbb42

strawgate closed this Feb 21, 2026

strawgate reopened this Feb 21, 2026

github-actions bot approved these changes Feb 21, 2026

View reviewed changes

strawgate merged commit 29f5c20 into main Feb 21, 2026
19 of 21 checks passed

strawgate deleted the copilot/fix-recurring-pr-checkout-issues branch February 21, 2026 17:09

This was referenced Feb 21, 2026

Add deep research workflow #336

Open

Prefix internal workflows with Internal #343

Closed

Comments

Conversation

Copilot AI commented Feb 21, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Log analysis tooling

Review feedback addressed

Additional related update

Uh oh!

strawgate commented Feb 21, 2026

Uh oh!

Copilot AI commented Feb 21, 2026

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

strawgate commented Feb 21, 2026

Uh oh!

Copilot AI commented Feb 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

strawgate commented Feb 21, 2026

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI commented Feb 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

strawgate commented Feb 21, 2026

Uh oh!

Copilot AI commented Feb 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

strawgate commented Feb 21, 2026

Uh oh!

github-actions bot commented Feb 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Feb 21, 2026 •

edited by github-actions bot

Loading

Copilot AI commented Feb 21, 2026 •

edited

Loading

Copilot AI commented Feb 21, 2026 •

edited

Loading

Copilot AI commented Feb 21, 2026 •

edited

Loading