Skip to content

Surface plugins in GitHub Actions and console run summaries - #763

Merged
Sun Haoran (haoranpb) merged 3 commits into
mainfrom
copilot/surface-loaded-plugins-in-summaries
Jul 31, 2026
Merged

Surface plugins in GitHub Actions and console run summaries#763
Sun Haoran (haoranpb) merged 3 commits into
mainfrom
copilot/surface-loaded-plugins-in-summaries

Conversation

Copilot AI commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

ExperimentConfiguration.plugins was populated by PR #753 but never rendered in run summaries — reviewers had no way to see which plugins were loaded from the job summary or console output.

Changes

src/bcbench/results/display.py

  • Added - Plugins: line to create_github_job_summary header, placed after - Custom Agent:, treating plugins as a list exactly like mcp_servers (, -joined, falls back to None)
  • Added matching Plugins: line to create_console_summary for parity
f"- Plugins: {', '.join(results[0].experiment.plugins) if results[0].experiment and results[0].experiment.plugins else 'None'}",

tests/test_result_hierarchy.py

  • Fixed test_github_summary_renders_markdown: split the "- Custom Agent: N/A\n\n## Result Summary" adjacency assertion since - Plugins: is now the last header line; also asserts - Plugins: None when no plugins are set
  • Added test_github_summary_shows_plugins_when_present: verifies both plugin entries ("<name>@<sha>" / "<name>@local") appear joined in the rendered content

Co-authored-by: haoranpb <27280733+haoranpb@users.noreply.github.com>
Copilot AI requested review from Copilot and removed request for Copilot July 30, 2026 08:03
Copilot AI changed the title [WIP] Surface loaded plugins in run summaries Surface plugins in GitHub Actions and console run summaries Jul 30, 2026
@haoranpb
Sun Haoran (haoranpb) marked this pull request as ready for review July 30, 2026 14:01
Copilot AI review requested due to automatic review settings July 30, 2026 14:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Surfaces configured plugins in console and GitHub Actions run summaries.

Changes:

  • Adds plugin lists to both summary formats.
  • Adds GitHub summary assertions for configured and absent plugins.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/bcbench/results/display.py Renders plugin configuration in summaries.
tests/test_result_hierarchy.py Tests GitHub plugin summary output.

Comment thread src/bcbench/results/display.py
@haoranpb
Sun Haoran (haoranpb) enabled auto-merge (squash) July 30, 2026 14:05
Copilot AI review requested due to automatic review settings July 31, 2026 07:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/bcbench/results/display.py:34

  • PluginConfig.name is unrestricted, so a valid plugin record may contain Rich markup such as [red]. Interpolating it inside a markup-enabled string makes Rich interpret the name; unknown or mismatched tags can raise MarkupError and prevent the console summary from being emitted, while valid tags alter the displayed identifier. Render the dynamic value with markup disabled (or escape it) instead.
    console.print(f"Plugins: [bold]{', '.join(results[0].experiment.plugins) if results[0].experiment and results[0].experiment.plugins else 'None'}[/bold]")

src/bcbench/results/display.py:102

  • Plugin names are not constrained to Markdown-safe single-line text. For example, a valid local plugin name containing a newline ends this list item and creates arbitrary additional Markdown, so the job summary no longer shows the recorded plugin identifier faithfully. Escape Markdown/HTML metacharacters and normalize line breaks before adding plugin records to the summary.
            f"- Plugins: {', '.join(results[0].experiment.plugins) if results[0].experiment and results[0].experiment.plugins else 'None'}",

@haoranpb
Sun Haoran (haoranpb) merged commit 53fdde9 into main Jul 31, 2026
15 checks passed
@haoranpb
Sun Haoran (haoranpb) deleted the copilot/surface-loaded-plugins-in-summaries branch July 31, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants