Skip to content

perf: fix ~28% BenchmarkYAMLGeneration regression by eliminating reflection hot path#30208

Merged
pelikhan merged 4 commits intomainfrom
copilot/fix-yaml-generation-performance
May 5, 2026
Merged

perf: fix ~28% BenchmarkYAMLGeneration regression by eliminating reflection hot path#30208
pelikhan merged 4 commits intomainfrom
copilot/fix-yaml-generation-performance

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 4, 2026

BenchmarkYAMLGeneration regressed ~28% (1.62ms → 2.08ms) due to a hot path executing on every compilation.

Root cause

hasAnySafeOutputEnabled / hasNonBuiltinSafeOutputsEnabled — both used reflect.Value.FieldByName in a loop across 41+ SafeOutputsConfig pointer fields (~54µs/iter). Because applyDefaultCreateIssue always produces a non-nil SafeOutputsConfig even for workflows without safe-outputs:, the reflection loop ran unconditionally on every compile.

Changes

  • safe_outputs_state.go: Replace reflection loops in both hasAnySafeOutputEnabled (43 fields) and hasNonBuiltinSafeOutputsEnabled (40 non-builtin fields) with direct != nil pointer comparisons — zero allocation, zero reflection overhead. Also adds CommentMemory which was previously missing from both checks: workflows using only tools: comment-memory: were incorrectly treated as having no safe outputs, preventing safe-output prompt/config generation and causing an unwanted auto-injected create-issue. Remove now-dead variables (safeOutputReflectionLog, builtinSafeOutputFields, nonBuiltinSafeOutputFieldNames).

  • safe_outputs_state_test.go (new): TestSafeOutputStateFieldCoverage iterates safeOutputFieldMapping via reflection and asserts each field triggers both check functions — fails immediately if a new safe output type is added to the mapping without updating the direct-check lists. TestSafeOutputStateCommentMemoryCoverage explicitly guards CommentMemory, which lives outside safeOutputFieldMapping.

  • safe_jobs_needs_validation.go: Update stale comment that referred to the removed reflection-based implementation.

Before: ~1,020,330 ns/op
After:  ~862,000 ns/op  (~15% improvement on benchmark host)

Copilot AI and others added 2 commits May 4, 2026 19:41
…ection

- Replace reflection-based hasAnySafeOutputEnabled with direct nil-checks for
  all 41 SafeOutputsConfig pointer fields — eliminates ~37 reflect.FieldByName
  calls per compilation, saving ~50µs/iteration
- Replace reflection-based hasNonBuiltinSafeOutputsEnabled similarly
- Cache os.Getwd() with sync.Once in ToRelativePath to eliminate repeated
  syscalls during compilation output formatting
- Remove now-unused safeOutputReflectionLog, builtinSafeOutputFields, and
  nonBuiltinSafeOutputFieldNames variables
- Update file header comment and safeOutputFieldMapping doc to guide future
  contributors when adding new safe output types

Benchmark result: BenchmarkYAMLGeneration improved ~14%
  Before: ~1,020,330 ns/op
  After:  ~862,000 ns/op

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/eaeef1d3-12cd-442d-902b-fcfb53133977

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…tions against drift

Add a reflection-based test that validates every field in safeOutputFieldMapping
is also covered by hasAnySafeOutputEnabled and hasNonBuiltinSafeOutputsEnabled.
If a new safe output type is added to safeOutputFieldMapping without updating the
direct-check functions, this test will fail immediately.

Also add field count comments to both functions (42 total, 39 non-builtin).

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/eaeef1d3-12cd-442d-902b-fcfb53133977

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix performance regression in YAML generation perf: fix ~28% BenchmarkYAMLGeneration regression by eliminating reflection hot path May 4, 2026
Copilot AI requested a review from gh-aw-bot May 4, 2026 19:48
@pelikhan pelikhan marked this pull request as ready for review May 4, 2026 20:39
Copilot AI review requested due to automatic review settings May 4, 2026 20:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes two compile-time hot paths by replacing reflective safe-output state checks with direct field checks and by caching the working directory used for path rendering.

Changes:

  • Replaced reflection in hasAnySafeOutputEnabled and hasNonBuiltinSafeOutputsEnabled with explicit nil checks.
  • Added a regression test to keep the manual safe-output field lists aligned with the mapped handler fields.
  • Cached os.Getwd() in ToRelativePath using sync.Once to avoid repeated syscalls.
Show a summary per file
File Description
pkg/workflow/safe_outputs_state.go Reworks safe-output enabled-state detection to use direct field checks instead of reflection.
pkg/workflow/safe_outputs_state_test.go Adds a reflection-driven regression test for the manual safe-output check lists.
pkg/console/render.go Caches the working directory for relative-path rendering.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

pkg/workflow/safe_outputs_state.go:153

  • CommentMemory is also missing from the non-builtin list. Because applyDefaultCreateIssue() runs after comment-memory is attached to SafeOutputs (compiler_orchestrator_workflow.go:322-329, 409-411), a workflow whose only meaningful output is comment-memory is still seen as having no non-builtin outputs and gets an auto-injected create-issue it did not ask for.
	// Direct nil checks for non-builtin pointer fields (39 fields = 42 total minus 3 builtins:
	// NoOp, MissingData, MissingTool).
	return safeOutputs.CreateIssues != nil ||
		safeOutputs.CreateAgentSessions != nil ||
		safeOutputs.CreateDiscussions != nil ||
  • Files reviewed: 3/3 changed files
  • Comments generated: 3

Comment thread pkg/console/render.go Outdated
Comment on lines +603 to +605
workingDirOnce.Do(func() {
workingDir, workingDirErr = os.Getwd()
})
Comment thread pkg/workflow/safe_outputs_state.go Outdated
Comment on lines +88 to +92
// Direct nil checks — no reflection, no heap allocation (42 fields matching safeOutputFieldMapping).
return safeOutputs.CreateIssues != nil ||
safeOutputs.CreateAgentSessions != nil ||
safeOutputs.CreateDiscussions != nil ||
safeOutputs.UpdateDiscussions != nil ||
Comment on lines +13 to +17
// TestSafeOutputStateFieldCoverage verifies that hasAnySafeOutputEnabled and
// hasNonBuiltinSafeOutputsEnabled cover every pointer field listed in
// safeOutputFieldMapping. This acts as a regression guard to ensure that when
// a new safe output type is added to safeOutputFieldMapping, the developer is
// reminded (via a failing test) to also update the two direct-check functions.
@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented May 4, 2026

@copilot review all changes

…hdir bug

- Add CommentMemory to hasAnySafeOutputEnabled and hasNonBuiltinSafeOutputsEnabled:
  a workflow that only enables tools.comment-memory was incorrectly treated as
  having no safe outputs, preventing safe-output prompt/config generation.
- Revert sync.Once caching in ToRelativePath: the codebase calls os.Chdir() in
  production paths (trial_helpers.go, trial_repository.go, pr_command.go) so
  caching the working directory on first access would yield stale paths after
  any directory change.
- Update stale "reflection-based" comment in safe_jobs_needs_validation.go.
- Add TestSafeOutputStateCommentMemoryCoverage to explicitly guard CommentMemory,
  which lives outside safeOutputFieldMapping and is not caught by the map-driven
  TestSafeOutputStateFieldCoverage test.
- Bump field counts to 43 total / 40 non-builtin.

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/3d0069d6-f3d5-47eb-b2b3-c39f758faeda

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 4, 2026

@copilot review all changes

Three issues were found and fixed in commit 4196f0d:

  1. CommentMemory missing from both direct-check functions — a workflow using only tools.comment-memory was incorrectly treated as having no safe outputs, which would prevent safe-output prompt/config generation and wrongly inject an auto create-issue. Added CommentMemory to both hasAnySafeOutputEnabled and hasNonBuiltinSafeOutputsEnabled.

  2. ToRelativePath sync.Once cache — reverted. The codebase calls os.Chdir() in production paths (trial_helpers.go, trial_repository.go, pr_command.go), so caching the working directory on first access would yield stale paths after any directory change.

  3. Test gap for CommentMemory — added TestSafeOutputStateCommentMemoryCoverage to explicitly assert both check functions return true when only CommentMemory is set, since it lives outside safeOutputFieldMapping and is not covered by the map-driven TestSafeOutputStateFieldCoverage test.

Copilot AI requested a review from pelikhan May 4, 2026 21:45
@pelikhan pelikhan merged commit 06e7da5 into main May 5, 2026
@pelikhan pelikhan deleted the copilot/fix-yaml-generation-performance branch May 5, 2026 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[performance] Regression in YAMLGeneration: 28.0% slower

4 participants