Skip to content

perf: fix FindIncludesInContent regression (+33.8%) by eliminating regex allocations#28865

Merged
pelikhan merged 6 commits intomainfrom
copilot/performance-fix-find-includes-regression
Apr 28, 2026
Merged

perf: fix FindIncludesInContent regression (+33.8%) by eliminating regex allocations#28865
pelikhan merged 6 commits intomainfrom
copilot/performance-fix-find-includes-regression

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 28, 2026

findIncludesInContent called parser.ParseImportDirective per line, which runs a complex alternation regex via FindStringSubmatch — allocating a submatch []string, backtracking state, and an ImportDirectiveMatch struct on every directive hit. This caused 18 allocs/op and was the source of the reported regression.

Changes

  • pkg/cli/remove_command.go: Replace ParseImportDirective with a new parseIncludePath helper that uses strings.HasPrefix + strings.Cut + strings.TrimSpace instead of regex. Handles all three directive forms: @include, @import, {{#import}}.
  • Correctness fix in {{#import}} colon handling: Explicit single-colon removal to match the original regex's \s*:?\s* semantics.
  • Correctness fix in {{#import}} trailing content: Validates that text after }} is empty/whitespace, rejecting lines like {{#import shared/x.md}} trailing (matching the original anchored regex's behavior).
  • Tests: Added 8 new test cases covering @import (basic, optional marker, section reference) and all {{#import}} variants (colon, no-colon, optional marker, section reference, trailing-content rejection).
  • Removed now-unused parser import.

Benchmark (AMD EPYC 7763)

ns/op B/op allocs/op
Before 2109 840 18
After 265 112 3
Δ ~8× faster ~7.5× less 6× fewer

Copilot AI and others added 2 commits April 28, 2026 06:34
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e0d1eeba-1e83-4b71-9ad8-7d48cbdb4d2a

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…rect string ops

Replace parser.ParseImportDirective (regex + struct allocs) in findIncludesInContent
with inline parseIncludePath helper using strings.HasPrefix/Cut/TrimSpace.

Benchmark (AMD EPYC 7763):
  Before: 2109 ns/op  840 B/op  18 allocs/op
  After:   265 ns/op  112 B/op   3 allocs/op  (~8x faster, 6x fewer allocs)

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/e0d1eeba-1e83-4b71-9ad8-7d48cbdb4d2a

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix performance regression in FindIncludesInContent perf: fix FindIncludesInContent regression (+33.8%) by eliminating regex allocations Apr 28, 2026
Copilot AI requested a review from gh-aw-bot April 28, 2026 06:45
@github-actions github-actions Bot mentioned this pull request Apr 28, 2026
@pelikhan pelikhan marked this pull request as ready for review April 28, 2026 09:10
Copilot AI review requested due to automatic review settings April 28, 2026 09:10
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves performance of include scanning in findIncludesInContent by replacing regex-based import directive parsing with a zero/low-allocation string parser, addressing a reported regression.

Changes:

  • Replaced parser.ParseImportDirective usage in findIncludesInContent with a new parseIncludePath helper using HasPrefix/Cut/TrimSpace.
  • Added pull-requests: write to the activation job permissions in several *.lock.yml workflows.
  • Removed now-unused parser import from pkg/cli/remove_command.go.
Show a summary per file
File Description
pkg/cli/remove_command.go Introduces parseIncludePath and switches include scanning to use it instead of the regex-based parser.
.github/workflows/unbloat-docs.lock.yml Adds pull-requests: write permission to the activation job.
.github/workflows/tidy.lock.yml Adds pull-requests: write permission to the activation job.
.github/workflows/plan.lock.yml Adds pull-requests: write permission to the activation job.
.github/workflows/pdf-summary.lock.yml Adds pull-requests: write permission to the activation job.
.github/workflows/mergefest.lock.yml Adds pull-requests: write permission to the activation job.
.github/workflows/brave.lock.yml Adds pull-requests: write permission to the activation job.
.github/workflows/ace-editor.lock.yml Adds pull-requests: write permission to the activation job.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

.github/workflows/tidy.lock.yml:95

  • This PR’s title/description focuses on the Go perf regression in FindIncludesInContent, but it also changes workflow token permissions by adding pull-requests: write here (and in other lock workflows). If this is intentional, please update the PR description to explain why these permission changes are needed (or consider splitting them into a separate PR).
      actions: read
      contents: read
      issues: write
      pull-requests: write
  • Files reviewed: 8/8 changed files
  • Comments generated: 9

Comment thread pkg/cli/remove_command.go Outdated
Comment on lines +434 to +436
// Extract path up to closing "}}"
before, _, ok := strings.Cut(rest, "}}")
if !ok {
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the {{#import}} form, this parses everything up to the first "}}" but does not verify that the closing braces are at the end of the directive (the previous anchored regex required the line to end at "}}" apart from whitespace). As a result, a line like "{{#import shared/x.md}} trailing" would now be treated as a directive and produce a false include. Consider validating that the text after the cut is empty/whitespace (e.g., capture the remainder from strings.Cut and require strings.TrimSpace(remainder)=="") before returning a path.

Suggested change
// Extract path up to closing "}}"
before, _, ok := strings.Cut(rest, "}}")
if !ok {
// Extract path up to closing "}}" and require only whitespace after it.
before, after, ok := strings.Cut(rest, "}}")
if !ok || strings.TrimSpace(after) != "" {

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/tidy.lock.yml Outdated
actions: read
contents: read
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

This issue also appears on line 92 of the same file.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/plan.lock.yml Outdated
contents: read
discussions: write
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/brave.lock.yml Outdated
actions: read
contents: read
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
Comment thread pkg/cli/remove_command.go
Comment on lines +405 to +409
// parseIncludePath extracts the file path from @include/@import/{{#import}} directive lines
// without allocating a regex submatch slice or a directive struct.
// Returns an empty string if the line is not a recognised directive.
// Section references (e.g. file.md#Section) are stripped from the returned path.
func parseIncludePath(line string) string {
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new parsing logic supports @import and {{#import}} (with optional '?' and optional ':'), but the existing findIncludesInContent tests only cover @include. Please add unit tests that exercise @import and the {{#import}} variants (colon/no-colon and optional marker) to ensure behavior stays aligned with the legacy regex-based parser.

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/unbloat-docs.lock.yml Outdated
actions: read
contents: read
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/mergefest.lock.yml Outdated
actions: read
contents: read
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/pdf-summary.lock.yml Outdated
actions: read
contents: read
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/ace-editor.lock.yml Outdated
actions: read
contents: read
issues: write
pull-requests: write
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding pull-requests: write increases the GITHUB_TOKEN scope for this job. The activation job’s “Add comment with workflow run link” script posts via the issues comments endpoint (/repos/{owner}/{repo}/issues/{number}/comments in actions/setup/js/add_workflow_run_comment.cjs), which only requires issues: write. Unless another step in this job truly needs PR write access, please drop pull-requests: write to follow least-privilege permissions.

Suggested change
pull-requests: write

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🛠️ Agentic Maintenance updated this pull request branch.

View workflow run

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

…t}} trailing content, add tests

- Revert unintended pull-requests:write additions to 7 lock files
- Fix {{#import}} parser to reject directives with trailing content after }}
- Add test cases for @import and {{#import}} directive variants

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/042e8ccf-a206-4fd4-8347-dcf4020b615f

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 28, 2026

@copilot review all comments

Addressed all comments in commit 482efaa:

  1. {{#import}} trailing content (line 434-436): fixed to capture and validate the remainder after }}, rejecting lines like {{#import shared/x.md}} trailing.
  2. Missing tests (lines 405-409): added 8 new test cases covering @import basic, optional marker, section reference, and all {{#import}} variants (colon, no-colon, optional marker, section reference, and trailing-content rejection).
  3. pull-requests: write in lock files: reverted the unintended additions to all 7 affected .lock.yml files.

Copilot AI requested a review from pelikhan April 28, 2026 09:35
@github-actions
Copy link
Copy Markdown
Contributor

🛠️ Agentic Maintenance updated this pull request branch.

View workflow run

@pelikhan pelikhan merged commit 49d5bc6 into main Apr 28, 2026
@pelikhan pelikhan deleted the copilot/performance-fix-find-includes-regression branch April 28, 2026 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[performance] Regression in FindIncludesInContent: +33.8% slower

4 participants