Skip to content

follow-up: preserve manual NOTES block in incremental benchmark generator #1042

@carlos-alm

Description

@carlos-alm

Deferred from PR #1032 review.

Original reviewer comment: #1032 (comment)

Context:
scripts/update-incremental-report.ts regenerates generated/benchmarks/INCREMENTAL-BENCHMARKS.md from scratch on every release run, overwriting the entire file. PR #1008 added a manual <!-- NOTES_START --> ... <!-- NOTES_END --> block explaining why 3.9.5 has null build metrics (workers hung past the 10-min timeout in scripts/lib/fork-engine.ts and were SIGKILL'd). The 3.9.6 auto-update silently dropped that note because the script has no awareness of the NOTES sentinel pair.

After the 3.9.6 release, anyone reading the summary table sees the version history jump 3.9.4 → 3.9.6 with no explanation for the gap.

Fix:
Update scripts/update-incremental-report.ts to:

  1. Before rewriting reportPath, read the existing file and extract any <!-- NOTES_START -->...<!-- NOTES_END --> block (mirroring how the existing code extracts INCREMENTAL_BENCHMARK_DATA).
  2. Re-emit the captured notes block in the same position in the regenerated markdown.

Affects: scripts/update-incremental-report.ts, regression test for the script roundtrip if one exists.

Metadata

Metadata

Assignees

No one assigned

    Labels

    follow-upDeferred work from PR reviews that needs tracking

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions