Skip to content

Add benchmarks and optimize diagnose overhead#4

Merged
kalverra merged 9 commits into
mainfrom
optimize
May 29, 2026
Merged

Add benchmarks and optimize diagnose overhead#4
kalverra merged 9 commits into
mainfrom
optimize

Conversation

@kalverra
Copy link
Copy Markdown
Collaborator

@kalverra kalverra commented May 28, 2026

Diagnose Overhead Benchmarks

just bench_overhead_matrix

This runs a few benchmark tests to measure the overhead the diagnose command adds to go test.

--------------------------------------------------------------------------------
Diagnose overhead vs raw go test (dogfood, target=./..., 5-run average per cell)
--------------------------------------------------------------------------------
iters  parallel  baseline    diagnose    overhead   overhead%  overhead/iter
1      1         1.618418s   1.566522s   0s         0.0%       0s
4      1         6.165101s   6.357416s   192.316ms  3.0%       48.079ms
4      4         4.994151s   5.099707s   105.555ms  2.1%       26.389ms
8      1         12.500664s  12.682847s  182.184ms  1.4%       22.773ms
8      8         9.77423s    9.753534s   0s         0.0%       0s

When I first ran the benchmark, overhead was a bit large (~50MB memory + ~7% of execution time), and so I had claude run some experiments and validate improvements with the benchmark. Memory overhead was cut to ~5MB and execution time to ~2.5%.

Also removes a bunch of dead code

@kalverra kalverra requested a review from a team as a code owner May 28, 2026 20:17
@kalverra kalverra requested a review from Copilot May 28, 2026 20:17
@kalverra kalverra enabled auto-merge (squash) May 28, 2026 20:17
@github-actions
Copy link
Copy Markdown

👋 kalverra, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.

Comment thread justfile Outdated
Comment thread justfile Outdated
Comment thread internal/runner/runner.go
@kalverra kalverra merged commit d3ac9e3 into main May 29, 2026
9 checks passed
@kalverra kalverra deleted the optimize branch May 29, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants