Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: "deno bench" subcommand #13713

Merged
merged 54 commits into from Mar 11, 2022
Merged

Conversation

bartlomieju
Copy link
Member

@bartlomieju bartlomieju commented Feb 19, 2022

Closes #9175

// Defaults to 1000 warmup iterations and 1000 bench iterations
Deno.bench("URL parsing", { n: 250, warmup: 100 }, () => {
  new URL("https://deno.land");  
});


Deno.bench("URL resolving", { n: 1000 }, () => {
  new URL("./foo.js", import.meta.url);
});
$ deno bench --unstable example_bench.ts
running 2 benches from file:///Users/ib/dev/deno/example_bench.ts
bench URL parsing ... 250 iterations 23,063 ns/iter (208..356,041 ns/iter) ok (1s)
bench URL resolving ... 1000 iterations 48,045 ns/iter (250..128,625 ns/iter) ok (4s)

bench result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out (5s)

Copy link
Contributor

@evanwashere evanwashere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mvp

  • support for up to nanoseconds accuracy and formatting
  • output should also contain min/max, can be displayed as avg unit/op (min..max unit/op) or we can take inspiration from criterion.rs BenchmarkDotNet hyperfine

optional/post mvp

  • warmup before measuring (10-1000 iterations or 1s worth of iterations with minimum of 4-10 iterations)
  • time option to limit benchmark to time quota
  • print hardware specs (see BenchmarkDotNet)
  • benchmarks grouping / baseline to compare to (see hyperfine)
  • compare results with previous run (see criterion)

@bartlomieju bartlomieju changed the title [WIP] feat: "deno bench" subcommand feat: "deno bench" subcommand Feb 26, 2022
@bartlomieju
Copy link
Member Author

Note to self: ops and resource sanitizers should be ripped out from the benchmarking setup. In preliminary tests they incurred significant cost and it's not clear that they are in anyway useful for benchmarking (might kick us out of some V8 optimizations). We can reconsider them at the later stage, but they shouldn't be added in the MVP PR.

deno.json Outdated Show resolved Hide resolved
example_bench.ts Outdated Show resolved Hide resolved
cli/tools/bench.rs Outdated Show resolved Hide resolved
Copy link
Member

@dsherret dsherret left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

cli/tools/bench.rs Show resolved Hide resolved
@bartlomieju bartlomieju merged commit 09ae512 into denoland:main Mar 11, 2022
@bartlomieju bartlomieju deleted the deno_bench branch March 11, 2022 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: deno bench subcommand
5 participants