benchmark: add bench for zlib gzip + gunzip cycle #20034

addaleax · 2018-04-14T16:28:55Z

Originally wrote this for some work that is going to take a while
longer before it’s ready to be PR’ed, so it seems fine to start
with this on its own.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

lpinca · 2018-04-15T06:54:16Z

benchmark/zlib/pipe.js

+});
+
+function main({ inputLen, duration, type }) {
+  const buffer = Buffer.alloc(inputLen, fs.readFileSync(__filename));


Nit: any reason for using fs.readFileSync(__filename) instead of an arbitrary fill character? Is it for making the chunks to gzip/gunzip realistic?

@lpinca Yes, exactly that. If you have other suggestions, I’ll take them too :)

Nah, that's ok and I have no better ideas.

BridgeAR · 2018-04-15T20:41:00Z

benchmark/zlib/pipe.js

+  inputLen: [1024],
+  duration: [5],
+  type: ['string', 'buffer']
+});


These options have to be reflected in the benchmark test as well (parallel/test-benchmark-zlib.js).

@BridgeAR they are no? See line 14.

As far as I can tell, inputLen and duration have to be added to the test.

~~Oh, true.~~

Actually inputLen and duration are used in the test.

Right now we just have:

runBenchmark('zlib', [ 'method=deflate', 'n=1', 'options=true', 'type=Deflate' ]);

It seems this benchmark is like net benchmarks where throughput is calculated after x sec. I don't think this will be run in CI. Or I am not understanding the issue :)

Right but we have tests that run these benchmarks to confirm that they work and also to confirm that we didn't break some functionality. See test/parallel/test-benchmark-zlib.js.

I see now, I didn't read the original comment carefully. Sorry for the noise.

BridgeAR · 2018-04-15T20:43:02Z

benchmark/zlib/pipe.js

+    bench.end(readFromOutput * 8 / (1024 ** 3));
+
+    // Cut off writing the easy way.
+    input.write = () => {};


Should not better write be replaced instead of input.write?

I’d find it a bit surprising to override something defined with function foo(){}, tbh…?

BridgeAR · 2018-04-23T08:21:03Z

Ping @addaleax

BridgeAR

Just to make sure it does not land as is.

Originally wrote this for some work that is going to take a while longer before it’s ready to be PR’ed, so it seems fine to start with this on its own.

addaleax · 2018-04-27T19:09:03Z

@BridgeAR PTAL

CI: https://ci.nodejs.org/job/node-test-pull-request/14552/

addaleax · 2018-04-27T20:01:07Z

CI: https://ci.nodejs.org/job/node-test-commit/18085/

refack · 2018-04-27T20:08:06Z

test/parallel/test-benchmark-zlib.js

-               'type=Deflate'
+               'type=Deflate',
+               'inputLen=1024',
+               'duration=1'


I'm -0 on this. It's heuristic (so still might fail on a really slow or clogged machine), probably safer to call bench.end(gb) with

// Give result in GBit/s, like the net benchmarks do. // Never return 0, so that tests won't fail on really slow machines. const gb = (readFromOutput * 8 / (1024 ** 3)) || 1

ohh and #20128 - settings duration=1 will add a 1sec test to the suite.

Do we do this elsewhere? Otherwise it seems like it would falsify test results – what if the test is actually broken and returns 0?

Would you be be okay with the current code if it passes a stress test run?

Regarding the duration: Yeah, that’s not great. It’s not quite as bad as if it were a sequential test though…

Let me read through the test and harness a little bit more. AFAIK the test-benchmark-* set are there "mostly" for syntax, API and regression testing. Designing correctness tests for the benchmarks might be out-of-scope.
Maybe we can just relax the Error: called end() with operation count <= 0 rule anyway 🤔

Found it:

node/benchmark/common.js

Line 197 in 48e90ed

if (!process.env.NODEJS_BENCHMARK_ZERO_ALLOWED && operations <= 0) {

so calling:

runBenchmark('zlib', [ 'method=deflate', 'n=1', 'options=true', 'type=Deflate', 'inputLen=1024', 'duration=0.001' ], { 'NODEJS_BENCHMARK_ZERO_ALLOWED': 1 });

should get the test to pass.

Thanks, done!

addaleax · 2018-04-27T20:38:06Z

CI: https://ci.nodejs.org/job/node-test-commit/18087/

apapirovski · 2018-04-28T06:31:00Z

~~@addaleax I can't get this properly landed, it seems to be giving me a conflict. Any chance you can rebase?~~ My bad. User error 😆

apapirovski · 2018-04-28T06:34:15Z

Landed in e797d5b

Originally wrote this for some work that is going to take a while longer before it’s ready to be PR’ed, so it seems fine to start with this on its own. PR-URL: #20034 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

nodejs-github-bot added benchmark Issues and PRs related to the benchmark subsystem. zlib Issues and PRs related to the zlib subsystem. labels Apr 14, 2018

jasnell approved these changes Apr 14, 2018

View reviewed changes

lpinca approved these changes Apr 15, 2018

View reviewed changes

lpinca reviewed Apr 15, 2018

View reviewed changes

BridgeAR reviewed Apr 15, 2018

View reviewed changes

BridgeAR requested changes Apr 23, 2018

View reviewed changes

addaleax added 2 commits April 27, 2018 21:04

benchmark: add bench for zlib gzip + gunzip cycle

d7f3530

Originally wrote this for some work that is going to take a while longer before it’s ready to be PR’ed, so it seems fine to start with this on its own.

[squash] add new parameter types to benchmark test

5c2517f

addaleax force-pushed the zlib-pipe-benchmark branch from c903705 to 5c2517f Compare April 27, 2018 19:07

BridgeAR approved these changes Apr 27, 2018

View reviewed changes

[squash] duration 0.1 → 1 for CI

4caf889

addaleax added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 27, 2018

refack reviewed Apr 27, 2018

View reviewed changes

addaleax removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 27, 2018

[squash] refack suggestion

1cf46df

addaleax added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 27, 2018

apapirovski closed this Apr 28, 2018

MylesBorins mentioned this pull request May 8, 2018

v10.1.0 proposal #20606

Merged

addaleax added the dont-land-on-v8.x label Jun 29, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

benchmark: add bench for zlib gzip + gunzip cycle #20034

benchmark: add bench for zlib gzip + gunzip cycle #20034

addaleax commented Apr 14, 2018

lpinca Apr 15, 2018 •

edited

Loading

addaleax Apr 15, 2018

lpinca Apr 15, 2018

BridgeAR Apr 15, 2018

lpinca Apr 22, 2018

apapirovski Apr 22, 2018

lpinca Apr 22, 2018 •

edited

Loading

lpinca Apr 22, 2018

apapirovski Apr 22, 2018

lpinca Apr 22, 2018

apapirovski Apr 22, 2018

lpinca Apr 22, 2018 •

edited

Loading

addaleax Apr 27, 2018

BridgeAR Apr 15, 2018

addaleax Apr 27, 2018

BridgeAR commented Apr 23, 2018

BridgeAR left a comment

addaleax commented Apr 27, 2018

addaleax commented Apr 27, 2018

refack Apr 27, 2018

refack Apr 27, 2018

addaleax Apr 27, 2018

addaleax Apr 27, 2018

refack Apr 27, 2018

refack Apr 27, 2018

addaleax Apr 27, 2018

addaleax commented Apr 27, 2018

apapirovski commented Apr 28, 2018 •

edited

Loading

apapirovski commented Apr 28, 2018

benchmark: add bench for zlib gzip + gunzip cycle #20034

benchmark: add bench for zlib gzip + gunzip cycle #20034

Conversation

addaleax commented Apr 14, 2018

Checklist

lpinca Apr 15, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lpinca Apr 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lpinca Apr 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented Apr 23, 2018

BridgeAR left a comment

Choose a reason for hiding this comment

addaleax commented Apr 27, 2018

addaleax commented Apr 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

addaleax commented Apr 27, 2018

apapirovski commented Apr 28, 2018 • edited Loading

apapirovski commented Apr 28, 2018

lpinca Apr 15, 2018 •

edited

Loading

lpinca Apr 22, 2018 •

edited

Loading

lpinca Apr 22, 2018 •

edited

Loading

apapirovski commented Apr 28, 2018 •

edited

Loading