Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve simple map benchmark #467

Merged
merged 1 commit into from
Mar 12, 2019

Conversation

puzpuzpuz
Copy link
Contributor

@puzpuzpuz puzpuzpuz commented Mar 7, 2019

  • Replaces existing SimpleMapBenchmark.js benchmark with two simple parallel ops benchmark.
  • Generic benchmark logic was moved into a common module.
  • The common benchmark module runs multiple operations in parallel. Thus it loads the client and the server more effectively.
  • A couple of runner scripts were introduced.
  • MapRandomOpRunner runner is using the original random operation generator.
  • MapPutRunner runner removes randomness factor and dependency on server-side from the benchmark: all operations now are map.put('foo', 'bar').
  • Also includes a readme to simplify running benchmarks.
  • Tested on node 4 & 10.

Old benchmark showed about 16-17K ops/sec on my laptop, while the new one (namely MapPutRunner) shows about 50-60K ops/sec.

@mustafaiman
Copy link
Contributor

I liked the new benchmark using the client to its fullest performance. Thanks for noticing the old benchmark was self throttling.

About this new one, is it possible to re-introduce the parameterized put-get-remove options? We can set PUT_PERCENTAGE to 100 to consistently test put throughput. Also, I think having a key space and some randomness makes more sense because we would like to test it against a cluster of multiple instances. Having a single key makes the client use only a single instance. Even with a single instance scenario, we are limiting hazelcast member to utilize only a single partition. Therefore, we remove parallelization from member by using a single key.

Also, I would like to have benchmark code compatible with node.js 4 as long as we support it.

@puzpuzpuz
Copy link
Contributor Author

@mustafaiman
I could extract generic benchmark logic into a common module and introduce a couple of runner scripts. The first one would be using the original random operation generator, while the second one would be doing map.put('foo', 'bar').
WDYT?

As for the node 4, I'll downgrade syntax to support it.

@mustafaiman
Copy link
Contributor

@puzpuzpuz sounds awesome

@puzpuzpuz puzpuzpuz force-pushed the enhancement/improve-benchmark branch from 4b9e3ef to c179115 Compare March 7, 2019 18:04
Also added a readme for the benchmark
@puzpuzpuz puzpuzpuz force-pushed the enhancement/improve-benchmark branch from c179115 to fb4c451 Compare March 7, 2019 18:13
@puzpuzpuz
Copy link
Contributor Author

@mustafaiman
I've updated the code according to our discussions. Could review it once again?

@puzpuzpuz puzpuzpuz merged commit d322ed0 into hazelcast:master Mar 12, 2019
@puzpuzpuz puzpuzpuz deleted the enhancement/improve-benchmark branch March 12, 2019 15:01
harunalpak pushed a commit to harunalpak/hazelcast-nodejs-client that referenced this pull request Dec 8, 2022
Also added a readme for the benchmark
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants