Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update benchmark project #112

Merged
merged 6 commits into from
Mar 31, 2020
Merged

Update benchmark project #112

merged 6 commits into from
Mar 31, 2020

Conversation

atais
Copy link
Contributor

@atais atais commented Mar 26, 2020

I need to run some benchmarks regarding the lib and I thought using the existing project could be useful. However, it's not easy to run now, because there is no build tool attached.

This branch will evolve into updating the jars and comparing versions.
For some reason, GitHub treats moved files as +/-

Roadmap:

  • added maven and removed most of the jar files
  • using docker as default memcache instance
  • using SLF4J for logging
  • saving bench results to csv file
  • HTML report can generate graphs from CSV, no processing with java.
  • rerun old tests for comparison
  • update Xmemcached
  • add new versions test results into report

Worth doing:

  • review the benchmarking method
  • update spymemcached
  • update Memcached-Java-Client

@atais atais changed the title (chore) benchmark project is now maven based Update benchmark project Mar 26, 2020
@killme2008
Copy link
Owner

Good job, thanks.

@atais atais force-pushed the perf branch 7 times, most recently from 88a07a7 to e3d71f5 Compare March 27, 2020 09:22
@atais
Copy link
Contributor Author

atais commented Mar 27, 2020

OK, so I am halfway there.

What worries me greatly is that on my machine I got completely different results than you did 😱 It might be because I am running it all on localhost...

Also looking at my hit rate statistic I think the test is not running correctly. I am getting less than 10% hit rate and you did 100%. So something is wrong here. Any clues?

I am now planning to improve the logging and graphing so it would be easier to repeat the tests and post updated results.

@atais atais force-pushed the perf branch 4 times, most recently from e812ae3 to 500db56 Compare March 30, 2020 14:28
@atais
Copy link
Contributor Author

atais commented Mar 31, 2020

OK I think I have done as much as I could here.

@atais
Copy link
Contributor Author

atais commented Mar 31, 2020

@killme2008 any comments or issues?

@atais
Copy link
Contributor Author

atais commented Mar 31, 2020

@killme2008 CI is still not triggered here, so I guess you need to work on the project configuration

it worked on my fork: https://github.com/atais/xmemcached/pulls?q=is%3Apr+is%3Aclosed

@killme2008
Copy link
Owner

@atais Thanks a lot, a really good job.

@killme2008 killme2008 merged commit ab7fe21 into killme2008:master Mar 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants