Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update README: The sentence "up to 160 million operations per second" #2

Closed
maxpert opened this issue Aug 17, 2018 · 4 comments
Closed

Comments

@maxpert
Copy link

maxpert commented Aug 17, 2018

It would have made more sense to compare it with RocksDB, LevelDB or similar on same environment and configurations. Putting out plain numbers sets no context, just to quote an example, MySQL Cluster has been benched at 200m ops/s (where operations are part of 2PC transactions!). And that in 2015! https://www.mysql.com/why-mysql/benchmarks/mysql-cluster/

@kierenj
Copy link

kierenj commented Aug 18, 2018

That's on 32 data nodes, vs this one CPU I believe?

@jahunter-m
Copy link
Contributor

The FASTER paper shows numbers for FASTER, RocksDB, etc. on the same hardware. If the paper isn't already linked from the README, it might be good to link it.

@badrishc
Copy link
Contributor

Yes, the MySQL number looks like it was on 32 nodes. This result is on a single machine. Of course, comparing to MySQL would be unfair as those are no txns here. The specs of the machine and the parameters for the experiments are discussed in https://www.microsoft.com/en-us/research/uploads/prod/2018/03/faster-sigmod18.pdf. I agree that the README needs to be explicit about the machine config, workload details, etc. Perhaps we should add a separate section covering these points.

@maxpert
Copy link
Author

maxpert commented Aug 18, 2018

Would be really nice if you can put in a benchmark wiki with table upfront so people don't have to digg in. I went through paper and found the benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants