perf: does cockroach have some benchmark tool?

[xiaoyulei]

Does cockoach have some benchmark function or we can use some other benchmark tool to test it?

[petermattis]

Can you provide a little more context about what you'd like to test? Are you trying to test performance? Correctness?

For performance, there are micro-benchmarks sprinkled in the *_test.go files and some larger scale benchmark tests as well. We also have a few example apps in the examples directory (shortly moving to the examples-go repo) which simulate particular application load. I would not classify any of these benchmarks as extensive. We haven't done any work comparing Cockroach on "standard" benchmarks, though eventually we'd like to. See #2309. Right now, correctness, stability and functionality, not performance, are the primary concerns.

For correctness, Cockroach has an extensive suite of unit tests.

[xiaoyulei]

I am trying to test performance. I see the *_test.go files contains some benchmark tests case. But it is a little simple.

[petermattis]

Yes, the benchmarks in the *_test.go files are simple. I wasn't trying to claim otherwise. We haven't turned our attention to benchmarking or performance tuning yet.

Do you have a specific application that you're looking to test the performance of? Or a known type of application load? I'm interested in understanding what type of benchmark numbers you'd find interesting. Or what type of application you're considering running on Cockroach.

[xiaoyulei]

I wish to do TPCC test on it.

[petermattis]

I'd love to see a TPC-C test performed on CockroachDB. And I'd love for it to be constructed in a way that we can run the test regularly as the software develops.

[kant111]

Besides the read and write performance I would also like to see what is performance of joins once the tuples/rows are distributed across nodes. I believe that would be a very interesting metric to compare with other relational distributed systems out there.

[spencerkimball]

Benchmarking tools are taking shape in https://github.com/cockroachdb/loadgen. There's a generic KV, YCSB, and TPC-H. TPC-C isn't added yet, but likely will be in the near future. Closing this for now as an issue against the main repo.