Core-local statistics #2258

Closed
wants to merge 5 commits into
from

Conversation

Projects
None yet
3 participants
@ajkr
Contributor

ajkr commented May 6, 2017

This diff changes StatisticsImpl from a thread-local approach to a core-local one. The goal is to perform faster aggregations, particularly for applications that have many threads. There should be no behavior change.

Test Plan:

  • sysbench with 4k concurrent transaction threads: stats aggregations 6.3x faster compared to thread-local (thread-local: 28ms, core-local: 4.4ms)
  • db_bench readwhilewriting, 32 threads, 100M keys, on SSD: stats updates 1.2x slower compared to thread-local
thread-local:

readwhilewriting : 3.072 micros/op 325566 ops/sec; 22.8 MB/s (631365 of 1000000 found)

+ 0.96% 0.11% db_bench.master db_bench.master [.] rocksdb::StatisticsImpl::measureTime
+ 0.89% 0.32% db_bench.master db_bench.master [.] rocksdb::StatisticsImpl::recordTick

core-local:

readwhilewriting : 3.050 micros/op 327826 ops/sec; 22.9 MB/s (631365 of 1000000 found)

+ 1.14% 0.38% db_bench.core-l db_bench.core-local.getcpu.bitshift [.] rocksdb::StatisticsImpl::recordTick
+ 1.09% 0.17% db_bench.core-l db_bench.core-local.getcpu.bitshift [.] rocksdb::StatisticsImpl::measureTime
  • db_bench readrandom, 32 threads, 100M keys, in-memory: stats updates 1.2x slower compared to thread-local
thread-local:

+ 0.88% 0.07% db_bench.master db_bench.master [.] rocksdb::StatisticsImpl::measureTime
0.55% 0.18% db_bench.master db_bench.master [.] rocksdb::StatisticsImpl::recordTick

readrandom : 2.049 micros/op 487965 ops/sec; 34.7 MB/s (1000000 of 1000000 found)

core-local:

+ 0.97% 0.13% db_bench.core-l db_bench.core-local.getcpu.bitshift [.] rocksdb::StatisticsImpl::measureTime
+ 0.74% 0.24% db_bench.core-l db_bench.core-local.getcpu.bitshift [.] rocksdb::StatisticsImpl::recordTick

readrandom : 2.086 micros/op 479325 ops/sec; 34.1 MB/s (1000000 of 1000000 found)
@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot May 6, 2017

@ajkr updated the pull request - view changes

@ajkr updated the pull request - view changes

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot May 6, 2017

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot May 6, 2017

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr ajkr referenced this pull request in facebook/mysql-5.6 May 8, 2017

Open

Make SHOW GLOBAL STATUS faster for MyRocks #596

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot May 12, 2017

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@siying

This comment has been minimized.

Show comment
Hide comment
@siying

siying May 12, 2017

Contributor

CLANG Analyze failure with:

internal_repo_rocksdb/repo/monitoring/statistics.h:71:9: error: no matching function for call to 'atomic_init'
std::atomic_init(&tickers_[i], static_cast<uint64_t>(0));
^~~~~~~~~~~~~~~~

Contributor

siying commented May 12, 2017

CLANG Analyze failure with:

internal_repo_rocksdb/repo/monitoring/statistics.h:71:9: error: no matching function for call to 'atomic_init'
std::atomic_init(&tickers_[i], static_cast<uint64_t>(0));
^~~~~~~~~~~~~~~~

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@ajkr

This comment has been minimized.

Show comment
Hide comment
@ajkr

ajkr May 12, 2017

Contributor

Thanks, @siying, I was confused about the array initialization but fixed it now.

Contributor

ajkr commented May 12, 2017

Thanks, @siying, I was confused about the array initialization but fixed it now.

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot May 12, 2017

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr

This comment has been minimized.

Show comment
Hide comment
@ajkr

ajkr May 22, 2017

Contributor

can somebody review in time for the June release?

Contributor

ajkr commented May 22, 2017

can somebody review in time for the June release?

@siying

siying approved these changes May 22, 2017

monitoring/statistics.h
+ __declspec(align(64)) struct StatisticsData {
+ std::atomic_uint_fast64_t tickers_[INTERNAL_TICKER_ENUM_MAX] = {{0}};
+ HistogramImpl histograms_[INTERNAL_HISTOGRAM_ENUM_MAX];
+ } __attribute__((__aligned__(64)));

This comment has been minimized.

@siying

siying May 22, 2017

Contributor

Please make sure it works in Windows. Looks like it is failing.

@siying

siying May 22, 2017

Contributor

Please make sure it works in Windows. Looks like it is failing.

@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

This comment has been minimized.

Show comment
Hide comment
@facebook-github-bot

facebook-github-bot May 23, 2017

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment