-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optionally construct Post Processing Info map in MemTableInserter #2011
Conversation
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Thanks! |
@IslamAbdelRahman unique_ptr is a pointer, so the map itself needs an extra allocation. |
@IslamAbdelRahman @siying is correct. We'd need an extra allocation for map itself. Preserving space reserved for map inside the inserter gives us the same cost-less stack allocation. |
@siying I do not know what's wrong with Travis |
@yuslepukhin what we can do? |
@yuslepukhin failing for our internal CI too:
|
@siying I have included <type_traits> for std::std::aligned_storage_t, Will post an update |
std::map. However, on Windows with 2015 STL the default constructed map still dynamically allocates one node which shows up on a profiler and we loose ~40% throughput on fillrandom benchmark. Solution: declare a map as std::aligned storage and optionally construct. Results on the same commit: Before: ************************ BulkLoadRandom_10M_fillonly ************************ Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillonly] fillrandom : 2.775 micros/op 360334 ops/sec; 280.4 MB/s Microseconds per write: Count: 10000000 Average: 2.7749 StdDev: 39.92 Min: 1 Median: 2.0826 Max: 26051 Percentiles: P50: 2.08 P75: 2.55 P99: 3.55 P99.9: 9.58 P99.99: 51.56 ------------------------------------------------------ After: ************************ BulkLoadRandom_10M_fillonly ************************ Initializing RocksDB Options from command-line flags DB path: [k:\data\BulkLoadRandom_10M_fillonly] fillrandom : 1.794 micros/op 557284 ops/sec; 433.7 MB/s Microseconds per write: Count: 10000000 Average: 1.7940 StdDev: 45.46 Min: 0 Median: 1.0812 Max: 23620 Percentiles: P50: 1.08 P75: 1.57 P99: 2.81 P99.9: 5.39 P99.99: 17.36 ------------------------------------------------------
495f602
to
03ea01f
Compare
@yuslepukhin updated the pull request - view changes - changes since last import |
@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
MemTableInserter default constructs Post processing info
std::map. However, on Windows with 2015 STL the default
constructed map still dynamically allocates one node
which shows up on a profiler and we loose ~40% throughput
on fillrandom benchmark.
Solution: declare a map as std::aligned storage and optionally
construct.
This addresses #1976
Before:
Initializing RocksDB Options from command-line flags
DB path: [k:\data\BulkLoadRandom_10M_fillonly]
fillrandom : 2.775 micros/op 360334 ops/sec; 280.4 MB/s
Microseconds per write:
Count: 10000000 Average: 2.7749 StdDev: 39.92
Min: 1 Median: 2.0826 Max: 26051
Percentiles: P50: 2.08 P75: 2.55 P99: 3.55 P99.9: 9.58 P99.99: 51.5**6
After:
Initializing RocksDB Options from command-line flags
DB path: [k:\data\BulkLoadRandom_10M_fillonly]
fillrandom : 1.794 micros/op 557284 ops/sec; 433.7 MB/s
Microseconds per write:
Count: 10000000 Average: 1.7940 StdDev: 45.46
Min: 0 Median: 1.0812 Max: 23620
Percentiles: P50: 1.08 P75: 1.57 P99: 2.81 P99.9: 5.39 P99.99: 17.36