Stop applying writes when RocksDB requires delay #1300
Comments
yugabyte-ci
pushed a commit
that referenced
this issue
May 14, 2019
Summary:
When there are too many SSTable files, RocksDB would delay or stop writes.
But we are trying to apply them while the replica state update lock is taken.
And this lock would not be released for a long time, causing multiple threads to get stuck on it.
Added ability to stop applying committed entries on the follower when a delay is required by RocksDB.
Also, the following minor fixes are being implemented:
1) Release lock while waiting to records to be written to the log, before sending a reply to UpdateConsensus.
2) Move flags that configures RocksDB options from `src/yb/rocksdb/util/options.cc` to `src/yb/docdb/docdb_rocksdb_util.cc`,
since most of such flags already reside there.
3) Fix LongOperationTracker. Reference to queue top could be invalidated during condition wait.
4) Change incorrect op id comparisons to index comparison. We should not be using OpId comparisons to decide whether a record has been committed. Consider the sequence of committed operations: 1.1, 1.2, 2.3. Based on OpId comparisons we would also consider a "1.3" record (perhaps from another replica's log) committed, which would be incorrect.
5) Update thread annotations from boost.
Test Plan: ybd --gtest_filter QLStressTest.DelayWrite
Reviewers: hector, amitanand, mikhail
Reviewed By: mikhail
Subscribers: rao, kannan, bogdan, ybase, bharat
Differential Revision: https://phabricator.dev.yugabyte.com/D6476
yugabyte-ci
pushed a commit
that referenced
this issue
May 15, 2019
Summary: During build, we specify `-DSNAPPY` only while building rocksdb. Because of it `Snappy_Supported` returns `false` when used outside RocksDB, because it is inline function. So `InitRocksDBOptions` is unable to correctly initialize `compression` option and compression is disabled, even when it is available. Before D6476 we initialized compression in options.cc, so this issue did not appear. This diff fixes the issue, by moving `-DSNAPPY` and other flags to root CMakeLists.txt. Test Plan: ybd release --cxx-test redisserver_redisserver-test --gtest_filter TestRedisService.HugeCommandInline Reviewers: timur, mikhail Reviewed By: mikhail Subscribers: kannan, ybase, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D6621
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When there are too many SSTable files, RocksDB would delay or stop writes.
But we are trying to apply them while the replicate state update lock is taken.
And this lock would not be released for a long time, causing multiple threads to get stuck on it.
The text was updated successfully, but these errors were encountered: