follower read with applied index #32
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: qupeng <qupeng@pingcap.com>
|
PTAL @sunxiaoguang @lysu thanks! |
5kbpers
reviewed
Nov 14, 2019
Signed-off-by: qupeng <qupeng@pingcap.com>
Hoverbear
added
the
Initial Comment Period
9 tasks
dcalvin
reviewed
Nov 26, 2019
text/2019-11-12-read-with-index.md
Outdated
| This RFC proposes a improvement about getting snapshot on followers, which is | ||
| if the read request carries an applied index, the peer can get snapshot locally | ||
| wihtout any commucation with its leader, if only it has applied to the given | ||
| index. It's useful to reduce latency if the cluster is deployed in multi |
There was a problem hiding this comment.
Suggested change
| index. It's useful to reduce latency if the cluster is deployed in multi | |
| index. It's useful to reduce latency if the cluster is deployed across multiple |
text/2019-11-12-read-with-index.md
Outdated
|
|
||
| ## Motivation | ||
|
|
||
| For clusters deployed in multi datacenters, the system latency could mainly |
There was a problem hiding this comment.
Suggested change
| For clusters deployed in multi datacenters, the system latency could mainly | |
| For clusters deployed across multiple data centers, system latency mainly |
text/2019-11-12-read-with-index.md
Outdated
| ## Motivation | ||
|
|
||
| For clusters deployed in multi datacenters, the system latency could mainly | ||
| depend on the network RTT between datacenters. For example, suppose PDs are |
There was a problem hiding this comment.
Suggested change
| depend on the network RTT between datacenters. For example, suppose PDs are | |
| depends on the network round-trip time (RTT) between data centers. For example, suppose PDs are |
text/2019-11-12-read-with-index.md
Outdated
|
|
||
| The implementation of `GetTsAndReadIndex` will get a timestamp from PD first, | ||
| and then call `ReadIndex` of service `Tikv` to get applied indices of all | ||
| regions which is carried in `Request`. If any retryable error occurs, the proxy |
There was a problem hiding this comment.
Suggested change
| regions which is carried in `Request`. If any retryable error occurs, the proxy | |
| Regions that are carried in `Request`. If any retryable error occurs, the proxy |
text/2019-11-12-read-with-index.md
Outdated
| } | ||
| ``` | ||
|
|
||
| A question is how clients can knows `applied_index` for every region? The |
There was a problem hiding this comment.
Suggested change
| A question is how clients can knows `applied_index` for every region? The | |
| A question is how clients can know `applied_index` for every Region? The |
text/2019-11-12-read-with-index.md
Outdated
| ``` | ||
|
|
||
| A question is how clients can knows `applied_index` for every region? The | ||
| answer is RPC `ReadIndex` in service `Tikv`. It's already ready, so we can |
There was a problem hiding this comment.
Suggested change
| answer is RPC `ReadIndex` in service `Tikv`. It's already ready, so we can | |
| answer is RPC `ReadIndex` in service `Tikv`. It's already implemented, so we can |
text/2019-11-12-read-with-index.md
Outdated
| ### Proxy | ||
|
|
||
| As above described, we need to add a proxy service in the major datacenter. | ||
| Considering the proxy service is better to be high available, we can put it |
There was a problem hiding this comment.
Suggested change
| Considering the proxy service is better to be high available, we can put it | |
| Considering that the proxy service is better to be highly available, we can put it |
|
@hicqu PTAL |
Signed-off-by: qupeng <qupeng@pingcap.com>
dcalvin
reviewed
Nov 28, 2019
text/2019-11-12-read-with-index.md
Outdated
|
|
||
| This RFC proposes an enhancement of getting snapshots on followers. The idea is | ||
| if the read request carries an applied index, the peer can get snapshot locally | ||
| wihtout any communication with its leader as long as only only it has applied |
There was a problem hiding this comment.
Suggested change
| wihtout any communication with its leader as long as only only it has applied | |
| without any communication with its leader as long as only it has applied |
| This RFC proposes an enhancement of getting snapshots on followers. The idea is | ||
| if the read request carries an applied index, the peer can get snapshot locally | ||
| wihtout any communication with its leader as long as only only it has applied | ||
| to the given index. It's useful to reduce latency if the cluster is deployed |
There was a problem hiding this comment.
Suggested change
| to the given index. It's useful to reduce latency if the cluster is deployed | |
| apply raft logs to the given index. It's useful to reduce latency if the cluster is deployed |
dcalvin
reviewed
Nov 28, 2019
text/2019-11-12-read-with-index.md
Outdated
| } | ||
| ``` | ||
|
|
||
| So TiKV can get a snapshot directly after it applys to `applied_index`. |
There was a problem hiding this comment.
With this implementation, TiKV can apply raft logs to the given index applied_index.
dcalvin
reviewed
Nov 28, 2019
text/2019-11-12-read-with-index.md
Outdated
| } | ||
| ``` | ||
|
|
||
| So TiKV can get a snapshot directly after it applys to `applied_index`. |
There was a problem hiding this comment.
Suggested change
| So TiKV can get a snapshot directly after it applys to `applied_index`. | |
| With this implementation, TiKV can apply raft logs to the given index applied_index. |
dcalvin
reviewed
Nov 28, 2019
Signed-off-by: qupeng <qupeng@pingcap.com>
|
PTAL @dcalvin thanks! |
|
@hicqu is there a tracking issue for this feature? I'd like to know how this feature get down. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support follower read with applied index. So that we don't need to get region's applied index again.
Signed-off-by: qupeng qupeng@pingcap.com