-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
follower read with applied index #32
Conversation
Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: qupeng <qupeng@pingcap.com>
PTAL @sunxiaoguang @lysu thanks! |
Signed-off-by: qupeng <qupeng@pingcap.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
text/2019-11-12-read-with-index.md
Outdated
This RFC proposes a improvement about getting snapshot on followers, which is | ||
if the read request carries an applied index, the peer can get snapshot locally | ||
wihtout any commucation with its leader, if only it has applied to the given | ||
index. It's useful to reduce latency if the cluster is deployed in multi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
index. It's useful to reduce latency if the cluster is deployed in multi | |
index. It's useful to reduce latency if the cluster is deployed across multiple |
text/2019-11-12-read-with-index.md
Outdated
|
||
## Motivation | ||
|
||
For clusters deployed in multi datacenters, the system latency could mainly |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For clusters deployed in multi datacenters, the system latency could mainly | |
For clusters deployed across multiple data centers, system latency mainly |
text/2019-11-12-read-with-index.md
Outdated
## Motivation | ||
|
||
For clusters deployed in multi datacenters, the system latency could mainly | ||
depend on the network RTT between datacenters. For example, suppose PDs are |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
depend on the network RTT between datacenters. For example, suppose PDs are | |
depends on the network round-trip time (RTT) between data centers. For example, suppose PDs are |
text/2019-11-12-read-with-index.md
Outdated
|
||
The implementation of `GetTsAndReadIndex` will get a timestamp from PD first, | ||
and then call `ReadIndex` of service `Tikv` to get applied indices of all | ||
regions which is carried in `Request`. If any retryable error occurs, the proxy |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
regions which is carried in `Request`. If any retryable error occurs, the proxy | |
Regions that are carried in `Request`. If any retryable error occurs, the proxy |
text/2019-11-12-read-with-index.md
Outdated
} | ||
``` | ||
|
||
A question is how clients can knows `applied_index` for every region? The |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A question is how clients can knows `applied_index` for every region? The | |
A question is how clients can know `applied_index` for every Region? The |
text/2019-11-12-read-with-index.md
Outdated
``` | ||
|
||
A question is how clients can knows `applied_index` for every region? The | ||
answer is RPC `ReadIndex` in service `Tikv`. It's already ready, so we can |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
answer is RPC `ReadIndex` in service `Tikv`. It's already ready, so we can | |
answer is RPC `ReadIndex` in service `Tikv`. It's already implemented, so we can |
text/2019-11-12-read-with-index.md
Outdated
### Proxy | ||
|
||
As above described, we need to add a proxy service in the major datacenter. | ||
Considering the proxy service is better to be high available, we can put it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering the proxy service is better to be high available, we can put it | |
Considering that the proxy service is better to be highly available, we can put it |
@hicqu PTAL |
Signed-off-by: qupeng <qupeng@pingcap.com>
text/2019-11-12-read-with-index.md
Outdated
|
||
This RFC proposes an enhancement of getting snapshots on followers. The idea is | ||
if the read request carries an applied index, the peer can get snapshot locally | ||
wihtout any communication with its leader as long as only only it has applied |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wihtout any communication with its leader as long as only only it has applied | |
without any communication with its leader as long as only it has applied |
This RFC proposes an enhancement of getting snapshots on followers. The idea is | ||
if the read request carries an applied index, the peer can get snapshot locally | ||
wihtout any communication with its leader as long as only only it has applied | ||
to the given index. It's useful to reduce latency if the cluster is deployed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to the given index. It's useful to reduce latency if the cluster is deployed | |
apply raft logs to the given index. It's useful to reduce latency if the cluster is deployed |
text/2019-11-12-read-with-index.md
Outdated
} | ||
``` | ||
|
||
So TiKV can get a snapshot directly after it applys to `applied_index`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this implementation, TiKV can apply raft logs to the given index applied_index.
text/2019-11-12-read-with-index.md
Outdated
} | ||
``` | ||
|
||
So TiKV can get a snapshot directly after it applys to `applied_index`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So TiKV can get a snapshot directly after it applys to `applied_index`. | |
With this implementation, TiKV can apply raft logs to the given index applied_index. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some changes as we talked about earlier. Rest LGTM
Signed-off-by: qupeng <qupeng@pingcap.com>
PTAL @dcalvin thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@hicqu is there a tracking issue for this feature? I'd like to know how this feature get down. |
Support follower read with applied index. So that we don't need to get region's applied index again.
Signed-off-by: qupeng qupeng@pingcap.com