Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coprocessor: split ranges if the slice is too big #7454

Merged
merged 4 commits into from Aug 23, 2018

Conversation

@winoros
Copy link
Member

commented Aug 21, 2018

What problem does this PR solve?

This PR aims to solve the issue that the TiKV timeout error is caused by the too large range in one request.

What is changed and how it works?

According to @coocood 's advice, split the range in one region when building copTask. So we won't send a message that is too large.

Check List

Tests

  • Existing tests.
  • Manual test, select count(*) from t where key in (800k constant list).
cmdType: cmdType,
})
rLen := ranges.len()
for i := 0; i < rLen; {

This comment has been minimized.

Copy link
@coocood

coocood Aug 22, 2018

Member

Need to add comments about why slice the task by range.

@coocood

This comment has been minimized.

Copy link
Member

commented Aug 22, 2018

/run-all-tests

@@ -235,6 +236,8 @@ func (r *copRanges) split(key []byte) (*copRanges, *copRanges) {
return r.slice(0, n), r.slice(n, r.len())
}

const rangesPerTask = 25000

This comment has been minimized.

Copy link
@zz-jason

zz-jason Aug 22, 2018

Member

Better to add some comment about what this value is used to do, why need it, why set it to this value.

respChan: make(chan *copResponse, 1),
cmdType: cmdType,
})
// etcd will return error if the message is too large. So we need to limit the length of the ranges slice

This comment has been minimized.

Copy link
@coocood

coocood Aug 22, 2018

Member

tikv will return gRPC error

@winoros winoros force-pushed the winoros:split-range branch from 34a48a0 to 5a08d3b Aug 22, 2018
@winoros winoros force-pushed the winoros:split-range branch from 5a08d3b to bdb5fda Aug 22, 2018
Copy link
Member

left a comment

LGTM

@coocood

This comment has been minimized.

Copy link
Member

commented Aug 22, 2018

LGTM

@coocood coocood added status/LGT2 and removed status/LGT1 labels Aug 22, 2018
@coocood coocood merged commit ef6590e into pingcap:master Aug 23, 2018
4 checks passed
4 checks passed
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@winoros winoros deleted the winoros:split-range branch Sep 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.