Skip to content
GitHub no longer supports this web browser. Learn more about the browsers we support.
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: Support both Prost and rust-protobuf libraries #5379

Merged
merged 4 commits into from Nov 27, 2019
Merged

Conversation

@nrc
Copy link
Contributor

nrc commented Sep 2, 2019

To use Prost, set the PROST env var, e.g.,: PROST=1 make dev. If using Cargo,
use --no-default-features --features prost-codec.

The most notable change is threading the prost-codec/protobuf-codec through the
Cargo.tomls of all crates. In addition, in order to make this work I had to move
integraton tests and benchmarks into their own crate (tests). This is because
Cargo features do not interact perfectly with dev-dependencies.

We're using a Git dep for Prost in order to get some optimisations which are on
master, but not in the latest release. We can change to a crates.io dep when there
is another release.

We must allow the identity_conversion lint because there are some conversions
which are meaningful with rust-protobuf, but no-ops with Prost.

The changes to src/coprocessor/endpoint.rs are because Prost does not permit
setting a custom recursion limit. We only did this for tests previously. We
now use the default recursion limit all the time for both codecs; the test must
be adjusted so that we hit the higher limit.

PTAL @breeswish @BusyJay @overvenus

What have you changed?

Add a feature flag for Prost which builds TiKV and its deps using Prost rather than rust-protobuf as the protobuf codec.

What is the type of the changes?

  • Engineering (engineering change which doesn't change any feature or fix any issue)

How is the PR tested?

make dev

Does this PR affect documentation (docs) or should it be mentioned in the release notes?

Should have dev docs (to come) and mentioned in release docs.

Does this PR affect tidb-ansible?

No

Refer to a related PR or issue link (optional)

#2452

@nrc nrc requested review from BusyJay, breeswish and overvenus Sep 2, 2019
@nrc nrc force-pushed the nrc:prost-3 branch from ae5a11c to e46905f Sep 2, 2019
@@ -1,5 +1,8 @@
// Copyright 2019 TiKV Project Authors. Licensed under Apache-2.0.

#[cfg(feature = "prost-codec")]
use prost;
use protobuf::ProtobufError;

This comment has been minimized.

Copy link
@siddontang

siddontang Sep 2, 2019

Contributor

em, do we need to use protobuf feature here?

seem if we pass prost-codec feature, we still need to import protobuf?

This comment has been minimized.

Copy link
@nrc

nrc Sep 2, 2019

Author Contributor

I use the protobuf::Message trait (and supporting error types) to support both Prost and rust-proto, so we still need the protobuf imports in the Prost case.

This comment has been minimized.

Copy link
@siddontang

siddontang Sep 3, 2019

Contributor

got it

failpoints = ["fail/failpoints"]
prost-codec = [

This comment has been minimized.

Copy link
@siddontang

siddontang Sep 3, 2019

Contributor

em, I see nearly all the workspaces use these, seem too verbose, but I also find no way to simply them.

This comment has been minimized.

Copy link
@BusyJay

BusyJay Oct 16, 2019

Contributor

Since features are unions, enabling one should be sufficient for all dependencies.

Cargo.toml Outdated
@@ -96,8 +89,8 @@ zipf = "5.0.1"
bitflags = "1.0.1"
fail = "0.3"
uuid = { version = "0.6", features = [ "serde", "v4" ] }
grpcio = { version = "0.5.0-alpha.3", features = [ "openssl-vendored" ] }
raft = "0.6.0-alpha"
grpcio = { version = "0.5.0-alpha.3", features = [ "openssl-vendored" ], default-features = false }

This comment has been minimized.

Copy link
@siddontang

siddontang Sep 3, 2019

Contributor

why disable default features here and following deps?

This comment has been minimized.

Copy link
@nrc

nrc Sep 3, 2019

Author Contributor

Because when we use grpc-rs etc., we must use either prost-codec or protobuf-codec (which is the default), but not both.

This comment has been minimized.

Copy link
@siddontang

siddontang Sep 4, 2019

Contributor

got it

@siddontang

This comment has been minimized.

Copy link
Contributor

siddontang commented Sep 4, 2019

@zhouqiang-cl

How to bench this PR with building TiKV "PROST=1"?

@siddontang

This comment has been minimized.

Copy link
Contributor

siddontang commented Sep 4, 2019

@mahjonp

This comment has been minimized.

Copy link
Contributor

mahjonp commented Sep 6, 2019

Hi @nrc, can we use PROST=1 make dist_release to release with using Prost? And if yes, it's easy to bench this PR.

@mahjonp

This comment has been minimized.

Copy link
Contributor

mahjonp commented Sep 6, 2019

/release PROST=1 make dist_release

@mahjonp

This comment has been minimized.

Copy link
Contributor

mahjonp commented Sep 6, 2019

/bench

@mahjonp

This comment has been minimized.

Copy link
Contributor

mahjonp commented Sep 6, 2019

@nrc Oops, benchmark failed to start a TiDB cluster using this commit, there were some problem.

@siddontang

This comment has been minimized.

Copy link
Contributor

siddontang commented Sep 8, 2019

friendly ping @nrc

@zhouqiang-cl

This comment has been minimized.

Copy link
Member

zhouqiang-cl commented Sep 8, 2019

/release PROST=1 make dist_release

@nrc nrc added the C: gRPC label Sep 9, 2019
@nrc nrc force-pushed the nrc:prost-3 branch 3 times, most recently from 12239b2 to 7b25341 Sep 9, 2019
@nrc

This comment has been minimized.

Copy link
Contributor Author

nrc commented Sep 10, 2019

/bench

@siddontang

This comment has been minimized.

Copy link
Contributor

siddontang commented Sep 10, 2019

CI failed with

image

PTAL @breeswish

@nrc nrc force-pushed the nrc:prost-3 branch from 7b25341 to b294b71 Sep 10, 2019
@nrc

This comment has been minimized.

Copy link
Contributor Author

nrc commented Sep 10, 2019

/bench

@mahjonp

This comment has been minimized.

Copy link
Contributor

mahjonp commented Sep 10, 2019

/release PROST=1 make dist_release

@mahjonp

This comment has been minimized.

Copy link
Contributor

mahjonp commented Sep 10, 2019

/test

@tikv tikv deleted a comment from zhouqiang-cl Sep 10, 2019
@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Sep 10, 2019

@@                               Benchmark Diff                               @@
================================================================================
tidb: 0f2434660c509ec0b493d8c7ee0bbbf8e8549184
--- tikv: ff82aa9eba331585aec1c6cdf9e1584512bccb34
+++ tikv: b294b71f9a4afd4ab1c47c2c452048d511954224
pd: c7c572a7dc9710aca57cb0bddf4d9bca6c4a111b
================================================================================
test-1: < oltp_insert >
    * QPS : 20840.65 ± 0.5553% (std=70.69) delta: -1.91%
    * AvgMs : 12.28 ± 0.5376% (std=0.04) delta: 1.93%
    * PercentileMs99 : 42.92 ± 1.0903% (std=0.38) delta: 0.73%
            
test-2: < oltp_update_non_index >
    * QPS : 29243.96 ± 0.1096% (std=20.95) delta: -0.69%
    * AvgMs : 8.75 ± 0.1143% (std=0.01) delta: 0.71%
    * PercentileMs99 : 30.70 ± 2.1758% (std=0.41) delta: -0.35%
            
test-3: < oltp_read_write >
    * QPS : 36624.11 ± 0.2892% (std=63.44) delta: -0.78%
    * AvgMs : 140.37 ± 0.2821% (std=0.25) delta: 0.79%
    * PercentileMs99 : 262.64 ± 0.0000% (std=0.00) delta: 1.82%
            
test-4: < oltp_point_select >
    * QPS : 73650.82 ± 0.2890% (std=134.00) delta: -0.62%
    * AvgMs : 3.47 ± 1.0387% (std=0.02) delta: 0.35%
    * PercentileMs99 : 7.48 ± 1.0425% (std=0.06) delta: 0.00%
            
test-5: < oltp_update_index >
    * QPS : 16519.31 ± 0.1599% (std=17.75) delta: -2.55%
    * AvgMs : 15.50 ± 0.1613% (std=0.02) delta: 2.62%
    * PercentileMs99 : 49.21 ± 0.0000% (std=0.00) delta: 1.80%
            

https://perf.pingcap.com

@nrc nrc force-pushed the nrc:prost-3 branch 3 times, most recently from 77c4deb to 3178b40 Nov 18, 2019
Copy link
Member

breeswish left a comment

This PR frequently facing conflicts. Approve it so that it can be merged sooner.

@Hoverbear Hoverbear requested a review from BusyJay Nov 26, 2019
@nrc nrc force-pushed the nrc:prost-3 branch from 3178b40 to e2d9d8d Nov 27, 2019
To use Prost, set the `PROST` env var, e.g.,: `PROST=1 make dev`. If using Cargo,
use `--no-default-features --features prost-codec`.

The most notable change is threading the prost-codec/protobuf-codec through the
Cargo.tomls of all crates. In addition, in order to make this work I had to move
integraton tests and benchmarks into their own crate (`tests`). This is because
Cargo features do not interact perfectly with dev-dependencies.

We're using a Git dep for Prost in order to get some optimisations which are on
master, but not in the latest release. We can change to a crates.io dep when there
is another release.

We must allow the `identity_conversion` lint because there are some conversions
which are meaningful with rust-protobuf, but no-ops with Prost.

The changes to src/coprocessor/endpoint.rs are because Prost does not permit
setting a custom recursion limit. We only did this for tests previously. We
now use the default recursion limit all the time for both codecs; the test must
be adjusted so that we hit the higher limit.

Signed-off-by: Nick Cameron <nrc@ncameron.org>
@nrc nrc force-pushed the nrc:prost-3 branch from e2d9d8d to b11c48d Nov 27, 2019
nrc added 2 commits Nov 27, 2019
Copy link
Contributor

overvenus left a comment

:shipit:

@nrc

This comment has been minimized.

Copy link
Contributor Author

nrc commented Nov 27, 2019

/merge

@sre-bot sre-bot added the S: CanMerge label Nov 27, 2019
@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 27, 2019

/run-all-tests

@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 27, 2019

@nrc merge failed.

@nrc

This comment has been minimized.

Copy link
Contributor Author

nrc commented Nov 27, 2019

/merge

@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 27, 2019

Your auto merge job has been accepted, waiting for 5964

@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 27, 2019

/run-all-tests

@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 27, 2019

@nrc merge failed.

@nrc

This comment has been minimized.

Copy link
Contributor Author

nrc commented Nov 27, 2019

/merge

@sre-bot

This comment has been minimized.

Copy link
Collaborator

sre-bot commented Nov 27, 2019

/run-all-tests

@sre-bot sre-bot merged commit 858c765 into tikv:master Nov 27, 2019
6 checks passed
6 checks passed
DCO All commits are signed off!
Details
idc-jenkins-ci-tikv/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-copr-test Jenkins job succeeded.
Details
idc-jenkins-ci-tikv/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details
hawkingrei added a commit to hawkingrei/tikv that referenced this pull request Dec 1, 2019
Signed-off-by: Nick Cameron <nrc@ncameron.org>
zhang555 added a commit to zhang555/tikv that referenced this pull request Dec 16, 2019
Signed-off-by: Nick Cameron <nrc@ncameron.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.