Skip to content

Commit

Permalink
chore: Update meta-service gRPC version spec and raft-protocol featur…
Browse files Browse the repository at this point in the history 
…e set (#15398)

There is no change in this commit but just update the doc about what
features are provided by raft-API, and the doc about version spec of
gRPC-API.
  • Loading branch information
@drmingdrmer
drmingdrmer committed May 6, 2024
1 parent 51065f3 commit ddfca5d
Show file tree
Hide file tree
Showing 3 changed files with 63 additions and 57 deletions.
28 changes: 17 additions & 11 deletions src/meta/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,17 +75,23 @@ To add a new feature(add new type or update an type), the developer should do:

## Compatibility with Databend Query

The following is an illustration of current query-meta compatibility:

| `Meta\Query` | [0.7.59, 0.8.80) | [0.8.80, 0.9.41) | [0.9.41, 1.1.34) | [1.1.34, 1.2.287) | [1.2.287, +∞) |
|:------------------|:-----------------|:-----------------|:-----------------|:---------------|:-----------|
| [0.8.30, 0.8.35) | ✅ | ❌ | ❌ | ❌ | ❌ |
| [0.8.35, 0.9.23) | ✅ | ✅ | ✅ | ❌ | ❌ |
| [0.9.23, 0.9.42) | ❌ | ✅ | ✅ | ❌ | ❌ |
| [0.9.42, 1.1.32) | ❌ | ❌ | ✅ | ❌ | ❌ |
| [1.1.32, 1.2.63) | ❌ | ❌ | ✅ | ✅ | ❌ |
| [1.2.63, 1.2.226) | ❌ | ❌ | ✅ | ✅ | ❌ |
| [1.2.226, +∞) | ❌ | ❌ | ✅ | ✅ | ✅ |
The following is an illustration of the latest query-meta compatibility:

| `Meta\Query` | [0.9.41, 1.1.34) | [1.1.34, 1.2.287) | [1.2.287, 1.2.361) | [1.2.361, +∞) |
|:-------------------|:-----------------|:---------------|:-----------|:-----------|
| [0.8.30, 0.8.35) | ❌ | ❌ | ❌ |❌ |
| [0.8.35, 0.9.23) | ✅ | ❌ | ❌ |❌ |
| [0.9.23, 0.9.42) | ✅ | ❌ | ❌ |❌ |
| [0.9.42, 1.1.32) | ✅ | ❌ | ❌ |❌ |
| [1.1.32, 1.2.63) | ✅ | ✅ | ❌ |❌ |
| [1.2.63, 1.2.226) | ✅ | ✅ | ❌ |❌ |
| [1.2.226, 1.2.258) | ✅ | ✅ | ✅ |❌ |
| [1.2.258, +∞) | ✅ | ✅ | ✅ |✅ |

History versions that are not included in the above chart:

- Query `[0.7.59, 0.8.80)` is compatible with Meta `[0.8.30, 0.9.23)`.
- Query `[0.8.80, 0.9.41)` is compatible with Meta `[0.8.35, 0.9.42)`.


## Compatibility between databend-meta
Expand Down
65 changes: 29 additions & 36 deletions src/meta/client/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ pub use grpc_client::MetaChannelManager;
pub use grpc_client::MetaGrpcClient;
pub use message::ClientWorkerRequest;
pub use message::Streamed;
use semver::BuildMetadata;
use semver::Prerelease;
use semver::Version;

pub static METACLI_COMMIT_SEMVER: LazyLock<Version> = LazyLock::new(|| {
Expand Down Expand Up @@ -65,48 +63,48 @@ pub static METACLI_COMMIT_SEMVER: LazyLock<Version> = LazyLock::new(|| {
/// `read_msg/write_msg` to `kv_api`
///
/// - 2023-02-16: since 0.9.41:
/// Meta client add `Compatible` layer to accept KVAppError or MetaAPIError
/// 👥 client add `Compatible` layer to accept KVAppError or MetaAPIError
///
/// - 2023-02-17: since 0.9.42:
/// Meta service only responds with MetaAPIError.
/// 🖥 server only responds with MetaAPIError.
///
/// - 2023-05-07: since 1.1.32:
/// Meta service: add: TxnDeleteRequest provides a `match_seq` field to delete a record if its `seq` matches.
/// 🖥 server: add: TxnDeleteRequest provides a `match_seq` field to delete a record if its `seq` matches.
///
/// - 2023-10-11: since 1.2.153:
/// Meta service: add: pb::SeqV.meta field to support record expiration.
/// 🖥 server: add: pb::SeqV.meta field to support record expiration.
///
/// - 2023-10-17: since 1.2.163:
/// Meta service: add: stream api: kv_read_v1().
/// 🖥 server: add: stream api: kv_read_v1().
///
/// - 2023-10-20: since 1.2.176:
/// Meta client: call stream api: kv_read_v1(), revert to 1.1.32 if server < 1.2.163
/// 👥 client: call stream api: kv_read_v1(), revert to 1.1.32 if server < 1.2.163
///
/// - 2023-12-16: since 1.2.258:
/// Meta service: add: ttl to TxnPutRequest and Upsert
/// 🖥 server: add: ttl to TxnPutRequest and Upsert
///
/// - 2024-01-02: since 1.2.279:
/// Meta client: remove `Compatible` for KVAppError and MetaAPIError, added in `2023-02-16: since 0.9.41`
/// 👥 client: remove `Compatible` for KVAppError and MetaAPIError, added in `2023-02-16: since 0.9.41`
///
/// - 2024-01-07: since 1.2.287:
/// client: remove calling RPC kv_api() with MetaGrpcReq::GetKV/MGetKV/ListKV, kv_api only accept Upsert;
/// client: remove using MetaGrpcReq::GetKV/MGetKV/ListKV;
/// client: remove falling back kv_read_v1(Streamed(List)) to kv_api(List), added in `2023-10-20: since 1.2.176`;
/// 👥 client: remove calling RPC kv_api() with MetaGrpcReq::GetKV/MGetKV/ListKV, kv_api only accept Upsert;
/// 👥 client: remove using MetaGrpcReq::GetKV/MGetKV/ListKV;
/// 👥 client: remove falling back kv_read_v1(Streamed(List)) to kv_api(List), added in `2023-10-20: since 1.2.176`;
///
/// - 2024-01-17: since 1.2.304:
/// server: do not use TxnPutRequest.prev_value;
/// server: do not use TxnDeleteRequest.prev_value;
/// Always return the previous value;
/// field index is reserved, no compatibility changes.
/// 🖥 server: do not use TxnPutRequest.prev_value;
/// 🖥 server: do not use TxnDeleteRequest.prev_value;
/// Always return the previous value;
/// field index is reserved, no compatibility changes.
///
/// - 2024-01-25: since 1.2.315:
/// server: add export_v1() to let client specify export chunk size;
/// 🖥 server: add export_v1() to let client specify export chunk size;
///
/// - 2024-03-01: since: 1.2.358:
/// server: add `server_time` to `get_client_info() -> ClientInfo`,
/// 🖥 server: add `server_time` to `get_client_info() -> ClientInfo`,
///
/// - 2024-03-01: since: TODO(update me when merged):
/// client: `MetaSpec` use `ttl`, remove `expire_at`, require 1.2.258
/// - 2024-03-04: since: 1.2.361
/// 👥 client: `MetaSpec` use `ttl`, remove `expire_at`, require 1.2.258
///
/// Server feature set:
/// ```yaml
Expand All @@ -115,21 +113,16 @@ pub static METACLI_COMMIT_SEMVER: LazyLock<Version> = LazyLock::new(|| {
/// pb_seqv_meta: ["2023-10-11", "1.2.153", ]
/// kv_read_v1: ["2023-10-17", "1.2.163", ]
/// ```
pub static MIN_METASRV_SEMVER: Version = Version {
major: 1,
minor: 2,
// The binary in the https://github.com/datafuselabs/databend/releases/tag/v1.2.258-nightly
// outputs version 1.2.257;
// ```
// ./databend-meta --single
// Databend Metasrv
// Version: v1.2.257-nightly-188426e3e6-simd(1.75.0-nightly-2023-12-17T22:09:06.675156000Z)
// ```
// Skip 1.2.258 use the next 1.2.259
patch: 259,
pre: Prerelease::EMPTY,
build: BuildMetadata::EMPTY,
};
// ------------------------------
// The binary in the https://github.com/datafuselabs/databend/releases/tag/v1.2.258-nightly
// outputs version 1.2.257;
// ```
// ./databend-meta --single
// Databend Metasrv
// Version: v1.2.257-nightly-188426e3e6-simd(1.75.0-nightly-2023-12-17T22:09:06.675156000Z)
// ```
// Skip 1.2.258 use the next 1.2.259
pub static MIN_METASRV_SEMVER: Version = Version::new(1, 2, 259);

pub fn to_digit_ver(v: &Version) -> u64 {
v.major * 1_000_000 + v.minor * 1_000 + v.patch
Expand Down
27 changes: 17 additions & 10 deletions src/meta/service/src/version.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@
use std::sync::LazyLock;

use feature_set::FeatureSet;
use semver::BuildMetadata;
use semver::Prerelease;
use semver::Version;

pub static METASRV_COMMIT_VERSION: LazyLock<String> = LazyLock::new(|| {
Expand Down Expand Up @@ -46,13 +44,7 @@ pub static METASRV_SEMVER: LazyLock<Version> = LazyLock::new(|| {
});

/// Oldest compatible nightly meta-client version
pub static MIN_METACLI_SEMVER: Version = Version {
major: 0,
minor: 9,
patch: 41,
pre: Prerelease::EMPTY,
build: BuildMetadata::EMPTY,
};
pub static MIN_METACLI_SEMVER: Version = Version::new(0, 9, 41);

/// The min meta-server version that can be deployed together in a cluster,
/// i.e., the network APIs are compatible.
Expand All @@ -71,14 +63,22 @@ pub static MIN_META_SEMVER: Version = Version::new(0, 9, 41);
/// - The server depends on a sub set of the features provided by the client.
/// - The client depends on a sub set of the features provided by the server.
///
/// For example, an RPC call may look like this:
/// For example, an RPC call may look like the following:
///
/// - Server provides features S1, S2, S3, and the client requires S1, S3,
/// which is a subset of S1, S2, S3, so the call can be made.
///
/// - The client provides features C1, C2, C3, and the server requires C2, C3,
/// which is a subset of C1, C2, C3, so the response can be read by client.
///
/// ```text
/// request
/// Client calls: ------------> Server API provides:
/// - S1 - S1
/// - S2
/// - S3 - S3
///
/// response
/// Client can receives: <------ Server replies with:
/// - C1
/// - C2 - C2
Expand All @@ -91,12 +91,16 @@ pub(crate) mod raft {
use feature_set::Provide;

/// Feature set provided by raft server.
///
/// This is a change-log of the features that raft server provides,
/// and can be built into a BTreeMap of features with `FeatureSet::from_provides`
#[rustfmt::skip]
pub const PROVIDES: &[Action<Provide>] = &[
add_provide(("vote", 0), "2023-02-16", (0, 9, 41)),
add_provide(("append", 0), "2023-02-16", (0, 9, 41)),
add_provide(("install_snapshot", 0), "2023-02-16", (0, 9, 41)),
add_provide(("install_snapshot", 1), "2023-11-16", (1, 2, 212)),
add_provide(("install_snapshot", 2), "2024-05-05", (1, 2, 452)),
];

/// The client features that raft server depends on.
Expand All @@ -113,6 +117,9 @@ pub(crate) mod raft {
use feature_set::Require;

/// The server features that raft client depends on.
///
/// This is a change-log of the features that raft client depends on,
/// and can be built into a BTreeMap of features with `FeatureSet::from_required`
#[rustfmt::skip]
pub const REQUIRES: &[Action<Require>] = &[
add_require( ("vote", 0), "2023-02-16", (0, 9, 41)),
Expand Down

0 comments on commit ddfca5d

Please sign in to comment.