Skip to content

Commit

Permalink
9.0.0 (#234)
Browse files Browse the repository at this point in the history
* feat: redis protocol 5, MSRV 1.75, rustls 0.23
* feat: opentelemetry tracing names
* feat: RPITIT / AFIT
* feat: cluster discovery policy
* feat: add SORT and SORT_RO
* feat: replica reset flag
* feat: add public interface feature flags

---------

Co-authored-by: Ethra <its.ethra.va@gmail.com>
  • Loading branch information
aembke and ItsEthra committed Apr 15, 2024
1 parent 52b5d3c commit c0dbf72
Show file tree
Hide file tree
Showing 143 changed files with 5,058 additions and 3,482 deletions.
40 changes: 36 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ commands:
steps:
- run:
name: Run mock tests
command: cargo test --lib --features mocks --no-default-features
command: cargo test --lib --features "mocks i-keys" --no-default-features
test_default_features:
steps:
- checkout
Expand Down Expand Up @@ -111,6 +111,18 @@ commands:
name: Run tests with unix socket features
command: source tests/environ && tests/runners/unix-socket.sh
- save-cargo-deps-cache
test_valkey:
steps:
- checkout
- restore-cargo-deps-cache
- run:
name: Run tests against local valkey deployments
command: |
source tests/environ
USE_VALKEY=1 tests/scripts/install_redis_centralized.sh
USE_VALKEY=1 tests/scripts/install_redis_clustered.sh
tests/runners/default-features.sh
- save-cargo-deps-cache

jobs:
test-default-nil-types-7_2:
Expand Down Expand Up @@ -196,18 +208,37 @@ jobs:
REDIS_VERSION: 7.2.4
steps:
- test_sentinel
test-valkey-7_2:
machine:
image: ubuntu-2204:2022.10.2
docker_layer_caching: true
resource_class: medium
environment:
REDIS_VERSION: 7.2.4
steps:
- test_valkey
test-misc:
docker:
- image: cimg/rust:1.72.1
- image: cimg/rust:1.77.1
environment:
CARGO_NET_GIT_FETCH_WITH_CLI: true
steps:
- checkout
- build_docs
- test_mocks
check-all-interface-features:
docker:
- image: cimg/rust:1.77.1
environment:
CARGO_NET_GIT_FETCH_WITH_CLI: true
steps:
- checkout
- run:
name: Check all features
command: tests/scripts/check_features.sh
clippy-lint:
docker:
- image: cimg/rust:1.75.0
- image: cimg/rust:1.77.1
environment:
CARGO_NET_GIT_FETCH_WITH_CLI: true
steps:
Expand All @@ -231,4 +262,5 @@ workflows:
- test-misc
- test-cluster-tls-features-7_2
- test-cluster-rustls-features-7_2
- clippy-lint
- clippy-lint
- check-all-interface-features
7 changes: 3 additions & 4 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,19 @@ assignees: aembke

---

Fred version - X
Redis version - X
Platform - linux|mac|windows
Using Docker and/or Kubernetes - yes|no
Deployment type - cluster|sentinel|centralized

**Describe the bug**

**To Reproduce**
Steps to reproduce the behavior:

1. X
2. X

**Logs**
(Set `RUST_LOG=fred=trace` and run with `--features debug-ids`)
(If possible set `RUST_LOG=fred=trace` and run with `--features debug-ids`)

**Additional context**
Add any other context about the problem here.
14 changes: 14 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
name: Feature Request
about: Discuss a feature request
title: "[Feature]"
labels: enhancement
assignees: aembke

---

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Additional context**
Add any other context or screenshots about the feature request here.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,5 @@ redis_centralized.conf
redis_server.pid
tests/users.acl
tests/docker/overrides/*
!tests/docker/overrides/.gitkeep
!tests/docker/overrides/.gitkeep
process.yml
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,28 @@
## 9.0.0

This version should reduce compilation times for most use cases.

* **RPITIT / AFIT**
* Set MSRV to 1.75
* Upgrade `rustls` to 0.23
* Upgrade `redis-protocol` to 5.0.0
* Split public interfaces with new feature flags.
* Add `ClusterDiscoveryPolicy` configuration options.
* Add `SORT` and `SORT_RO`
* Add `cluster_hash` policy to `Options`
* Change tracing span names to
follow [OpenTelemetry naming conventions](https://opentelemetry.io/docs/specs/semconv/general/attribute-naming/).

### Upgrading from 8.x

* Callers that use certain managed services or Kubernetes-based deployment models should review the
new `ClusterDiscoveryPolicy`.
* Double-check the new feature flags. The `codec` feature was also moved
to [redis-protocol](https://github.com/aembke/redis-protocol.rs).
* Rustls - Check the new [aws-lc-rs](https://aws.github.io/aws-lc-rs/requirements/index.html) requirements or switch
back to `rustls/ring`.
* Note the new [tracing span names](src/trace/README.md).

## 8.0.6

* Add `TransactionInterface` to `RedisPool`
Expand Down
81 changes: 45 additions & 36 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,44 @@
Contributing
===========

This document gives some background on how the library is structured and how to contribute. It focuses primarily on how to add new commands. See the [design doc](docs/README.md) for more background on how the library is designed.
This document gives some background on how the library is structured and how to contribute. It focuses primarily on how
to add new commands. See the [design doc](docs/README.md) for more info.

# General

* Use 2 spaces instead of tabs.
* Run rustfmt and clippy before submitting any changes.
* Clean up any compiler warnings.
* Use the `async` syntax rather than `impl Future` where possible.
* Please use [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).

# File Structure

The code has the following structure:

* The [commands](src/commands) folder contains the public interface and private implementation for each of the Redis commands, organized by category. This is roughly the same categorization used by the [public docs](https://redis.io/commands/). Each of these public command category interfaces are exposed as a trait with default implementations for each command.
* The [clients](src/clients) folder contains public client structs that implement and/or override the traits from [the command category traits folder](src/commands/impls). The [interfaces](src/interfaces.rs) file contains the shared traits required by most of the command category traits, such as `ClientLike`.
* The [monitor](src/monitor) folder contains the implementation of the `MONITOR` command and the parser for the response stream.
* The [protocol](src/protocol) folder contains the implementation of the base `Connection` struct and the logic for splitting a connection to interact with reader and writer halves in separate tasks. The [TLS interface](src/protocol/tls.rs) is also implemented here.
* The [router](src/router) folder contains the logic that implements the sentinel and cluster interfaces. Clients interact with this struct via a message passing interface. The interface exposed by the `Router` attempts to hide all the complexity associated with sentinel or clustered deployments.
* The [trace](src/trace) folder contains the tracing implementation. Span IDs are manually tracked on each command as they move across tasks.
* The [types](src/types) folder contains the type definitions used by the public interface. The Redis interface is relatively loosely typed but Rust can support more strongly typed interfaces. The types in this module aim to support an optional but more strongly typed interface for callers.
* The [modules](src/modules) folder contains smaller helper interfaces such as a lazy [Backchannel](src/modules/backchannel.rs) connection interface and the [response type conversion logic](src/modules/response.rs).

## Examples
* The [commands](src/commands) folder contains the public interface and private implementation for each of the Redis
commands, organized by category. This is roughly the same categorization used by
the [public docs](https://redis.io/commands/). Each of these public command category interfaces are exposed as a trait
with default implementations for each command.
* The [clients](src/clients) folder contains public client structs that implement and/or override the traits
from [the command category traits folder](src/commands/impls). The [interfaces](src/interfaces.rs) file contains the
shared traits required by most of the command category traits, such as `ClientLike`.
* The [monitor](src/monitor) folder contains the implementation of the `MONITOR` command and the parser for the response
stream.
* The [protocol](src/protocol) folder contains the implementation of the base `Connection` struct and the logic for
splitting a connection to interact with reader and writer halves in separate tasks.
The [TLS interface](src/protocol/tls.rs) is also implemented here.
* The [router](src/router) folder contains the logic that implements the sentinel and cluster interfaces. Clients
interact with this struct via a message passing interface. The interface exposed by the `Router` attempts to hide all
the complexity associated with sentinel or clustered deployments.
* The [trace](src/trace) folder contains the tracing implementation. Span IDs are manually tracked on each command as
they move across tasks.
* The [types](src/types) folder contains the type definitions used by the public interface. The Redis interface is
relatively loosely typed but Rust can support more strongly typed interfaces. The types in this module aim to support
an optional but more strongly typed interface for callers.
* The [modules](src/modules) folder contains smaller helper interfaces such as a
lazy [Backchannel](src/modules/backchannel.rs) connection interface and
the [response type conversion logic](src/modules/response.rs).

## Examples

## Add A New Command

Expand All @@ -40,27 +54,26 @@ pub enum RedisCommandKind {
}

impl RedisCommandKind {

// ..

pub fn to_str_debug(&self) -> &str {
match *self {
// ..
RedisCommandKind::Mget => "MGET",
// ..
}
}

// ..

pub fn cmd_str(&self) -> Str {
match *self {
// ..
RedisCommandKind::Mget => "MGET"
// ..
}
}

// ..
}
```
Expand All @@ -76,7 +89,7 @@ pub async fn mget<C: ClientLike>(client: &C, keys: MultipleKeys) -> Result<Redis
// time spent here will show up in traces in the `prepare_command` span
Ok((RedisCommandKind::Mget, keys.into_values()))
})
.await?;
.await?;

protocol_utils::frame_to_results(frame)
}
Expand All @@ -86,32 +99,33 @@ Or use one of the shorthand helper functions or macros.

```rust
pub async fn mget<C: ClientLike>(client: &C, keys: MultipleKeys) -> Result<RedisValue, RedisError> {
utils::check_empty_keys(&keys)?;
utils::check_empty_keys(&keys)?;
args_values_cmd(client, keys.into_values()).await
}
```

3. Create the public function in the [src/commands/interfaces/keys.rs](src/commands/interfaces/keys.rs) file.
3. Create the public function in the [src/commands/interfaces/keys.rs](src/commands/interfaces/keys.rs) file.

```rust

// ...

#[async_trait]
pub trait KeysInterface: ClientLike {

// ...

/// Returns the values of all specified keys. For every key that does not hold a string value or does not exist, the special value nil is returned.
///
/// <https://redis.io/commands/mget>
async fn mget<R, K>(&self, keys: K) -> RedisResult<R>
where
R: FromRedis,
K: Into<MultipleKeys> + Send,
fn mget<R, K>(&self, keys: K) -> impl Future<Output=RedisResult<R>> + Send
where
R: FromRedis,
K: Into<MultipleKeys> + Send,
{
into!(keys);
commands::keys::mget(self, keys).await?.convert()
async move {
into!(keys);
commands::keys::mget(self, keys).await?.convert()
}
}
// ...
}
Expand All @@ -133,18 +147,15 @@ impl KeysInterface for Transaction {}

# Adding Tests

Integration tests are in the [tests/integration](tests/integration) folder organized by category. See the tests [README](tests/README.md) for more information.
Integration tests are in the [tests/integration](tests/integration) folder organized by category. See the
tests [README](tests/README.md) for more information.

Using `MGET` as an example again:

1. Write tests in the [keys](tests/integration/keys/mod.rs) file.

```rust
pub async fn should_mget_values(client: RedisClient, _: RedisConfig) -> Result<(), RedisError> {
check_null!(client, "a{1}");
check_null!(client, "b{1}");
check_null!(client, "c{1}");

let expected: Vec<(&str, RedisValue)> = vec![("a{1}", 1.into()), ("b{1}", 2.into()), ("c{1}", 3.into())];
for (key, value) in expected.iter() {
let _: () = client.set(key, value.clone(), None, None, false).await?;
Expand All @@ -160,7 +171,6 @@ pub async fn should_mget_values(client: RedisClient, _: RedisConfig) -> Result<(

```rust
mod keys {

// ..
centralized_test!(keys, should_mget_values);
}
Expand All @@ -171,7 +181,6 @@ mod keys {

```rust
mod keys {

// ..
cluster_test!(keys, should_mget_values);
}
Expand Down
Loading

0 comments on commit c0dbf72

Please sign in to comment.