Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tokio v1.0.1 with redis - task mixing #3500

Closed
pronvis opened this issue Feb 3, 2021 · 4 comments
Closed

Tokio v1.0.1 with redis - task mixing #3500

pronvis opened this issue Feb 3, 2021 · 4 comments
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug.

Comments

@pronvis
Copy link

pronvis commented Feb 3, 2021

Version

    │   ├── tokio v1.1.0
    │   │   └── tokio-macros v1.0.0
    │   ├── tokio-util v0.6.2
    │   │   ├── tokio v1.1.0 (*)
    │   │   └── tokio-stream v0.1.2
    │   │       └── tokio v1.1.0 (*)
    │   ├── tokio v1.1.0 (*)
    ├── tokio v1.1.0 (*)
    ├── tokio-stream v0.1.2 (*)
    ├── tokio-util v0.6.2 (*)
    │   ├── tokio v1.1.0 (*)
    │   ├── tokio-stream v0.1.2 (*)
│   └── tokio v1.1.0 (*)
│       │   └── tokio v1.1.0 (*)
│       ├── tokio v1.1.0 (*)
│       ├── tokio-util v0.6.2 (*)
├── tokio v1.1.0 (*)
├── tokio-util v0.6.2 (*)

cargo tree | grep redis:

├── bb8-redis v0.8.0
│   └── redis v0.19.0

cargo tree | grep bb8:

├── bb8 v0.7.0
├── bb8-redis v0.8.0
│   ├── bb8 v0.7.0 (*)

Platform
Linux videotest01.dev.wb.ru 4.19.0-13-cloud-amd64 #1 SMP Debian 4.19.160-2 (2020-11-28) x86_64 GNU/Linux

Description
After moving to Tokio v1.0.1 our "reddis-proxy" service start to fail with "magic" errors:
Response was of incompatible type: "Response type not hashmap compatible" (response was status("PONG"))
This error happens when we call let response: redis::RedisResult<collections::HashMap<String, String>> = conn.hgetall(key).await;
Looks like async tasks mixed somehow and hgetall get response from PING request. Do not know where to dig into, will try with newer version of tokio...

@pronvis pronvis added A-tokio Area: The main tokio crate C-bug Category: This is a bug. labels Feb 3, 2021
@Darksonn
Copy link
Contributor

Darksonn commented Feb 3, 2021

Can you perhaps provide more detail about what the issue is? It's very unclear from this description. What do you mean by "async tasks mixed"?

@pronvis
Copy link
Author

pronvis commented Feb 3, 2021

We have async fn:

async fn ask_redis(
    redis_pool: Pool<RedisConnectionManager>,
) -> Result<_> {
    let mut conn = redis_pool.get().await?;
    let key = todo!();

    let response: redis::RedisResult<collections::HashMap<String, String>> =
        conn.hgetall(key).await;
    
    ...
}

That called ~10 times per second. And sometimes (in 10-20% cases) this line says: Response was of incompatible type: "Response type not hashmap compatible" (response was status("PONG")).

Redis-pool have logic inside, that sends PING requests to validate connection (I think it do that before each request in my case). So, looks like response on that requests goes in wrong future task... it is the only way (as I think) why hgetall can receive PONG string as a response. We do not have those values in redis.
And all works fine before we moved to tokio 1.x

@Darksonn
Copy link
Contributor

Darksonn commented Feb 3, 2021

This sounds like a bug in the redis crate you are using. You should open an issue there, although you can include a link to this issue in the new issue.

@carllerche
Copy link
Member

Closing as it does not seem to be a Tokio bug. If this assessment is incorrect, we can reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug.
Projects
None yet
Development

No branches or pull requests

3 participants