Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Adds support for the Redis cluster #1170

Open
powersemmi opened this issue Jan 25, 2024 · 1 comment
Open

Feature: Adds support for the Redis cluster #1170

powersemmi opened this issue Jan 25, 2024 · 1 comment
Labels
enhancement New feature or request Redis Issues related to `faststream.redis` module and Redis features

Comments

@powersemmi
Copy link

Is your feature request related to a problem? Please describe.
We use redis cluster at our company, but there is no cluster mode support in faststream. When trying to connect to one of the nodes and send a message, faststream displays an error

redis.exceptions.ResponseError: MOVED 7304 127.0.0.1:1234

Describe the solution you'd like
I would like to get support for redis cluster mode. And specifically:

  1. I am most interested in working with redis streams from cluster mode, so it would be cool if you added a RedisClusterBroker that would take as an argument a list of connections to clusters
  2. The method of sending a message must correctly distribute messages across clusters (you can read about this in the redis documentation) and have a parameter for the name of the consumer group when the "stream" parameter is specified
  3. The consumer must create a stream in the cluster and correctly receive messages from different nodes
@powersemmi powersemmi added the enhancement New feature or request label Jan 25, 2024
@powersemmi
Copy link
Author

powersemmi commented Feb 26, 2024

I experimented a bit and came to the conclusion that the following solution will solve the problem

from typing import Any

from faststream.redis.broker import RedisBroker
from faststream.redis.producer import RedisFastProducer
from faststream.redis.security import parse_security
from faststream.types import AnyDict
from redis.asyncio.cluster import RedisCluster as Redis
from redis.asyncio.connection import parse_url
from typing_extensions import override


class RedisClusterBroker(RedisBroker):
    @override
    async def _connect(  # type: ignore[override]
        self,
        url: str,
        **kwargs: Any,
    ) -> 'Redis[bytes]':
        url_options: AnyDict = parse_url(url)
        url_options.update(kwargs)
        url_options.update(parse_security(self.security))

        client = Redis.from_url(url=self.url)
        self._producer = RedisFastProducer(
            connection=client,
            parser=self._global_parser,  # type: ignore[arg-type]
            decoder=self._global_parser,  # type: ignore[arg-type]
        )
        return client

But in order to be able to fully use the arguments of the RedisCluster class, you need to make a few more changes. Maybe one of these days I'll create an MR with everything I need.

@Lancetnik Lancetnik added the Redis Issues related to `faststream.redis` module and Redis features label May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Redis Issues related to `faststream.redis` module and Redis features
Projects
Status: Backlog
Development

No branches or pull requests

2 participants