Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis Cluster support #385

Open
kamkash opened this issue Jan 5, 2023 · 1 comment
Open

Redis Cluster support #385

kamkash opened this issue Jan 5, 2023 · 1 comment

Comments

@kamkash
Copy link

kamkash commented Jan 5, 2023

I have a requirement to support Redis in cluster and non-cluster mode.
I have been researching adding Redis Cluster support to arq.
This is what I have done so far:

  1. Add "cluster: bool = False" flag to RedisSettings
  2. import RedisCluster
    from redis.asyncio.cluster import RedisCluster
  3. In cluster mode extend class ArqRedis from RedisCluster
    class ArqRedis(RedisCluster)

This works and I am able to connect to a Redis cluster.
However, there are several issues with redis.asyncio.cluster.RedisCluster

  1. It does not support the pipeline command with transactions
  2. It does not support MULTI/EXEC

arq uses the above commands heavily for atomic access to jobs, job results, etc
Does anyone have any insights into this? Has anyone looked at cluster support in arq?
I looked at aioredis-cluster python client, and that too has the same missing transactions support

@Wh1isper
Copy link

I developed brq for the same reason, maybe you can have a look :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants