Add cluster support #119
Add cluster support #119
Conversation
ghost
mentioned this pull request
ghost
It is working only with masters nodes properly.
…tertests.py. Fixed a bug in key_slot.
…ctionality is not implemented yet or doesn't make sense on a cluster.
…t it can be imported in application tests.
… anymore to download the redis-trib.rb file from Redis source.
… race conditions when accessing the cluster_pool
…rned info. Various improvements.
|
👍 when is this functionality expected to be ready? |
|
I really appreciate everyone's efforts here. It looks like quite an undertaking. Given that everyone has limited amounts of time to spend on open source projects, it feels like this PR is moving in spurts, but then gets out of sync with the master branch. Would it help to break this into lots of smaller PRs that we could start merging into master and continue to consider the cluster support a WIP during that time? For example, maybe a single PR that defines the Redis cluster commands. Then, a PR for how to establish a connection pool for Redis cluster. I'm not sure what the best breakdown would be, but I might be able to offer some help if @The-Panda or @popravich (or anyone more knowledgable than me) had some ideas on the best way to break this into smaller chunks. |
|
@The-Panda does this fork support the full functionality of cluster? If not could you direct me to the working repo with cluster support. |
|
@The-Panda do you have any ETA on rebasing this PR? I would be willing to finish it if you don't have the time. |
|
Hi everyone, I have created a PR on I will create a direct PR on this repo, if above does not get merged in a day or two (not sure if that's right way of doing this). |
|
Thanks to the work of @lemurchik and @mmniaziqb this pull request is now again up-to-date with aioredis:master. However, @popravich has stated above that he is not considering to merge this pull request and has his own plans for working towards a cluster commands implementation. Until then, this PR provides almost all functionality of Redis cluster mode. My organization uses it successfully, but is not planning to do any work on it besides bugfixes and sometimes merging the original aioredis master. |
|
Thanks for all of your work @The-Panda! @popravich is there anything that we can help with to bring this in line with your expectations for cluster support in aioredis? |
|
Thank you guys for all the work! |
|
@popravich any updates on the review of this PR? |
|
@popravich any updates on this? |
|
Hi @The-Panda, do you have any plans to add support for the |
|
@The-Panda @popravich I have made a bunch of improvements to this fork including better handling of error responses (MOVED/ASK/CLUSTERDOWN/etc), Connection Errors, as well as support for pipelines and transactions. Would there be any interest in merging those changes in as well? @popravich are you generally happy with the approach taken here or were you planning on making wholesale changes? |
|
@asgoel @The-Panda @popravich Any updates on this PR? The library is really lacking without this feature. |
|
@asgoel have you published those improvements on github? |
|
@eoghanmurray yes they're on this branch here: https://github.com/ramjet-labs/aioredis. Note a lot of transaction stuff is quite hacky, but if you don't intend it using that too heavily it should be fine. Furthermore, there are probably some bugs around when nodes go down, but I think it generally works pretty well. We've been using this branch in production for over a year with minimal issues. Feel free to make any improvements you see fit! |
|
FYI there's a redis cluster proxy in the work, by the redis team -> https://github.com/artix75/redis-cluster-proxy |
|
Maintenance for this project has finally resumed! This is a very long-running PR and it looks like attempting to merge in the existing request may not be fruitful, but I'd love to get Cluster support in with the next release! I'm going to close out this PR for now, but if anyone is willing to take on the work of finalizing this feature and getting it into master, please let me know. I'm happy to invite you as an official contributor to ease the work and reduce the risk of it going stale again. |
|
Advertising time =) I finally released cluster support extension library for aioredis. Enjoy! https://pypi.org/project/aioredis-cluster/ Library works in production for over a year now and work's great. |
|
Well here's mine :)
https://github.com/machinezone/rcc <https://github.com/machinezone/rcc>
It is actually more than just a library, it has a CLI to do many cluster operation such as a fancy resharding. Anton if you want to give it a try it would be great.
I think it would be great to get aioredis working as it's the one that most people use.
… On Dec 11, 2020, at 8:55 AM, Anton Ilyushenkov ***@***.***> wrote:
Advertising time =)
I finally released cluster support extension library for aioredis. Enjoy!
https://pypi.org/project/aioredis-cluster/ <https://pypi.org/project/aioredis-cluster/>
Library works in production for over a year now and work's great.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#119 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC2O6UPEGZ3IF2GSTEKT3GTSUJFHVANCNFSM4CAA3W2Q>.
|
I have continued the work of trezorg to add cluster support. All the old tests now run against both a standard Redis and a Redis cluster (except those which test features that are not available or not yet supported on a cluster).