-
Notifications
You must be signed in to change notification settings - Fork 23.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redis cluster should support announce-ip & announce-port #2527
Comments
I did try the recommended workaround of specifying an explicit bind, but it fails too. It seems that it won't work without setting IP_TRANSPARENT option on the socket, which redis currently does not. |
Related to #2335 |
Agreed, will write here a design spec in the next days. p.s. in the meanwhile you may want to prevent Docker from mapping ports, there is a specific feature for 1:1 mapping. |
I'm having the same trouble - see #2621
however i've noticed they all act the same -- the ones on the 192.168.1.246 IP behave exactly the same (for their own IP). Relevant bits as seen by 192.168.1.246:6379:
So they each have a (machine local) 172.17.* net that is not available from outside the machine |
+1 please |
If have the same problem. It isn't possible to create a redis cluster on different servers with docker. +1 please |
+1 |
Prioritized in order to be shipped with the next release (3.2). Thanks for the feedbacks. |
👍 i there a proposed work-around available for this issue? |
+1 |
+1 |
3 similar comments
+1 |
+1 |
+1 |
As a workaround, if possible in your env, run the containers with the option |
+1 |
Scheduled for Redis 3.2 that will go in RC in a matter of weeks (~4 hopefully). |
p.s. this is going to be backported into 3.0.x anyway... |
+1 |
@fatduo the workaround is using |
@mostolog Can you explain how elastic search does this and what is incompatible with swarm mode? Swarm Mode is smth. Kubernetes does for a long time :) As soon as ip announcement works everything is fine, cluster nodes can be found by contacting an infrastructure service which routes the request to the appropriate node. |
Hi Consider the following docker command:
Instead of creating a 3 node redis cluster, it creates a 3 node redis array (3 standalone instances of redis). It would be great to have an option to configure a cluster name, multicast group, SSDP...to have a "PnP" cluster, if they shared the same password.. Elasticsearch had multicast support enabled by default on previous versions, but they just moved to unicast, and we are experiencing the same issues: "the need of address to be able to form a cluster" Regards PS: Even more: it would be great allowing redis listen on a ip/mask (eg: 172.16.0.2/16) instead of fixed IP. this would enable dynamic IP configuration while preserving the redis-network isolation |
I don't think multicast is an option :( It'd mean a lot of redesign in Redis, afaik Redis is TCP/IP only, on top this answer might be interesting: https://groups.google.com/forum/#!topic/coreos-user/tbh1op__qNM Multicast is very hard to implement with cloud providers and from all available networking layers for docker only weave supports multicast. |
Ok about multicast...but what about the other options? Have you considered an easier way of deploying a cluster not involving redis-trib/utilities? If you consider other proposals (bonjour or listening on network/mask) are interesting I can create those issues Regards |
@vvanholl I didn't mention those, 'cause as a docker user, swarm mode is replacing external registry needs (IMHO: that's the way to take) |
Docker Swarm is Good to me |
An additional feedback based on my own try-and-fail attempt with Redis cluster and K8s: |
It supports cluster-announce-ip and cluster-announce-port redis/redis#2527
Can someone summarize -- can redis be run under Docker Swarm in the master/slave/sentinel configuration? I have tried (redis 3.2) but the cluster seems to get befuddled as containers change IP addresses when they are restarted. It seems to get into a state where it will never elect a master even though there are two redis servers running + 3 sentinels. |
@drnybble If I am not mistaken, this feature was first introduced in Redis 4.0. Here is what I read from the release history log's section "Redis 4.0.0-RC1 Released Fri Dec 2 10:40:01 CEST 2016" located in (https://raw.githubusercontent.com/antirez/redis/4.0/00-RELEASENOTES):
|
Thanks -- I am talking master/slave/sentinel configuration (replication) though, not cluster. I seem to have some success specifying this for each master/slave: slave-announce-ip [my-virtual-ip] I specified the hostname of the service -- there still seems to be some confusion here since the sentinels seem to recognize both the hostname and the ip address as separate entities: 1:X 17 Aug 18:04:37.371 # -sdown slave redis1:6379 10.0.0.3 6379 @ redis-cluster 10.0.0.7 6379 I can always resolve the hostname before writing it to the .conf file if it is a problem. I was able to force update services for masters and slaves (docker stack update --force redis_redis1 for instance) and so far it seems to be working as expected. |
@drnybble I am a bit mislead: I think this is wrong issue to discuss about announce-ip in HA deployment, as this issue explicitly deals with Redis Cluster. |
@ovanes You are correct. However, I think this information is also relevant for running Redis Cluster in Docker Swarm. The announce-ip should announce the Virtual IP of the service so it is stable over container updates. |
@drnybble Yes, you are right. But than it should be tested with the version where it was introduced than, i.e. 4.0 and not 3.2. And redis release docs explicitly state, that there were bus protocol updates which are incompatible with version 3.2, etc. IMO, all the findings presented for 3.2 are pointless :( As one can't make any kind of conclusion to the version 4.0 because of severe changes. |
So is this supposed to work in redis 4.0.2? There are cluster-announce directives now though I still can't get it to work. Cheers |
The square brackets are part of the error message formatting, so the address is fine. As for the actual problem, sorry but don't have any idea about that. |
@wizzu thank you for pointing this out. So if the ip address is correct I have no idea what else could be wrong at this point. |
+1 |
Hey all. I ran into this as well. I wrote a proxy that appears to work for simple set/gets on small clusters. This is a work in progress and was created only for testing a RedisCluster running in a docker-swarm, but it should work for K8s, too. This proxy runs on the cluster-side and alters the IP addresses from the Redis Cluster to lie to clients so that they can route to a single proxy end-point. The proxy end-point listens on a single port for each node in the cluster and passes the traffic onto the actual nodes. https://github.com/wojnosystems/redis_cluster_proxy Please feel free to contribute for anything that's broken. This code has some limitations, but this enables me to test my cluster-integration tests. Hope it helps you as well. |
This has been implemented long ago in 5ac5e3e, closing. This issue has diverted to other topics, please feel free to re-open new issues if relevant. |
We're running into problems when trying to run redis cluster from docker on different hosts. Dockerized redis node advertises its internal docker address, which is unreachable from other nodes in the cluster. I'm aware of various hacks and workarounds, but I'm proposing that redis cluster supports
announce-ip
andannounce-port
configuration options, similarly to redis sentinel.The text was updated successfully, but these errors were encountered: