Which is the best way to replicate between beanstalkd?

[achetronic]

Hello there!

I have been testing coolbeans but it does not have all the commands implemented for beanstalkd, so using that project is not an option yet, and I was wondering which is the best way to replicate the data between beanstalkd servers.

The idea is to have a Haproxy in front of it and only send the the traffic to one available beanstalkd, and if it fails, then send it to the next one, like (Redis + Sentinel + Haproxy) pattern, but the problem is that, to be able to do it replication need to be done, at least pasive replication. So my question is, do you have some idea or best practise to do this with beanstalkd?

Thank you in advance!

[ptjm]

I think there isn't a good solution to this problem. The most reliable thing is probably to run with -f0 on either a distributed file system or a system replicated with something like DRDB.

[achetronic]

Hello @ptjm I was doing a PoC with rsync synchronizing the binlogs and it seems to work, but the bad point is that beanstalkd needs to be restarted to read them, so if I switch the LB from a node to another, the second need a restart in the middle.
I was trying this as a PoC because, if clustering mode is not implemented on beanstalkd side, it is possible to implement some process in the same Docker image to select a master and trigger the replication of binlogs files always from the master to the slaves, with an endpoint to know who is the master.

The idea could be for this process to restart the beanstalkd to force it to use up-to-date binlogs, when a slave is elected as a master

That way, a Haproxy in front could forward the traffic always to the master, doing tcp-check against all the beanstalkd backends

I was thinking about this, because Coolbeans is a great project, and working pretty fine, but some commands are not implemented yet. Doing this way, you have all the commands available, always, because you would be using the upstream daemon.

Do you see this like a madness?

[ysmolski]

Please do not do this. This project is not suited for any kind of replication across multiple instances. If you want something supporting distributed queues, you can look at quorums of RabbitMQ.

[achetronic]

Please do not do this. This project is not suited for any kind of replication across multiple instances. If you want something supporting distributed queues, you can look at quorums of RabbitMQ.

Hello there. I know, I know, but i am looking for a possible solution in the infra side because of a legacy application not too well designed in this aspect. Still evaluating things, but the truth is not even Coolbeans project is suitable for it, due to some specific beanstalkd features used not implemented in Coolbeans yet

I think it is time to close this thread? :)