Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swappable Persistence Backends? #12

Closed
stemcc opened this issue Aug 11, 2018 · 12 comments · Fixed by #32
Closed

Swappable Persistence Backends? #12

stemcc opened this issue Aug 11, 2018 · 12 comments · Fixed by #32

Comments

@stemcc
Copy link

stemcc commented Aug 11, 2018

I'm curious if making the persistence backends swappable is on the roadmap for this library.

In my specific use case, it would be nice to use the lightweight Bolt for persistence while still utilizing centrifuge's pub sub system, unlike the way you have it set up with Redis now which seems to require using Redis's pub/sub system if you want persistence.

If there was an API for writing pluggable backends, I'd happily get started on one for bolt.

@FZambia
Copy link
Member

FZambia commented Aug 12, 2018

Hello, my plan is making internal engine parts pluggable in the end. Though I am not sure about correct implementation and currently concentrated on finishing work on Centrifugo v2 built on top of this lib.

I don't fully understand your intents - this library should only persist a short-term cache of messages. What is your use case for persistent store? Are you going to keep unlimited amount of messages in it? Also Redis Pub/Sub is not currently used for persistence - it is used for Centrifuge node scalability.

@stemcc
Copy link
Author

stemcc commented Aug 17, 2018

Great, I'm glad to hear that's on the roadmap.

My use case for persistence is to have a cache that can survive reboots, while ideally using something a little simpler than Redis but that can still provide simple kv lookups (like Bolt for example). Otherwise I would just load a log file into memory when the app comes back online. I'm using Centrifuge to pass messages and data from a local daemon/server to a local gui client, local daemon to remote client, or remote daemon/server to local daemon/server.

@ghost
Copy link

ghost commented Sep 17, 2018

Badger or bolt with raft leader election should be enough to get HA.

I think redis is HA in that you can specify how often it syncs to disk ?

@FZambia
Copy link
Member

FZambia commented Sep 17, 2018

Yep, badger or bolt with raft leader election is an interesting thing to extend current possibilities.

The hardest part of making engine exported at this time is the fact that publish into PUB/SUB should be atomic with saving message to history cache. So publish operation in a whole can be atomic. This works with Redis (to certain degree though, I suppose atomicity of Redis LUA scripts could be not guaranteed when PUB/SUB operations involved), also in Memory engine. But if we split engine on PUB/SUB part and storage part we lose this atomicity. In this case message can be published to current subscribers but not saved in cache, or saved in cache but not published to subscribers. Currently I don't know what should we do in case of these errors at publish time.

Redis has no sync replication so there is a chance to lose messages even with AOF with fsync on every query when master goes down and last messages not achieved slave yet. Having nature of Redis in mind current Redis engine implementation can only be best effort mechanism to provide message persistence for later recovery process. It will work in practice for most real-world apps but there are no strong guarantees as far as I understand.

@ghost
Copy link

ghost commented Sep 17, 2018 via email

@FZambia
Copy link
Member

FZambia commented Sep 17, 2018

@gedw99 thanks for suggestions. Centrifuge lib in its current state is very close to event sourcing system. At least this is already kind of. Its hard to say at moment in what direction we should expand it - maybe the understanding will come with usage reports if any:)

@ghost
Copy link

ghost commented Sep 18, 2018

I guess it's also because your relying on redis to do the hard stuff which is totally understandable.

You could use emitter golang project instead of redis. Then you can wrap it. It uses mqtt as the messaging layer and supports scaling honizontally

@thienpow
Copy link

maybe can use mongodb instead of redis

@FZambia
Copy link
Member

FZambia commented Mar 5, 2019

Guys, just want to update status on this. I remember about this issue but the task is not straightforward. I have some ideas but they ruin under various edge cases. At moment I am in progress of validating new idea - maybe it will fit fine but can't be sure.

At least I proved that I can make current engines separate if neseccary. Also played a bit with custom Nats engine which implements only PUB/SUB logic without any persistence layer.

@FZambia
Copy link
Member

FZambia commented Mar 9, 2019

@stemcc please take a look at referenced pull request where I am trying to split Engine on three parts - Broker, HistoryManager and PresenceManager. This is a road to pluggable components, hope this will allow to do more with this library. At least look at interesting concept example where Nats used for PUB/SUB and Redis for other stuff.

@FZambia
Copy link
Member

FZambia commented Mar 11, 2019

Let's reopen until it will be documented

@FZambia FZambia reopened this Mar 11, 2019
@FZambia
Copy link
Member

FZambia commented Mar 25, 2019

Just posted a note about pluggable components on Medium, also we have an example of setting custom broker implementation. I suppose this is enough for a moment. If there are any questions about current implementation - feel free to ask.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants