-
Notifications
You must be signed in to change notification settings - Fork 76
Feature idea: KVS pub sub #45
Comments
Hi gedw99, Thank you for your feature ideas. |
hey @mosuka thanks for considering it. I am working on getting the Change to a Bucket to be put onto NATS. This will mena its easy to then update users when any KV bucket changes and the nature of the change. I can send you links if you want. |
@gedw99 |
@gedw99 I do not know if I can implement it, but I'm interested. Can you post the link to the following issue? :) |
closing since its over in cete now |
I just had a brain wave about another way to do this. The raft Wal log ! We can listen to it and see what mutations are occuring Do you have any links to an API to listen to the RAFT log ? This btw is also how this is done with databases. The whole area is call CDC ( change data capture ). It's a reliable way because you can be sure it's committed transactionally |
It is an interesting idea. |
I noticed you added a watch ability to the Federation branch. I am in the middle of some other work, but will try to get time to try some things out soon. |
Hi, |
ok makes sense for the Nodes. Very cool. Regarding watching for Index changes, your approach is good. Then we could surface it in the GRPC API. This is a good thing to examine the aspects of it: I wish i had time to dive into the code, but i don't so just writing it down here. |
hey @mosuka Did you see this !! |
The KVS is a generic bucket store.
I was thinking of extending it to publish changes over GRPC.
Use Case ?
When you build Clients using this system, its very useful to know when anything has change and he nature of the change. Then as a subscriber i can update my many microservices or even gui client using this. It keeps them all in sync basically.
The event would be like:
Also should have the ability to turn off eventtype Read because no one normally needs that, but it can be useful for dynamically knowing who is reading where.
Implementation:
GRPC Middleware might be the perfect fit !
https://github.com/grpc-ecosystem/go-grpc-middleware
Also great for adding other things like security etc.
Because its GRPC, it will be easy to then recieve it and put it onto NATS message broker later also as a 2nd bit of work.
Index.
I thought also about the index but thinks its not worth the effort. What you can do is make each index query output to a cache into the KVS store. then you can use PUB SUB from that. Almost like a Materialized View with pub sub on top of it. And also gives you caching for free to a degree.
The text was updated successfully, but these errors were encountered: