Issue #537 (alternative kafka approach)

[FayKounara]

As discussed under the issue #537 me and @ElenaSkep would like to propose an enhancement and use apache kafka as an alternative for backend storage.

What it does: Kafka listens to the changes made in a db/resource and stores them in a topic which takes the name of the db.Doing that you can see from the topic of your db the history of all the changes.

Prerequisite:Please download the zip file of apache kafka and run it in the bash with these 2 commands $ bin/zookeeper-server-start.sh config/zookeeper.properties and this $ bin/kafka-server-start.sh config/server.properties.

After that whenever you change something in your db you can see the changes by doing this $ bin/kafka-console-consumer.sh --topic your-db-name --from-beginning --bootstrap-server localhost:9092

[JohannesLichtenberger]

This could be part of an external connector or change data connector (Maybe a separate repository using sirix-core as a dependency). That said as in the case with the websocket client (which sould rather be the server where a TypeScript based client might subscribe to a specific database/resource via a dedicated route in sirix-rest-api) we have to invert the dependency, as it should be no requirement to run Kafka.

[JohannesLichtenberger]

Hope that makes sense and that you understand the issue a bit more