-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Cargo workspaces provide a neat way to manage multiple libraries and binaries which share utilities into a single repository. More information can be found here The many binaries that will be in this project are
-
controller: A controller is responsible for coordinating resources between multiple producers. It should implement a efficient way to buffer the data sent by producers. Provide a layer of abstraction around message queues for producers and consumers.
-
producer: Scrape data from leetcode and send it to controller. This can be scaled very effeciently.
-
consumer: Flush the data from message queues to database and file storage.
The multiple libraries that will be included in this project are
- Redis: Provide redis helper functions to communicate to redis.
- Kafka: Provide kafka helper functions to communicate to kafka brokers.
- Configs: To manage configs.
- protobuff: To compile proto files to rust specific structures and functions.
Since this would be a large refactor, I will be breaking this down to child issues