Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REFACTOR] move libraries to cargo workspaces #1

Open
3 tasks done
Narayanbhat166 opened this issue Sep 19, 2023 · 0 comments · Fixed by #5
Open
3 tasks done

[REFACTOR] move libraries to cargo workspaces #1

Narayanbhat166 opened this issue Sep 19, 2023 · 0 comments · Fixed by #5

Comments

@Narayanbhat166
Copy link
Owner

Narayanbhat166 commented Sep 19, 2023

Cargo workspaces provide a neat way to manage multiple libraries and binaries which share utilities into a single repository. More information can be found here The many binaries that will be in this project are

  • controller: A controller is responsible for coordinating resources between multiple producers. It should implement a efficient way to buffer the data sent by producers. Provide a layer of abstraction around message queues for producers and consumers.

  • producer: Scrape data from leetcode and send it to controller. This can be scaled very effeciently.

  • consumer: Flush the data from message queues to database and file storage.

The multiple libraries that will be included in this project are

  • Redis: Provide redis helper functions to communicate to redis.
  • Kafka: Provide kafka helper functions to communicate to kafka brokers.
  • Configs: To manage configs.
  • protobuff: To compile proto files to rust specific structures and functions.

Since this would be a large refactor, I will be breaking this down to child issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant