Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/kafka collector #4

Draft
wants to merge 12 commits into
base: dev
Choose a base branch
from
Draft

Feat/kafka collector #4

wants to merge 12 commits into from

Conversation

microwavekonijn
Copy link
Member

@microwavekonijn microwavekonijn commented Sep 21, 2022

Collector that uses Kafka. Each message will be assigned a topic based on event name, world id, and optionally the zone id. Messages are retained by default for 3 to 7 days and are stored in a single partition. The use of Kafka allows previous alerts to be replayed which can be used to back-fill data, or test new features that are being developed(which can then also be compared to alerts on the production).

  • Documentation of config;
  • Build pipeline for docker;
  • Integration into the Aggregator.

This is also a prototype. It can be expanded with cron jobs and data fetching, such as to fetch map data on a regular basis or characters related to a event. This requires however bigger changes on the aggregator side as it changes the data ingress format.

@microwavekonijn microwavekonijn added the enhancement New feature or request label Sep 21, 2022
@microwavekonijn microwavekonijn self-assigned this Sep 21, 2022
@microwavekonijn microwavekonijn changed the base branch from dev to master September 21, 2022 15:46
@microwavekonijn microwavekonijn changed the base branch from master to dev September 21, 2022 15:46
@microwavekonijn microwavekonijn changed the base branch from dev to master September 21, 2022 15:47
@microwavekonijn microwavekonijn changed the base branch from master to dev September 21, 2022 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants