Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor database logging backend #164

Open
hagertnl opened this issue Mar 28, 2024 · 1 comment
Open

Refactor database logging backend #164

hagertnl opened this issue Mar 28, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@hagertnl
Copy link
Contributor

To prepare for other database logging, we need to abstract the InfluxDB logging code out of the harness core code.

Initial thoughts are that we can create a class called something like db_logger, which can enable multiple database backends simultaneously, and receives a dictionary of the test data (or metrics or nodescreen) that it logs to the respective databases.

This will enable us to go and add a Kafka database backend.

@hagertnl
Copy link
Contributor Author

Note -- when we get to the metrics & node screen data, can we try to bundle messages together please? :) We did not in the original implementation, but when we run a 1000-node node screen, it currently has to log 1000 messages to InfluxDB, instead of 1 large message that has 1000 insertions. I don't know the API for this off the top of my head, but it should be possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant