Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import mimir whisper converter #19

Merged
merged 3 commits into from
Apr 20, 2023
Merged

Import mimir whisper converter #19

merged 3 commits into from
Apr 20, 2023

Conversation

ywwg
Copy link
Contributor

@ywwg ywwg commented Apr 20, 2023

Imports the mimir whisper converter code wholesale, erasing the commit history.

mimir-whisper-converter is a utility for converting Graphite Whisper archives to Mimir blocks. Because archives can be very large, it does this conversion in multiple steps, designed to run separately to reduce memory consumption. It currently only supports untagged metrics.

Contains a useful filetype, an Unsorted String Table, which is used to store the intermediate data in the two-pass process. A USTable stores string-keyed protobuf data and files can be opened either for appending or for reading.

Tests pass and command runs.

This required some dependency updates which required API changes, I think they are ok.

Copy link
Collaborator

@alexgreenbank alexgreenbank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ywwg ywwg merged commit 7a0ff01 into main Apr 20, 2023
@ywwg ywwg deleted the owilliams/whisper branch April 20, 2023 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants