Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

input: stream alignment and show progress bar #13

Open
ktmeaton opened this issue Dec 5, 2023 · 1 comment
Open

input: stream alignment and show progress bar #13

ktmeaton opened this issue Dec 5, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@ktmeaton
Copy link
Collaborator

ktmeaton commented Dec 5, 2023

When working with large datasets (VirusSeq) it would be nice to monitor progress. We could also use multithreading for it (I think), if we don't mind the output being in a different order from the input.

@ktmeaton
Copy link
Collaborator Author

ktmeaton commented Dec 5, 2023

Since reading a file is inherently a linear operation, multi-threading won't be appropriate there. But potentially we can stream the file reading directly into the recombination search, with par_bridge:
https://morestina.net/blog/1432/parallel-stream-processing-with-rayon#Stream_processing

@ktmeaton ktmeaton added this to the v0.3.0 milestone Dec 6, 2023
@ktmeaton ktmeaton self-assigned this Dec 6, 2023
@ktmeaton ktmeaton added the enhancement New feature or request label Dec 6, 2023
@ktmeaton ktmeaton changed the title Add progress bar for loading query alignment input: stream alignment and show progress bar Jan 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant