Skip to content

Conversation

@michaelmckinsey1
Copy link
Collaborator

@michaelmckinsey1 michaelmckinsey1 commented Jan 29, 2024

Summary

Use tqdm to add a progress bar to the thicket reader since for larger amount of files (~1000+) in can take in the magnitude of minutes without any indication to the user (we don't want it to seem like it's hanging). The progress bar is in 2 stages for reading the files, and creating the thickets.

image

@michaelmckinsey1 michaelmckinsey1 added area-thicket Issues and PRs involving Thicket's core Thicket datastructure and associated classes priority-normal Normal priority issues and PRs status-work-in-progress PR is currently being worked on type-feature Requests for new features or PRs which implement new features status-ready-for-review This PR is ready to be reviewed by assigned reviewers and removed status-work-in-progress PR is currently being worked on labels Jan 29, 2024
@michaelmckinsey1
Copy link
Collaborator Author

michaelmckinsey1 commented Feb 2, 2024

Need to remove second stage and add flag to turn off tqdm by default

Copy link
Collaborator

@slabasan slabasan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelmckinsey1 Two things: (1) Can you take a look at the conflicting file? (2) What do you think about setting disable_tqdm=True in our unit tests? For one, it'll show an example of using the new parameter, and also unit tests are small enough that we don't need to be showing progress. Not sure how this will render in our static jupyter notebook outputs in RTD, but maybe we disable it there as well.

@michaelmckinsey1
Copy link
Collaborator Author

michaelmckinsey1 commented Apr 5, 2024

@michaelmckinsey1 Two things: (1) Can you take a look at the conflicting file? (2) What do you think about setting disable_tqdm=True in our unit tests? For one, it'll show an example of using the new parameter, and also unit tests are small enough that we don't need to be showing progress. Not sure how this will render in our static jupyter notebook outputs in RTD, but maybe we disable it there as well.

  1. Fixed
  2. Would we rather have it manually set in every test that uses a reader or concat_thickets or done like in d92a8a1 done manually 4fea8f3

@slabasan slabasan merged commit 24c1ed5 into llnl:develop Apr 8, 2024
@michaelmckinsey1 michaelmckinsey1 deleted the feature-progress_bar branch April 9, 2024 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-thicket Issues and PRs involving Thicket's core Thicket datastructure and associated classes priority-normal Normal priority issues and PRs status-ready-for-review This PR is ready to be reviewed by assigned reviewers type-feature Requests for new features or PRs which implement new features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants