Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Feature/progress bar #28

Closed

Conversation

CarloLepelaars
Copy link
Contributor

@CarloLepelaars CarloLepelaars commented Nov 9, 2022

Fixes issue #20

  • Adds progress bar to all text and image embedders.
  • Tests for SentenceEncoder.
  • Use perfplot for progress bar?
  • Can we ensure fast NumPy vectorization while using a progress bar?

@koaning
Copy link
Owner

koaning commented Nov 9, 2022

One thing that I'm also wondering is... Technically nothing is stopping you from doing something like this:

tfm = SentenceTransformer()
tfm.transform([t for t in tqdm(long_list_of_texts)])

It's not a huge ask from an end user to do this, and it might just be enough to add an FAQ on the docs for this.

Still, any performance number/chart on this would be grand to share there as well.

@CarloLepelaars
Copy link
Contributor Author

Agree, if you are iterating one by one the user can easily add a progress bar and configure it as they like.

I'm not that familiar with mkdocs, but can have a look on integrating a perfplot in the docs.

@koaning
Copy link
Owner

koaning commented Nov 10, 2022

There's a course on calmcode 😉

@CarloLepelaars
Copy link
Contributor Author

Calmcode saves the day, as usual! 😁

@CarloLepelaars
Copy link
Contributor Author

Closing this to avoid introducing unnecessary complexity and slow down these encoders.

@CarloLepelaars CarloLepelaars deleted the feature/progress-bar branch November 16, 2022 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants