Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added loader from HF spaces #860

Merged
merged 4 commits into from
May 24, 2024
Merged

Added loader from HF spaces #860

merged 4 commits into from
May 24, 2024

Conversation

pawelknes
Copy link
Collaborator

Added a possibility to load data from Huggingface spaces as per #857

Signed-off-by: Paweł Knes <pawel.knes@ibm.com>
src/unitxt/loaders.py Outdated Show resolved Hide resolved
Copy link

codecov bot commented May 23, 2024

Codecov Report

Attention: Patch coverage is 87.50000% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 91.56%. Comparing base (4064804) to head (42ff0e4).
Report is 14 commits behind head on main.

Current head 42ff0e4 differs from pull request most recent head 4cad6dd

Please upload reports for the commit 4cad6dd to get more accurate results.

Files Patch % Lines
src/unitxt/loaders.py 83.33% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #860      +/-   ##
==========================================
- Coverage   92.06%   91.56%   -0.51%     
==========================================
  Files         104      104              
  Lines       10738    10977     +239     
==========================================
+ Hits         9886    10051     +165     
- Misses        852      926      +74     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@yoavkatz yoavkatz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Looks good. One request regarding token in the API.

Signed-off-by: Paweł Knes <pawel.knes@ibm.com>
Copy link
Member

@elronbandel elronbandel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done. Can you make sure it works like other loaders with loader_limit? If its too complicated we can discuss leaving it aside for now

@pawelknes
Copy link
Collaborator Author

Well done. Can you make sure it works like other loaders with loader_limit? If its too complicated we can discuss leaving it aside for now

I made the loader inherit from LoadHF since the only difference is that it needs to download the data first, and then it operates on HF dataset as well

@pawelknes pawelknes merged commit ccc0f95 into main May 24, 2024
7 checks passed
@pawelknes pawelknes deleted the loader-huggingface-space branch May 24, 2024 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants