Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add model info JSON files for 2.7k models #54

Merged
merged 3 commits into from
May 11, 2023
Merged

Conversation

viswavi
Copy link
Collaborator

@viswavi viswavi commented May 11, 2023

Description

Add model info (description, # of downloads, model size in bytes), extracted using @zhaochenyang20's script (which will be added in a subsequent PR).

We fetch all text generation and text2text generation models from HuggingFace. For all PyTorch-based models with a nonempty README.md file and a nonempty pytorch_model.bin file (or similar file), we store the following pieces of information in a json file:

  1. HuggingFace model name
  2. model description
  3. # of downloads
  4. model size (in bytes)

References

N/A

Blocked by

N/A

Copy link
Collaborator

@zhaochenyang20 zhaochenyang20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@viswavi viswavi changed the title [wip] Add model info JSON files for 1k models Add model info JSON files for 1k models May 11, 2023
@zhaochenyang20 zhaochenyang20 merged commit fa7e47e into main May 11, 2023
@zhaochenyang20 zhaochenyang20 deleted the vijay_add_modelinfos branch May 11, 2023 11:22
@zhaochenyang20 zhaochenyang20 mentioned this pull request May 13, 2023
11 tasks
@viswavi viswavi changed the title Add model info JSON files for 1k models Add model info JSON files for 2.7k models May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants