More Information about adding libraries to HF collection

[qcbegin]

Hi HF maintainers!

I have some questions regarding the review and approval process for integrating an library to HF's library collection (https://huggingface.co/docs/hub/models-adding-libraries).

Could I know the review process timeline for an library-integration PR? How long does it usually take to get the PR approved and merged? I know that duration can vary depending on the responsiveness of applicants and the speed for dealing the comments, but is there an average waiting period for each reviewer to provide feedback?

Also, I noticed that previously, two reviewers were assigned to such reviews, but now it appears that up to six reviewers may be involved in reviewing the changes on related configuration files. Does the required num of reviewers differ depending on the nature of the integration, or is there a standard policy?

Besides, what are the main criteria when deciding whether to approve the integration application? It looks the usage and popularity of library counts.

Thank you for reading my questions.

[coyotte508]

cc @Vaibhavs10 @Wauplin

[julien-c]

which library are you looking to add @qcbegin?

[qcbegin]

Thanks for your prompt reply! Actually I’m part of a research team studying HF developer community and how HF facilitates open innovation by collaborating with third-party framework developers. I thought these questions might also be useful for applicants, so I posted them here.

[Wauplin]

Hey @qcbegin, best way to answer the process timeline to integrate libraries is to check the PRs in this list: https://github.com/huggingface/huggingface.js/commits/main/packages/tasks/src/model-libraries.ts. We aim at reviewing these PRs at least once a week so that the integrations are shipped continuously. Regarding the numbers of reviewers, usually 1 or 2 approvals are enough. We do not have a policy regarding which library should be integrated or not. As soon as at least 1 model on the Hub is tagged as "compatible with the library", we consider that it's ok to get it integrated officially.

what are the main criteria when deciding whether to approve the integration application?

Clean PRs are usually merged quickers (i.e. the ones following the docs https://huggingface.co/docs/hub/models-adding-libraries).

It looks the usage and popularity of library counts.

What made you think that? Is it from docs or a real case example?

[qcbegin]

Thanks for clarifying, @Wauplin! From your quick response, I can really see how committed HF is to building an open and vibrant AI/ML community.

I mentioned usage and popularity after seeing this example: huggingface/hub-docs#622. As you pointed out, it seems the main factor was just the limited number of models using that library.

Thanks again for your helpful response!

[Wauplin]

Regarding this specific PR, it was at a time where we did not have the same process to "officially support a library". At that time, the PR would have added snippets and documentation to the official HF docs. This is more commitment on our side as we need to make sure it's correct + maintain it. The process described in https://huggingface.co/docs/hub/models-adding-libraries allows any library author to add a nice UI + code snippets related to their libraries ONLY on the models tagged with the library name. Meaning that we do not end up with a gigantic doc page with all possible libraries documented one after the other (which was more the case before).

Hence why the filtering was more tighten before compared to now. And yes, we do believe it's important for us to do it that way as we don't want to assess the importance of potentially niche libraries (i.e. popular in a niche, but not widely known otherwise).