Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve client.zero_shot_classification() #2340

Merged

Conversation

MoritzLaurer
Copy link
Contributor

@MoritzLaurer MoritzLaurer commented Jun 14, 2024

This PR does 3 things:

  1. It adds support for the hypothesis_template argument from the 0-shot pipeline, which is important for customizing 0-shot classification
  2. It removes the requirement for having at least 2 labels in the label list. This wasn't really necessary and there are cases where checking just one label with multi_label=True is useful.
  3. It updates the doc strings and adds an example

Tested it with a HF endpoint running the model deberta-v3-large-zeroshot-v2

Example for use with hypothesis_template:

from huggingface_hub import InferenceClient

client = InferenceClient()

output = client.zero_shot_classification(
    model="https://h9qyt7jenlitt7j6.us-east-1.aws.endpoints.huggingface.cloud",
    text="I really like our dinner and I'm very happy. I don't like the weather though.",
    labels=["positive", "negative", "pessimistic", "optimistic"],
    multi_label=True,
    hypothesis_template="This text is {} towards the weather"
)

# output:
ZeroShotClassificationOutputElement(label='negative', score=0.9231801629066467),
ZeroShotClassificationOutputElement(label='pessimistic', score=0.8760990500450134),
ZeroShotClassificationOutputElement(label='optimistic', score=0.0008674879791215062),
ZeroShotClassificationOutputElement(label='positive', score=0.0005250611575320363)

Example for use with single label:

client = InferenceClient()

output = client.zero_shot_classification(
    model="https://h9qyt7jenlitt7j6.us-east-1.aws.endpoints.huggingface.cloud",
    text="I really like our dinner and I'm very happy. I don't like the weather though.",
    labels=["positive"],
    multi_label=True,
    hypothesis_template="This text is {} towards the weather"
)

# output
[ZeroShotClassificationOutputElement(label='positive', score=0.0001598795352037996)]

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 3 changes looks good to me. Thanks @MoritzLaurer for improving these!

I left a few minor comments. Could you also run make style locally and push the changes? Doing so is supposed to fix styling issues (e.g. line to long) + generate the exact same code for the _async client. Thanks in advance!

src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
@MoritzLaurer
Copy link
Contributor Author

Thanks for the review @Wauplin ! I've applied your suggestions and ran make style.
One weird thing is that running python utils/generate_async_inference_client.py --update removes references to async from the async client docstring (see this commit). I've first added references to async manually, but then the checks didn't pass anymore. Just flagging this in case that's unintended

Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating @MoritzLaurer! Left a couple more comments. About the example issue, I will have a look at it. Don't worry about manually changing _async then. Just apply make style and I'll take care of it. Thanks!

src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Show resolved Hide resolved
src/huggingface_hub/inference/_client.py Outdated Show resolved Hide resolved
kallewoof and others added 16 commits July 3, 2024 10:17
…o Korean (huggingface#2311)

* docs: ko: package_reference/environment_variables.md

* feat: nmt draft

* feat: manual translate

* fix: manual edits

* fix: a minor typo

* fix: added manual translate

---------

Co-authored-by: Lucain <lucain@huggingface.co>
…uggingface#2335)

* Do not raise on .resume() if Inference Endpoints is already running

* make style

* fix test
…an (huggingface#2344)

* docs: ko: package_reference/webhooks_server

* feat: nmt draft

* fix: manual edits
* Support resource_group_id in create_repo

* docs

* as tip

* typoe
…ngface#2329)

* Translated toctree & git_vs_htttp

* Translated the cli guide

* Testing to see if user is well configured

* TEsting to see if branch is well configured

* Translated index

* Translated quick-start

* Corrected a minor error in the english doc

* Reverting a wrong change

* Translated installation.md

* Translated community.md

* Translated download.md

* Translated hf_file_system.md

* Translated inference_client

* Translated inference_endpoints.md

* Fixed a few errors in the translation

* Translated integrations.md

* Translated mana-cache.md. Did not find a good translfation for cache-system

* Translated manage-space.md

* Translated overview.md

* Translated package-reference file

* Translated hf_api, login, mixins, repository, space_runtime and tensorboard in the package reference

* Finished the translation of package reference by translating utilities and webhooks servers

* Translated model-cards.md

* Translated collections.md

* translated community.md

* Translated download.md, corrected an error in the english version and changed a n inacurrate translation in other files

* translated hf_file_system

* Translated inference_endpoints.md

* Translated inference.md

* translated repository.md

* Translated search.md

* Translated webhooks_server.md

* corrected webhooks

* Translated upload.md

* Added "fr" to .github/workflow

* Deleted redirects as it is not needed

* Corrected toctree.yml

* Corrected errors o quick-start.md

* Corrected installation.md

* Changed the translation of the product "space" in the whole documentation.

* Corrected git_vs_http

* Corrected CLI.md

* Corrected collections.md

* Corrected community.md

* Corrected download.md

* Corrected hf_file_system

* Correction of docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Corrected git vs http and index.md

* Finished correcting index.md

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/installation.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Finished correcting index.md

* Corrected inference endpoint, a lot of mistakes might have to look at it again

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/concepts/git_vs_http.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Deleted files

* Revert "Deleted files"

This reverts commit 34c49a7.

* Deleted files

* Deleted files

* Delete files

* Update docs/source/fr/_toctree.yml

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

* Update docs/source/fr/guides/integrations.md

Co-authored-by: Lucain <lucainp@gmail.com>

---------

Co-authored-by: JibrilEl <jibril.elhassani@gmail.com>
Co-authored-by: Lucain <lucainp@gmail.com>
* faster fs walk

* add missing args
* add proxy support on async client

* modifications suggested

* fix script

* fix test

---------

Co-authored-by: Noe_Chabanon <noe.chabanon@partnre.com>
Co-authored-by: Lucain Pouget <lucainp@gmail.com>
…e#2347)

* docs: ko: guides/manage-cache.md

* feat: nmt draft

* Update docs/source/ko/_toctree.yml

* Apply suggestions from code review

Co-authored-by: Harheem Kim <49297157+harheem@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Kyungmin Jeon <83350060+seoulsky-field@users.noreply.github.com>
Co-authored-by: Harheem Kim <49297157+harheem@users.noreply.github.com>

---------

Co-authored-by: Harheem Kim <49297157+harheem@users.noreply.github.com>
Co-authored-by: Kyungmin Jeon <83350060+seoulsky-field@users.noreply.github.com>
Co-authored-by: Lucain <lucain@huggingface.co>
* Fix updat eruff to 0.5.0 in dev

* setup.py
* Promote chat_completion in inference guide

* fix KO docs

* update example + list conversational models

* feedback

* fix KO?

* fix fix KO

* ko

* fix fix fix
)

* Add  to feature-extraction + update types

* add truncation_direction to feature-extraction
Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thanks @MoritzLaurer for taking the time to also improve the docstrings :)

Some tests are failing but it's unrelated to your PR. I fixed the script that generates the _async code to properly transcribed the async examples (see f479f08).

@MoritzLaurer MoritzLaurer merged commit 79f51dc into huggingface:main Jul 3, 2024
13 of 14 checks passed
@MoritzLaurer
Copy link
Contributor Author

great, thanks a lot for your review @Wauplin !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet