Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance AsyncClient for Flexible Session Management in Text Generation Client #1784

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

BenHaimItay
Copy link

@BenHaimItay BenHaimItay commented Apr 21, 2024

Updates the AsyncClient to efficiently manage sessions for both direct await usage and within async with blocks, improving flexibility for various async operation scenarios.

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Copy link
Collaborator

@Narsil Narsil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thanks for the fix. FWIW we're soft deprecating this client in favor of https://huggingface.co/docs/huggingface_hub/package_reference/inference_client (Which should have more or less the same features and more important has more broad support in the HF ecosystem).

@Narsil
Copy link
Collaborator

Narsil commented Apr 30, 2024

Seems the tests are not passing though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants