Add pydantic support in response_format

[lhoestq]

• support passing a Pydantic schema in response_format in InferenceClient.chat_completion

class MyExampleResponseFormat(BaseModel):
    foo: str
    bar: str

response = client.chat_completion(messages=messages, response_format=MyExampleResponseFormat)

• added response.choices[0].message.parsed (and response.choices[0].message.refusal if the generation failed as in the openai client)

message = response.choices[0].message
if message.parsed:
    print(message.parsed)
else:
    print(message.refusal)

close #2646

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Wauplin]

As discussed, we'll have to see what we can do for this. Especially because ChatCompletionOutputMessage is an auto-generated class (also the stream version ChatCompletionStreamOutputMessage would have to be updated). And do the same for InferenceClient.text_generation(..., grammar=...).

[lhoestq]

Ok, note that this is mostly useful for the non-streaming case imo

[patrickvonplaten]

Big +1 on adding pydantic to both requests and responses. If HF becomes the gateway for many inference providers, it'd be super important to make the user experience with it as flawless as possible. To do so, I think both requests and responses objects should have very strict typing and validation that throw directly once the user makes a mistake => Pydantic is a must for this.

E.g.:

• huggingface_hub/src/huggingface_hub/inference/_generated/types/base.py

  Line 59 in 290aa26

  raise ValueError(f"Invalid input data for {cls}. Expected a list, but got {type(output)}.")

  => these manual checks are not needed when making this a Pydantic object
• There is a lot of validation missing at the moment (besides simple type checking):
  • Tools are very weakly typed:

    huggingface_hub/src/huggingface_hub/inference/_generated/types/chat_completion.py

    Line 70 in 290aa26

    class ChatCompletionInputFunctionDefinition(BaseInferenceType):

    => there should be more validations (e.g. arguments should be a dict)
  • Generally the whole message structure needs to be validated (e.g. there cannot be more tool answers than tool calls in history, there cannot be two assistant messages following each other, ....) => is this done?

Strict validation would also be very important here - otherwise people might waste quite some money on ill-formatted requests

[Wauplin]

@patrickvonplaten could you open a separate issue for that topic please? @lhoestq's initial request is simply that the response_format parameter from chat_completion accepts Pydantic model as well, not only a jsonschema.
Adding validation to all inputs and outputs is an orthogonal (and big!) topic so I'd prefer to have the discussion in a dedicated issue.