Add support for dedicated and serverless inference endpoints via inference API

[philschmid]

What does this PR do?

This PR currently adds a dirty implementation of how we could support dedicated Inference Endpoints and serverless Inference Endpoints via the Inference API.

On init we try to check if the provided "endpoint_name_or_model_id" (happy to revert back to endpoint_name) is available serverless using the list_deployed_models method.

example

import os

from distilabel.llm import InferenceEndpointsLLM
from distilabel.tasks import TextGenerationTask

token = os.getenv("HF_TOKEN")  # hf_...

llm = InferenceEndpointsLLM(
    "openchat/openchat-3.5-0106",
    token=token,
    task=TextGenerationTask(),
    max_new_tokens=512,
)


result = llm.generate([{"input": "What are critique LLMs?"}])
result

Note: didn't work on docs yet.

[Wauplin]

Made a quick review. Usage of InferenceClient/InferenceEndpoint looks good to me!

[plaguss]

Thank you very much @philschmid for your contribution and @Wauplin for the fast review 😄. Looks good, let me take a look at the docs, it shouldn't need many updates

[plaguss]

Could you please update the new variable name in this example and in the docs in this snippet? Also if you could install pre-commit, the errors from the tests should be solved.
Note: I updated the PR name.

[ignacioct]

Some minor remarks on code, but it looks really nice! I'll test the functionality in a bit and get back with more feedback.

[ignacioct]

Code's working on my side