Inference API wrapper client

[osanseviero]

This is still WIP, but I would like to get some feedback early on. We use the validation from the inference API to validate the inputs are ok.

Ideally this would be in a single call, but the API can expect both inputs and params (see zero-shot), so I added set_inputs and set_params methods. Let me know what you think

from huggingface_hub.inference_api import InferenceApi

# Mask filling (only inputs, string)
api = InferenceApi("bert-base-uncased")
print(api("The goal of life is [MASK]."))

# Zero-shot (input string, with params)
api = InferenceApi("typeform/distilbert-base-uncased-mnli")
inputs="Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!"
params = {
    "candidate_labels": ["refund", "legal", "faq"]
}
print(api(inputs, params))

# Question answering (input is dict, with params)
api = InferenceApi("deepset/roberta-base-squad2")
inputs={
    "question":"What's my name?",
    "context":"My name is Clara and I live in Berkeley."
}
print(api(inputs, params))

# Feature extraction by overriding pipeline tag (input is string)
api = InferenceApi("osanseviero/full-sentence-distillroberta2", task="feature-extraction")
inputs="This is an example again"
print(api(inputs))

# Error, missing context
api = InferenceApi("deepset/roberta-base-squad2")
print(api({"question":"What's my name?"}))

# Error, task does not exist
api = InferenceApi("bert-base-uncased", task="invalid_task")

@julien-c @Narsil FYI

[Narsil]

I'm fine with adding wrapper over the inference but as already said elsewhere, I am a bit concerned with maintenance of this as it could really go out of sync, as the groundtruth for transformers is really the pipelines part of transformers.

[julien-c]

looks neat, but do we actually want to perform validation client-side, vs. calling the API and displaying a clear validation error here?

[osanseviero]

My initial discussion with @LysandreJik led to the idea of client-side validation, but based on the comments, I think we should remove it.

I checked the returned values of the inference API when missing inputs and they are very explicit.

Benefits of client-side validation:

• We can throw errors early on (totally ok if we don't have)
• Users can easily find the inputs/params (fixable with good documentation and good error messages from server-side)

Cons of client-side validation

• Hard to maintain
• Incompatibility between transformers and some community pipelines (list of lists as inputs, for example).

Based on this and the discussion, I removed all client-side validation :)

[LysandreJik]

Thanks for the PR, this looks cool!

Regarding client-side validation, I am quite bullish on IntelliSense being a big component of the wrapper; but if it explodes complexity/maintainability then we can definitely overlook it in this first version and revisit if it makes sense later on.

[julien-c]

also tagging @cbensimon with whom we've discussed this wrapper recently

[LysandreJik]

As mentioned offline with @osanseviero, I think handling exceptions here is the most pythonic way to handle things - and that without such logic there's little value in having a Python wrapper for the inference API if it's a glorified cURL utility.

I would argue that the wrapper's ability to return appropriate errors such as raise ModelNotLoadedError or raise MissingKeyError, which inherit from the appropriate canonic errors (ValueError, OSError), is very important API-wise.

[LysandreJik]

Looking forward to the next PR to address the comments - agree to merge this now so that Flax users can use it for their spaces