API Key Authentication for Michaelfeil Infinity

[AjayKarma05]

Model description

Could you please provide guidance on how to enable API_KEY authentication? Alternatively, is there a plan to implement API_KEY authentication similar to OpenAI's approach?

Open source status

• The model implementation is available on transformers
• The model weights are available on huggingface-hub
• I verified that the model is currently not running in infinity

Provide useful links for the implementation

No response

[semoal]

It's just a fastapi server, implement it as usual like you do on FastAPI.

[michaelfeil]

I added an example on how to use infinity with runpod.io

note, that API_Keys are typically not set, they are generated. Also they are not verified by the application itself, but by e.g. AWS API Gateway. I understand ypur interest, but i think such a feature is easy to implement yourself or better be added with another service

[michaelfeil]

monotykamary@36680ae

@monotykamary has implemented it. Changed my mind: If the contribution is easy enough, comes with a unit test that covers every LOC, and defaults to no api key as default behauviour, would be accepted

[monotykamary]

Oh, I just did a dirty implementation to quickly spin up a quick embedding server with auth on Modal: https://github.com/dwarvesf/llm-hosting/blob/main/infinity_snowflake_arctic_embed_l_335m.py

[michaelfeil]

Looks not dirty at all @monotykamary! Awesome

[michaelfeil]

@monotykamary @semoal @AjayKarma05 Added auth, you can set a Bearer token via INFINITY_API_KEY=mykey123 or --api-key mykey123

[Jimmy-Newtron]

How to set the API_KEY in the Langchain Infinity Embedding?

[michaelfeil]

@Jimmy-Newtron PRs to langchain are welcome. I think this feature is missing there.