Endpoint authorization with API key

[Bunoviske]

Hello,

I have a multi-tenant application and I would like to control who have access to each endpoint with API Keys. That is still a bit unclear for me. How can I authorize users before they consume some endpoint?

This question also extends to serving engines in general, like TorchServe. How people are normally controlling access to the inference APIs?

Thanks,
Bruno

[jkhenning]

Hi @Bunoviske ,

Common practice is to protect these endpoints using some service that supports basic authentication (or JWT token authentication), placed before the endpoints.
Which multi-tenant application do you have? How do you generate credentials/tokens for this app?

[Bunoviske]

Hello @jkhenning,

In my application, every tenant has an endpoint and every endpoint has a different authentication token. For that, my own application generate tokens for each endpoint, when they are deployed. How to implement this with clearml-serving, any ideas?

Besides that, can you elaborate more on this common practice you wrote? Or do you have some links where I can get deeper in the topic?

Thank you!

[jkhenning]

Hi @Bunoviske,

If you already have a way to generate tokens (and a secret used to generate them), you would normally set up some layer in front of your endpoints (or the serving endpoints) that can parse any token provided with the call and determine if the call is allowed to reach the endpoint (or redirect to the appropriate endpoint). Common tools used for that are nginx (see here and here, for example) or envoy (see here for example)

[Bunoviske]

Thank you, that is very helpful. Maybe you could also suggest this as a feature for ClearML: BentoML secure endpoint or Integrate BentoML server with WSGI

For me, it would much easier if I could do clearml-serving --token "jsnkjsagn", something like this. Let me know your thoughts on that :)