Seldon Serving with Cognito

[dilzeem]

/kind bug

What steps did you take and what happened:
using: kfctl_aws_cognito.v1.0.0.yaml

I want to get some way to serve the models. I was going to use kf-serving but because of this issue, kserve/kserve#760 I am unable to progress unless a clear way forward is presented here.

So then I tried looking at Seldon-Core and followed along here: I followed the steps outlined here: https://www.kubeflow.org/docs/components/serving/seldon/

What did you expect to happen:

Most of the steps work, except I am not able to query the service.
The error that I get is:
Cannot POST /seldon/seldon-serve/seldon-model/api/v1.0/predictions

With the namespace of where i deployed the model seldon-serve

Anything else you would like to add:

From here https://www.kubeflow.org/docs/components/central-dash/overview/#using-kubectl-and-port-forwarding it seems to suggest that port forwarding will not work with any authentication.

Is there a way for me to get seldon or kf-serving working here with authentication?

[Miscellaneous information that will assist in solving the issue.]

Environment:

• Kubeflow version: dev_local
• kfctl version: kfctl v1.0-0-g94c35cf
• Kubernetes platform: AWS (EKS)
• Kubernetes version: 1.15
• OS: Linux (WSL)

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
kind/bug	0.85

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[Jeffwan]

/assign

[Jeffwan]

@dilzeem Can you try to query the service inside the mesh? I'd like to know if that's a real authentication issue or serving deployment issue.

I am working on programmatic access guidance for Cognito. Currently, you can do this way.

1. Use the cookie from your web login
   

2. Make request using that cookie from outside

curl --insecure -v -H "Host: ${SERVICE_HOSTNAME}" https://$CLUSTER_IP/v1/models/$MODEL_NAME:predict -d $INPUT_PATH --cookie "AWSELBAuthSessionCookie-0=xaD3ZVr8EEeCxWxxxxxxxx"

[dilzeem]

So I tried what you requested. Not sure if what I did was exactly correct.

This was the response:

* Trying XXX.2X.X1X.1XX...
* TCP_NODELAY set
* connect to XXX.2X.X1X.1XX port 443 failed: Connection refused
* Failed to connect to XXX.2X.X1X.1XX port 443: Connection refused
* Closing connection 0
curl: (7) Failed to connect to XXX.2X.X1X.1XX195 port 443: Connection refused

The $CLUSTER_IP that I used was the istio-ingressgateway cluster ip. The $SERVICE_HOSTNAME was the domain that I have set up for accessing kubeflow.

The original issue of following along with this: https://www.kubeflow.org/docs/components/serving/seldon/

has given me a different result after I followed the instructions exactly. The error that I get now after I have done the port forwarding is:

RBAC: access denied⏎

Which I guess means there is some authentication issue going on?

Just taking this further, ideally I want to be able to restrict access to who can query the models by providing some API keys (and have the model be public facing). This can be done with API gateway, but I am still unsure how to manage this with the current setup. I assume this is a thing that I will have to setup in istio-ingressgateway.

[Jeffwan]

RBAC: access denied is from istio, it's actually from authorization failure. AWS has a authorization layer, it retrieve user info from your token, and verify with your profile email. If it doesn't match, it will show this.

Trying to get more requirement on your ideal case. Do you want to managed API keys using API Gateway? Do you think Cognito or IDP user auth is good enough to make rest calls?

We actually has some discussion on this, it's not well designed yet. Your feedback is very important for next phase work

[dilzeem]

Okay. Is there anything else I can do with regards to the issue?
If it is something on my end, I can try have a look further if you are able to point me in the right direction.

With regards to our use case, API Gateway would be great for us to handle incoming request from the public in a secure manner. Giving out API keys to clients that will be using our service. It has a lot of other nice features, and allows us to expose our services with terraform.

I have used Cognito as an Authorizer in API Gateway, and I guess this could be one solution which I didn't think off.

I am not sure how to handle after the request has been authorized, but I should be able to figure it out. Though having fine grained controlled for a specific cognito user I will have to further investigate. Though currently with our current set up this won't be a requirement.

[Jeffwan]

Can you give the seldon manifest example? I can test in my env as well.

[jtfogarty]

/area seldon
/priority p1

[jbottum]

@jtfogarty is this in the proper area? @Jeffwan @dilzeem Hi Guys - Can this issue be closed in the Kubeflow 1.1 timeframe. I have downgraded it to a P2.

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
area/inference	0.71

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[dilzeem]

@jbottum we put using kubeflow within our org/team on hold until this is has been resolved. I am sure we can find ways around it, but we don't have the resources currently to investigate this further.

[dan-slinky-ckpd]

@Jeffwan We also operate a KF deployment with Cognito and are starting to look at Seldon. We can successful access the models if we grab our cookies from the browser but this is less than ideal. Is there any more information on a programatic access approach when using Cognito?

We're also going to be looking at tfserving, and I suspect we'll face the same issues?

[ukclivecox]

I think this is an issue with Dex related to dexidp/dex#764 ?
Or are you using IAP?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.