Need help deploying simple model with tf-serving and enable Rest API

[fboylu]

Hi, I am having issues with the following.

VERSION=v0.4.0
ks pkg install kubeflow/tf-serving@${VERSION}

ks generate tf-serving tf-serving
ks generate tf-serving-service tf-serving-service

ks param set tf-serving-service modelName half_plus_two
ks param set tf-serving modelName half_plus_two
ks param set tf-serving modelPath /models/half_plus_two
ks param set tf-serving numGpus 1
ks param set tf-serving defaultGpuImage fboylu/tf_serving_gpu
ks param set tf-serving serviceType LoadBalancer

ks apply default -c tf-serving-service
ks apply default -c tf-serving

I have my model baked into the container and logs from tf-serving pod confirm it is serving (it works locally anyways).

I see the LoadBalancer external IP but the following times out
curl -d '{"instances": [1.0, 2.0, 5.0]}' -X POST http://EXTERNAL_IP/v1/models/half_plus_two:predict

Can you let me know what is missing/wrong?

thanks

[lluunn]

tf-serving is the legacy prototype, and we are removing it soon.
tf-serving-service should be used with tf-serving-deployment-gcp or tf-serving-deployment-aws

document:
https://www.kubeflow.org/docs/guides/components/tfserving_new/

[fboylu]

I am not usig gcp or aws, I have the model locally in the container. Can I still use tf-serving-deployment-gcp or tf-serving-deployment-aws?

[lluunn]

should be. The two prototypes just have some credential setup to read from gcp/aws.

[fboylu]

Ok, I tried it and seems like it now serves on 8500.

[jlewi]

@fboylu Is this working now for you? Can we close this issue?

[fboylu]

Yes it is working, please close the issue. Many thanks for the help.