tf-serving how to add volumes/volumeMounts

[fboylu]

What is the suggested way to add new volumeMounts and volumes parameters to tf-serving deployments? I don't see these as optional parameters. Thank you.

[lluunn]

We don't support it now.
Could you describe your use case? Is it that the model is located at some local disk?

https://github.com/kubeflow/kubeflow/blob/master/kubeflow/tf-serving/tf-serving.libsonnet#L158
In our legacy template, there is a way to a NFS volume. Will that work for you?
#2321

[fboylu]

Correct, my model is hosted on blobfuse. any work arounds to try? Just need to add these
volumeMounts = [ {"name": "blob", "mountPath": /data},]
volumes = [
{
"name": "blob",
"flexVolume": {
"driver": "azure/blobfuse",
"readOnly": False,
"secretRef": {"name": "blobfusecreds"},
"options": {
"container": aks,
"tmppath": "/tmp/blobfuse",
"mountoptions": "--file-cache-timeout-in-seconds=120 --use-https=true",
},
},
},
]

[lluunn]

For now, you can modify your ks app directly to add the volume.

There are two ways:

1. edit the prototype directly, which should be in vendor/kubeflow/tf-serving/XXX.libsonnet in the ksonnet app. This will affect all of your tf-serving component in your ks app.

2. After generating a component, your should have a jsonnet as components/XXX.jsonnet in your ks app. edit the component, using ksonnet mixin. Example: https://github.com/kubeflow/kubeflow/blob/master/kubeflow/tf-serving/prototypes/tf-serving-gcp.jsonnet#L26-L57 This part is adding environment variable and volume. This won't affect other tf serving components in the ks app

[fboylu]

I am trying 1. above and modified the file. I am getting

tf-serving-template.libsonnet:120:31-36 Unknown variable: False

              "readOnly": False,

from the volumes I added. Any ideas what the problem would be?

[fboylu]

closed by mistake sorry, let me know if any ideas on the error. thanks.

[lluunn]

Try making it a string, "false"

[fboylu]

I got the below but then changed to ReadOnly instead of readOnly and it worked.
ERROR handle object: creating object: creating object: Deployment in version "v1beta1" cannot be handled as a Deployment: v1beta1.Deployment.Spec: v1beta1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Volumes: []v1.Volume: v1.Volume.VolumeSource: FlexVolume: v1.FlexVolumeSource.ReadOnly: ReadBool: expect t or f, but found ", error found in #10 byte of ...|eadOnly":"false","se|..., bigger context ...|ttps=true","tmppath":"/tmp/blobfuse"},"readOnly":"false","secretRef":{"name":"blobfusecreds"}},"name|...

However container is taking long time to stand up, still waiting on that, will let you know.

[fboylu]

I had an issue with secrets, it worked after resolving that. thank you.