TF-Serving: How to add new Param to Service Prototype (running in Field does not exist)

[rummens]

Hi,

I am trying to add a new parameter to the Service Prototype of TF-Serving. I want to customize the port the HTTP service is exposing.

I have added the new optional param in the tf-serving service prototype:

// @apiVersion 0.1
// @name io.ksonnet.pkg.tf-serving-service
// @description TensorFlow serving
// @shortDescription A TensorFlow serving model
// @param name string Name to give to each of the components
// @optionalParam serviceType string ClusterIP The k8s service type for tf serving.
// @optionalParam modelName string null The model name
// @optionalParam trafficRule string v1:100 The traffic rule, in the format of version:percentage,version:percentage,..
// @optionalParam injectIstio string false Whether to inject istio sidecar; should be true or false.
// @optionalParam enablePrometheus string true Whether to enable prometheus endpoint (requires TF 1.11)
// @optionalParam portHttp string 8500 Port to use for tf-serving HTTP

local k = import "k.libsonnet";
local tfservingService = import "kubeflow/tf-serving/tf-serving-service-template.libsonnet";
local util = import "kubeflow/tf-serving/util.libsonnet";

local testvar = std.trace(params, 0);

tfservingService.new(env, params).all

I changed the tf-serving service template accordingly:

[...]
      spec: {
        ports: [
          {
            name: "grpc-tf-serving",
            port: 9000,
            targetPort: 9000,
          },
          {
            name: "http-tf-serving",
            port: params.portHttp,
            targetPort: 8500,
          },
        ],
        selector: {
          app: modelName,
        },
        type: params.serviceType,
      },
    },  // tfService
    tfService:: tfService,

[...]

After generating a new component the param is added:

ks param list test-service4
COMPONENT     PARAM            VALUE
=========     =====            =====
test-service4 enablePrometheus 'true'
test-service4 injectIstio      'false'
test-service4 modelName        'null'
test-service4 name             'test-service4'
test-service4 portHttp         '8500'
test-service4 serviceType      'ClusterIP'
test-service4 trafficRule      'v1:100'

But when trying to apply the component it fails with:

ERROR find objects: RUNTIME ERROR: Field does not exist: portHttp
        /home/NOVA/marcel.rummens/kubeflow/kfapp/ks_app/vendor/kubeflow/tf-serving/tf-serving-service-template.libsonnet:58:19-34       object <anonymous>
        During manifestation

Using the trace function, I can confirm that the param is there:
TRACE: <extvar:__ksonnet/components>:248 {"namespace": "kubeflow", "server": "https://airbusds-566384cc.hcp.northeurope.azmk8s.io:443"}

What am I doing wrong? Am I adding the wrong files or just using ks wrong?

Thank you very much
Marcel

[jlewi]

Check you params.libsonnet file. Is your parameter listed there?

Can you also do ks param list and provide the output?

[rummens]

Hello,
yes the params show up both in the file and using the command:

params.libsonnet:

"test-service4": {
      enablePrometheus: 'true',
      injectIstio: 'false',
      modelName: 'null',
      name: 'test-service4',
      portHttp: '8500',
      serviceType: 'ClusterIP',
      trafficRule: 'v1:100',
    }

ks param list test-service4
COMPONENT     PARAM            VALUE
=========     =====            =====
test-service4 enablePrometheus 'true'
test-service4 injectIstio      'false'
test-service4 modelName        'null'
test-service4 name             'test-service4'
test-service4 portHttp         '8500'
test-service4 serviceType      'ClusterIP'
test-service4 trafficRule      'v1:100'

[rummens]

I have no idea what caused this issue but I tried it again this morning after rebooting the machine and it works :-) I have created a Pull request for this because I think this could be a usefull feature: #2340