NVIDIA inference server docs need to be updated to not use ksonnet

[jlewi]

https://www.kubeflow.org/docs/components/serving/trtinferenceserver/

Docs still use ksonnet.

For serving, I think its simpler than rather than provide kustomize manifests, our docs just contain example YAML specs that people can copy and paste in order to create a deployment.

@pdmack Any interest in picking this up?

[issue-label-bot]

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

[sarahmaddox]

We've added a comment in the doc, saying that it still uses ksonnet and needs updating:
https://www.kubeflow.org/docs/components/serving/trtinferenceserver/#kubernetes-generation-and-deploy

It'd be good to get this updated before we archive the v0.6 branch of the docs.

@pdmack is this something you could take on?

[asispatra]

Do this document available for kustomize ?

[svnoesis]

We are trying to setup Nvidia TensorRT Inference Server Image on Kubernetes/Kubeflow. Looking at the readme instructions in the github, it talks about ksonnet.
https://github.com/kubeflow/kubeflow/tree/master/kubeflow/nvidia-inference-server#kubernetes-generation-and-deploy

Is there any updated steps for using kustomize for deploying nvidia-inference-server?
Could anyone please help ?

[jlewi]

@svnoesis Here's a sample Kubernetes spec (I generated it using the ksonnet commands) for deploying the NVIDIA inference server.

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: inference-server
    ksonnet.io/component: iscomp
  name: inference-server
  namespace: default
spec:
  ports:
  - name: http-inference-server
    port: 8000
    targetPort: 8000
  - name: grpc-inference-server
    port: 8001
    targetPort: 8001
  - name: metrics-inference-server
    port: 8002
    targetPort: 8002
  selector:
    app: inference-server
  type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: inference-server
    ksonnet.io/component: iscomp
  name: inference-server-v1
  namespace: default
spec:
  template:
    metadata:
      labels:
        app: inference-server
        version: v1
    spec:
      containers:
      - args:
        - trtserver
        - --model-store=gs://inference-server-model-store/tf_model_store
        image: nvcr.io/nvidia/inferenceserver:18.08.1-py2
        imagePullPolicy: IfNotPresent
        livenessProbe:
          httpGet:
            path: /api/health/live
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5
        name: inference-server
        ports:
        - containerPort: 8000
        - containerPort: 8001
        - containerPort: 8002
        readinessProbe:
          httpGet:
            path: /api/health/ready
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 5
        resources:
          limits:
            nvidia.com/gpu: 1
        securityContext:
          fsGroup: 1000
          runAsUser: 1000
      imagePullSecrets:
      - name: ngc

You should just modify that spec as needed to make it work for your use case; i.e change model-store to point to your model.

It would be great if someone would open up a PR updating the docs.
https://www.kubeflow.org/docs/components/serving/trtinferenceserver/