Support gRPC and HTTP protocols at the same time

[simon-cj]

According to following two code block:
https://github.com/SeldonIO/seldon-core/blob/master/operator/controllers/seldondeployment_controller.go#L502-L532
https://github.com/SeldonIO/seldon-core/blob/master/operator/controllers/seldondeployment_controller.go#L732-L735

it seems like that it is not support gRPC and HTTP protocols at the same time in noengine mode now.
now we are researching the tensorrt model server, it support gRPC and HTTP, so suggest it that seldon core can support it.

as results，we imagine it will update following logic:

• Set more port information in container Service.

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2020-08-31T04:44:04Z
  labels:
    app.kubernetes.io/managed-by: seldon-core
    seldon-app: savedmodel-seldon-savedmodel
    seldon-deployment-id: savedmodel-seldon
  name: savedmodel-seldon-savedmodel
  namespace: kleveross-system
  ownerReferences:
  - apiVersion: machinelearning.seldon.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: SeldonDeployment
    name: savedmodel-seldon
    uid: 9935e9cc-eb44-11ea-87e4-5254001948db
  resourceVersion: "1239727"
  selfLink: /api/v1/namespaces/kleveross-system/services/savedmodel-seldon-savedmodel
  uid: 9939e748-eb44-11ea-87e4-5254001948db
spec:
  clusterIP: 10.254.216.225
  ports:
  - name: http2
    port: 8000
    protocol: TCP
    targetPort: 8000
  - name: grpc
    port: 8001
    protocol: TCP
    targetPort: 8001
  selector:
    seldon-app: savedmodel-seldon-savedmodel
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

• Set more port information in predict Service.

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2020-08-31T04:44:04Z
  labels:
    app.kubernetes.io/managed-by: seldon-core
    default: "true"
    model: "true"
    seldon-app-svc: savedmodel-seldon-savedmodel-savedmodel
    seldon-deployment-id: savedmodel-seldon
  name: savedmodel-seldon-savedmodel-savedmodel
  namespace: kleveross-system
  ownerReferences:
  - apiVersion: machinelearning.seldon.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: SeldonDeployment
    name: savedmodel-seldon
    uid: 9935e9cc-eb44-11ea-87e4-5254001948db
  resourceVersion: "1239724"
  selfLink: /api/v1/namespaces/kleveross-system/services/savedmodel-seldon-savedmodel-savedmodel
  uid: 99388e43-eb44-11ea-87e4-5254001948db
spec:
  clusterIP: 10.254.27.0
  ports:
  - name: http
    port: 8000
    protocol: TCP
    targetPort: 8000
  - name: grpc
    port: 8001
    protocol: TCP
    targetPort: 8001
  selector:
    seldon-app-svc: savedmodel-seldon-savedmodel-savedmodel
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

• Set more port information in istio VirtualService.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  creationTimestamp: 2020-08-31T11:20:00Z
  generation: 4
  name: savedmodel-seldon
  namespace: kleveross-system
spec:
  gateways:
  - kleveross-gateway.istio-system
  hosts:
  - '*'
  http:
  - match:
    - headers:
        namespace:
          exact: kleveross-system
        seldon:
          exact: savedmodel-seldon
      uri:
        regex: .*nvidia\.inferenceserver.*
    route:
    - destination:
        host: savedmodel-seldon-savedmodel
        port:
          number: 8000
        subset: savedmodel
    - destination:
        host: savedmodel-seldon-savedmodel
        port:
          number: 8001
        subset: savedmodel

the main change block is Service.port part and VirtualService.route part.

what do you think about？ @cliveseldon

[ukclivecox]

We plan to support both externally see #2299
We will try to prioritise this for 1.4 release

[simon-cj]

We plan to support both externally see #2299
We will try to prioritise this for 1.4 release

OK, great

[simon-cj]

Have any roadmap or timeline for 1.4 release?