SendonDeployment with name > 31 characters

[sasvaritoni]

seldonio/cluster-manager:0.2.6

When I try to create a seldondeployment with a name of lenght 32 or more, it will go to error state:

"name": "longname30longname30longname3012"
...
    "status": {
        "description": "Failed to create deployment seldon-6cbb0f8",
        "state": "Failed"
    }

In the logs of Seldon cluster manager:

2019-05-08 16:43:34.134 ERROR 1 --- [pool-1-thread-1] i.s.c.k.SeldonDeploymentControllerImpl   : Error creating deployment:{
  "metadata": {
    "selfLink": "",
    "resourceVersion": ""
  },
  "status": "Failure",
  "message": "Deployment.apps \"seldon-6cbb0f8\" is invalid: [metadata.labels: Invalid value: \"longname30longname30longname3012-longname30longname30longname3012\": must be no more than 63 characters, spec.selector.matchLabels: Invalid value: \"longname30longname30longname3012-longname30longname30longname3012\": must be no more than 63 characters, spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"seldon-app\":\"longname30longname30longname3012-longname30longname30longname3012\", \"seldon-app-longname30longname30longname3012\":\"seldon-vmx-vmx171-5000-img-vgg16-v0-0-1-4961034\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: invalid label selector.]",
  "reason": "Invalid",
  "details": {
    "name": "seldon-6cbb0f8",
    "group": "apps",
    "kind": "Deployment",
    "causes": [{
      "reason": "FieldValueInvalid",
      "message": "Invalid value: \"longname30longname30longname3012-longname30longname30longname3012\": must be no more than 63 characters",
      "field": "metadata.labels"
    }, {
      "reason": "FieldValueInvalid",
      "message": "Invalid value: \"longname30longname30longname3012-longname30longname30longname3012\": must be no more than 63 characters",
      "field": "metadata.labels"
    }, {
      "reason": "FieldValueInvalid",
      "message": "Invalid value: \"longname30longname30longname3012-longname30longname30longname3012\": must be no more than 63 characters",
      "field": "spec.selector.matchLabels"
    }, {
      "reason": "FieldValueInvalid",
      "message": "Invalid value: v1.LabelSelector{MatchLabels:map[string]string{\"seldon-app-longname30longname30longname3012\":\"seldon-vmx-vmx171-5000-img-vgg16-v0-0-1-4961034\", \"seldon-app\":\"longname30longname30longname3012-longname30longname30longname3012\"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: invalid label selector.",
      "field": "spec.selector"
    }],
    "retryAfterSeconds": 0,
    "uid": ""
  },
  "code": 422
}

2019-05-08 16:43:34.135 ERROR 1 --- [pool-1-thread-1] i.s.c.k.SeldonDeploymentControllerImpl   : Failed to create deployment 

io.seldon.clustermanager.k8s.SeldonDeploymentException: Failed to create deployment seldon-6cbb0f8
	at io.seldon.clustermanager.k8s.SeldonDeploymentControllerImpl.createDeployments(SeldonDeploymentControllerImpl.java:94) ~[classes!/:0.2.6]
	at io.seldon.clustermanager.k8s.SeldonDeploymentControllerImpl.createOrReplaceSeldonDeployment(SeldonDeploymentControllerImpl.java:306) ~[classes!/:0.2.6]
	at io.seldon.clustermanager.k8s.SeldonDeploymentWatcher.processWatch(SeldonDeploymentWatcher.java:85) [classes!/:0.2.6]
	at io.seldon.clustermanager.k8s.SeldonDeploymentWatcher.watchSeldonMLDeployments(SeldonDeploymentWatcher.java:196) [classes!/:0.2.6]
	at io.seldon.clustermanager.k8s.SeldonDeploymentWatcher.watch(SeldonDeploymentWatcher.java:228) [classes!/:0.2.6]
	at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) ~[na:na]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_181]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_181]
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) [spring-context-4.3.20.RELEASE.jar!/:4.3.20.RELEASE]
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-4.3.20.RELEASE.jar!/:4.3.20.RELEASE]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_181]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_181]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_181]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]

When the name has length of 30 characters, it works fine.
It seems that the deployment name is actually repeated twice in that metadata, so the real limitation on length this way is 31 characters, not 63.

Is this a bug? Or am I doing something wrong?

Full json:

kubectl get seldondeployment longname30longname30longname3012 -o json
{
    "apiVersion": "machinelearning.seldon.io/v1alpha2",
    "kind": "SeldonDeployment",
    "metadata": {
        "annotations": {
            "ksonnet.io/managed": "{\"pristine\":\"H4sIAAAAAAAA/8xSwU7cQAy99zN8TpcERCvlDKoqlXbVoF4qtDIzJnGZsUczk4gVyr9Xk8AW1B7YW2+W7ff8nu1HwMA/KCZWgRY8moGFHGEUln6TyFmVDevJ1KALA55CBfcsFlroltoFBad7T5KhAk8ZLWaE9hEc3pJLJcIQoIWVqsCTilAupEZ9UCnYFpxKL+jprP5X1JzCXEGJ39K6dqaAprT7ByrgFMgsckQ0Y2aVRZ09GNhNhz1MDVQQov4ik3dvnXqUwBDJsskaE7Q//xJ1qL7WNFdwWFkXyKzYZ2NGJSMLPVGyx76ImfzDeyJsPjbteV3XJ+z73dT3zYd2qjf1pjhdWrejc1t1bPbQwue7r5q3kdJ6WMcTCaW0jXpLZdYdshsjXQ+R0qDOQntWwYBiHcVSzyZ0au4pL240lgsPOQeY5wpYODO6C3K478io2ARtU9cVBIqs9pA7ryCNxlBKLwY1FWT2pGP+gz1q9ZHQ8n/uZlI3errSUXK55s188+JKHZlIa76CTNGzLK/zKaKh7euhzTPXSlN4+ohhWB5mYGcjyUpEYoOyLB7zPpRlfr/sro976yfg1beLyy/HISMFxwYX/zfz/O43AAAA//8BAAD//078TreaBAAA\"}",
            "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"machinelearning.seldon.io/v1alpha2\",\"kind\":\"SeldonDeployment\",\"metadata\":{\"name\":\"longname30longname30longname3012\",\"namespace\":\"mmx\",\"selfLink\":\"/apis/machinelearning.seldon.io/v1alpha2/namespaces/mmx/seldondeployments/longname30longname30longname3012\",\"uid\":\"66740d40-71b0-11e9-8667-0050569a816e\",\"generation\":1,\"creationTimestamp\":\"2019-05-08T16:43:25Z\",\"labels\":{\"app\":\"seldon\",\"app.kubernetes.io/deploy-manager\":\"ksonnet\",\"ksonnet.io/component\":\"longname30longname30longname3012\"},\"annotations\":{\"ksonnet.io/managed\":\"{\\\"pristine\\\":\\\"H4sIAAAAAAAA/8xSwU7cQAy99zN8TpcERCvlDKoqlXbVoF4qtDIzJnGZsUczk4gVyr9Xk8AW1B7YW2+W7ff8nu1HwMA/KCZWgRY8moGFHGEUln6TyFmVDevJ1KALA55CBfcsFlroltoFBad7T5KhAk8ZLWaE9hEc3pJLJcIQoIWVqsCTilAupEZ9UCnYFpxKL+jprP5X1JzCXEGJ39K6dqaAprT7ByrgFMgsckQ0Y2aVRZ09GNhNhz1MDVQQov4ik3dvnXqUwBDJsskaE7Q//xJ1qL7WNFdwWFkXyKzYZ2NGJSMLPVGyx76ImfzDeyJsPjbteV3XJ+z73dT3zYd2qjf1pjhdWrejc1t1bPbQwue7r5q3kdJ6WMcTCaW0jXpLZdYdshsjXQ+R0qDOQntWwYBiHcVSzyZ0au4pL240lgsPOQeY5wpYODO6C3K478io2ARtU9cVBIqs9pA7ryCNxlBKLwY1FWT2pGP+gz1q9ZHQ8n/uZlI3errSUXK55s188+JKHZlIa76CTNGzLK/zKaKh7euhzTPXSlN4+ohhWB5mYGcjyUpEYoOyLB7zPpRlfr/sro976yfg1beLyy/HISMFxwYX/zfz/O43AAAA//8BAAD//078TreaBAAA\\\"}\"},\"clusterName\":\"\"},\"spec\":{\"name\":\"longname30longname30longname3012\",\"predictors\":[{\"name\":\"longname30longname30longname3012\",\"graph\":{\"name\":\"longname30longname30longname3012\",\"type\":\"MODEL\",\"endpoint\":{\"type\":\"REST\"}},\"componentSpecs\":[{\"spec\":{\"containers\":[{\"name\":\"longname30longname30longname3012\",\"image\":\"vmx-vmx171:5000/img_vgg16:v0.0.1\",\"livenessProbe\":{\"handler\":{\"tcpSocket\":{\"port\":\"http\"}},\"initialDelaySeconds\":100,\"timeoutSeconds\":1,\"periodSeconds\":5,\"successThreshold\":1,\"failureThreshold\":3},\"readinessProbe\":{\"handler\":{\"tcpSocket\":{\"port\":\"http\"}},\"initialDelaySeconds\":100,\"timeoutSeconds\":1,\"periodSeconds\":5,\"successThreshold\":1,\"failureThreshold\":3},\"imagePullPolicy\":\"IfNotPresent\"}],\"terminationGracePeriodSeconds\":1}}],\"replicas\":1,\"annotations\":{\"predictor_version\":\"v1\"}}],\"annotations\":{\"deployment_version\":\"v1\",\"project_name\":\"longname30longname30longname3012\"}},\"status\":{\"state\":\"Failed\",\"description\":\"Failed to create deployment seldon-6cbb0f8\"}}\n"
        },
        "clusterName": "",
        "creationTimestamp": "2019-05-08T16:43:25Z",
        "generation": 1,
        "labels": {
            "app": "seldon",
            "app.kubernetes.io/deploy-manager": "ksonnet",
            "ksonnet.io/component": "longname30longname30longname3012"
        },
        "name": "longname30longname30longname3012",
        "namespace": "mmx",
        "resourceVersion": "28705471",
        "selfLink": "/apis/machinelearning.seldon.io/v1alpha2/namespaces/mmx/seldondeployments/longname30longname30longname3012",
        "uid": "66740d40-71b0-11e9-8667-0050569a816e"
    },
    "spec": {
        "annotations": {
            "deployment_version": "v1",
            "project_name": "longname30longname30longname3012"
        },
        "name": "longname30longname30longname3012",
        "predictors": [
            {
                "annotations": {
                    "predictor_version": "v1"
                },
                "componentSpecs": [
                    {
                        "spec": {
                            "containers": [
                                {
                                    "image": "vmx-vmx171:5000/img_vgg16:v0.0.1",
                                    "imagePullPolicy": "IfNotPresent",
                                    "livenessProbe": {
                                        "failureThreshold": 3,
                                        "handler": {
                                            "tcpSocket": {
                                                "port": "http"
                                            }
                                        },
                                        "initialDelaySeconds": 100,
                                        "periodSeconds": 5,
                                        "successThreshold": 1,
                                        "timeoutSeconds": 1
                                    },
                                    "name": "longname30longname30longname3012",
                                    "readinessProbe": {
                                        "failureThreshold": 3,
                                        "handler": {
                                            "tcpSocket": {
                                                "port": "http"
                                            }
                                        },
                                        "initialDelaySeconds": 100,
                                        "periodSeconds": 5,
                                        "successThreshold": 1,
                                        "timeoutSeconds": 1
                                    }
                                }
                            ],
                            "terminationGracePeriodSeconds": 1
                        }
                    }
                ],
                "graph": {
                    "endpoint": {
                        "type": "REST"
                    },
                    "name": "longname30longname30longname3012",
                    "type": "MODEL"
                },
                "name": "longname30longname30longname3012",
                "replicas": 1
            }
        ]
    },
    "status": {
        "description": "Failed to create deployment seldon-6cbb0f8",
        "state": "Failed"
    }
}

[ukclivecox]

This is probably a bug. We hash the core deployment names if the name is too long. But it looks like there is a leak into the labels metadata.

[ukclivecox]

Fixed by SeldonIO/seldon-operator#5