ci-cd demo uses old jenkins and kubectl fails on newer k8s

[ryandawsonuk]

The kubectl binary available to the jenkins image used in https://github.com/SeldonIO/seldon-core/tree/master/examples/cicd-argocd is old now. The result on a newer k8s now is that you get this error when the job for building images tries to use kubectl:

error: SchemaError(io.k8s.api.core.v1.ConfigMapNodeConfigSource): invalid object doesn't have additional properties
make: *** [Makefile.build-model-images-job:8: do_build] Error 1

I saw this and added a line to print the kubectl version, which output:

/bin/kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:55:54Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}

So the issue is https://stackoverflow.com/a/55581050/9705485

I think we need to update the jenkins being used

Might have to make changes like https://github.com/SeldonIO/seldon-core/pull/506/files#diff-b0f64032faabe39ddbcfd9a55b142519R14 for jenkins (that one is argocd)

[ryandawsonuk]

We could also try to add the github plugin via the chart to avoid installing it manually:

https://github.com/helm/charts/blob/master/stable/jenkins/values.yaml#L138

[ryandawsonuk]

We may also be able to get the jenkins to use local identity store out of the box too (rather than having to set that manually). I think that's a choice of realm in jenkins terminology. See https://github.com/helm/charts/blob/master/stable/jenkins/values.yaml#L21 and https://github.com/jenkins-x/cloud-environments/blob/4010385ead7bc7a1816cb6d0fb540878dd888f78/env-minishift/myvalues.yaml#L56

A way to find the configuration might be to make the changes in the UI and find what has changed in the config.xml file

[ryandawsonuk]

Actually turns out the kubectl binary needing updating was in the seldon-deployer image. But updating jenkins chart allows us to specify plugins and simplify the demo setup a bit.

[ukclivecox]

Fixed.