Add support to detect Kubernetes platform in CloudPlatform

[markpollack]

In Spring Cloud Data Flow we would like to detect when an application is running in Kuberenetes.

One approach maybe to check for an environment variable such as KUBERNETES_SERVICE_HOST since k8s sets a property of this form - see https://github.com/kubernetes/kubernetes/blob/v1.11.0/pkg/kubelet/envvars/envvars.go#L45 .

I haven't yet seen a (service.Name) value different than KUBERNETES but I guess it is possible.

https://github.com/spring-cloud/spring-cloud-kubernetes/blob/master/spring-cloud-kubernetes-core/src/main/java/org/springframework/cloud/kubernetes/StandardPodUtils.java

provides a isInsideKubernetes method but it requires a KubernetesClient to be created and I actually don't think it isn't answering the question if the code is running inside of a kubernetes pod.

Given that Kubernetes is quite a popular platform adding some support for detecting this would be useful.

[geoand]

@markpollack The code you are pointing does the best it can to determine whether the code is running inside a pod on Kubernetes without first calling the API server.
So it just checks for the standard files that Kubernetes injects into every container and also the HOSTNAME env var

[markpollack]

If the implementation turns out to use a throwaway KubernetesClient, then Boot has to bring in a dependency on that library. I thought it could be as simple an an env-var check perhaps (as is the case in the other cloud providers). The HOSTNAME check doesn't seem like it would be too useful to distinguish if it is running on k8s, since pretty much every machine is going to have HOSTNAME set.

[geoand]

Ah sorry, I forgot what was actually being done...

It seems like the api server is being contacted if the expected files that kubernetes injects are found.
The api server is contacted to make sure the name of the pod is what the env var says

[wilkinsona]

We discussed something similar in a Spring Cloud Kubernetes pull request where concerns were raised about making a network call during condition evaluation. I still have those concerns and still believe it is not something that a condition should do.

If there's a way to accurately detect Kubernetes based on certain files being present in well-known locations then we may be able to do something. We'd need to be careful to align with whatever the state of the art is in Spring Cloud Kubernetes.

[spencergibb]

Spring Cloud Kubernetes is doing work in the bootstrap phase and sets values in the environment based on that work. Then those values can be used in conditions

[markpollack]

But isn't SCK is making network connections? I think the env-var as set by k8s here - https://github.com/kubernetes/kubernetes/blob/v1.11.0/pkg/kubelet/envvars/envvars.go#L45 will 99% of the time be KUBERNETES_SERVICE_HOST but for K8s checking via Boot's CloudPlatform an additional parameter could be passed in by the user in case the service name of kubernetes itself isn't "kubernetes"

[wilkinsona]

Perhaps we can use the first three of the four bullets describing how the Java client figures out the best possible ApiClient for your environment? The implementation is in ClientBuilder.

[markpollack]

That approach would not work as it is more geared toward figuring out if the client has the appropriate configuration to make an API call vs. determining if the app is running inside a kubernetes pod.

*   <li>If $KUBECONFIG is defined, use that config file.
*   <li>If $HOME/.kube/config can be found, use that.

quite common to have $HOME/.kube/config on our local dev box for development with k8s due to use of the kubectl command line app. Same for $KUBECONFIG. These are also not going to be set/available on the app running in the pod afaik. See https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/

As for the 3rd option, I don't think the docker image for my app is going to have that file or env-var set and not sure either about the presence of a file called /ca.crt, seems brittle.

The approach of using and env-var set in https://github.com/kubernetes/kubernetes/blob/v1.11.0/pkg/kubelet/envvars/envvars.go#L45 seems to me like the best way. we can look directly for an env-var named KUBERNETES_SERVICE_HOST and if that is not found look for an env-var that matches the _SERVICE_HOST suffix.

[wilkinsona]

@markpollack and I discussed this during our face-to-face in New York. The environment variable-based approach is the way to go.

I have a slight concern about false positives triggered by finding a variable that ends in _SERVICE_HOST, but I think we could largely mitigate that concern by looking for a matching _SERVICE_PORT variable. For example, if we find EXAMPLE_SERVICE_HOST we'd then check that there's an EXAMPLE_SERVICE_PORT variable too.