Application Insights Kubernetes failed to start.

[mfaizanse]

I integrated application insights kubernetes plugin in my ASP.NET core web api project and deployed it on a kubernetes, but there was no metrics/stats on app-insights dashboard related to kubernetes. Then I enabled self diagnostics for ApplicationInsights.Kubernetes (link) and checked the logs and found that it throws an exception and then fails to start:

PS C:\WINDOWS\system32> kubectl logs todoappcore-1886517599-52jfh
info: K8sEnvInitializer[0]
      ApplicationInsights.Kubernetes.Version:1.0.0-beta1
crit: Microsoft.ApplicationInsights.Kubernetes.K8sEnvironment[0]
      System.InvalidCastException: Can't figure out docker id. Input: 11:blkio:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      10:net_cls:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      9:pids:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      8:memory:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      7:freezer:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      6:cpuset:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      5:devices:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      4:hugetlb:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      3:cpu,cpuacct:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      2:perf_event:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      1:name=systemd:/kubepods/besteffort/podaf692797-a1d6-11e7-a192-00155da0364d/3a65f1c8605ac1121e9a7002e7319a6929d8dee4c29b582e86d01d1e64d93a32
      . Pattern: cpu.+docker/(.*)$
         at Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider.FetchContainerId(String pathToCGroup)
         at Microsoft.ApplicationInsights.Kubernetes.KubeHttpClientSettingsProvider..ctor(ILoggerFactory loggerFactory, String pathToToken, String pathToCert, String pathToNamespace, String pathToCGroup, String
kubernetesServiceHost, String kubernetesServicePort)
         at Microsoft.ApplicationInsights.Kubernetes.K8sEnvironment.<CreateAsync>d__11.MoveNext()
fail: K8sEnvInitializer[0]
      Application Insights Kubernetes failed to start.
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[3]
      Hosting starting
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[4]
      Hosting started
Hosting environment: Production
Content root path: /app
Now listening on: http://+:80
Application started. Press Ctrl+C to shut down.

I have tried on a minikube cluster (kubernetes 1.7.0) and also tried on kubernetes cluster 1.6.6 on Azure Container Service as well.

[mfaizanse]

Update: I tried it on kubernetes 1.5.3 ACS cluster and its working. It means that the plugin is not compatible with latest versions of kubernetes.

[brahmnes]

Cool that you found details on this problem. A quick inspection of the code says it's reading the file /proc/self/cgroup and finding the container id by the line with "cpu" and "docker" in it.
Can you compare /proc/self/cgroup between Kube 1.5.3 and Kube 1.6.6 to see how different they are?

[xiaomi7732]

#71 should have this fixed.