Generated SSL certs issues

[tudvari]

Hi Guys,

I would like to ask a little help. I tried to provision a Kubernetes cluster with the acs-engine CLI tool. The tool siad provision is successfully ended.

 kubectl get pods --namespace=kube-system
NAME                                            READY     STATUS    RESTARTS   AGE
heapster-v1.2.0-194960081-8xrxh                 0/2       Pending   0          8m
kube-addon-manager-k8s-master-39229988-0        1/1       Running   0          8m
kube-apiserver-k8s-master-39229988-0            1/1       Running   0          9m
kube-controller-manager-k8s-master-39229988-0   1/1       Running   0          9m
kube-dns-v19-dmvvc                              0/3       Pending   0          8m
kube-dns-v19-hnrqp                              0/3       Pending   0          8m
kube-proxy-7mqm5                                1/1       Running   0          8m
kube-proxy-ebtw9                                1/1       Running   0          8m
kube-proxy-lxlbu                                1/1       Running   0          8m
kube-proxy-r5bez                                1/1       Running   0          5m
kube-scheduler-k8s-master-39229988-0            1/1       Running   0          9m
kubernetes-dashboard-1872324879-kqdsy           0/1       Pending   0          8m

 sudo docker ps
CONTAINER ID        IMAGE                                                    COMMAND                  CREATED             STATUS              PORTS               NAMES
176cbfdf5235        gcr.io/google_containers/hyperkube-amd64:v1.4.5          "/hyperkube proxy --k"   11 minutes ago      Up 11 minutes                           k8s_kube-proxy.6534244d_kube-proxy-lxlbu_kube-system_a62e7361-abd5-11e6-80f0-000d3a2613fc_4ae5ff30
25f91309aa05        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 11 minutes ago      Up 11 minutes                           k8s_POD.d8dbe16c_kube-proxy-lxlbu_kube-system_a62e7361-abd5-11e6-80f0-000d3a2613fc_1cbcf764
0243775c437b        gcr.io/google_containers/kube-addon-manager-amd64:v5.1   "/opt/kube-addons.sh"    11 minutes ago      Up 11 minutes                           k8s_kube-addon-manager.ed858faf_kube-addon-manager-k8s-master-39229988-0_kube-system_c0133a504dee133427d4802c1f2c3314_8e1e67dc
72ca17c7edb0        gcr.io/google_containers/hyperkube-amd64:v1.4.5          "/hyperkube scheduler"   11 minutes ago      Up 11 minutes                           k8s_kube-scheduler.22257f8_kube-scheduler-k8s-master-39229988-0_kube-system_6203373493987263d369756729453b5f_9bf5a243
1119a6276383        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 11 minutes ago      Up 11 minutes                           k8s_POD.d8dbe16c_kube-scheduler-k8s-master-39229988-0_kube-system_6203373493987263d369756729453b5f_79ff11bf
158391dcd7cc        gcr.io/google_containers/hyperkube-amd64:v1.4.5          "/hyperkube controlle"   11 minutes ago      Up 11 minutes                           k8s_kube-controller-manager.954cbc53_kube-controller-manager-k8s-master-39229988-0_kube-system_ee5fb6e3d925965b0048e6cc77534a6e_e0ba6b18
af6c5c83eeef        gcr.io/google_containers/hyperkube-amd64:v1.4.5          "/hyperkube apiserver"   11 minutes ago      Up 11 minutes                           k8s_kube-apiserver.e54c022a_kube-apiserver-k8s-master-39229988-0_kube-system_1b3fae831a29391607f2e670f7f1e21a_21cb5974
cb14c133721d        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 11 minutes ago      Up 11 minutes                           k8s_POD.d8dbe16c_kube-controller-manager-k8s-master-39229988-0_kube-system_ee5fb6e3d925965b0048e6cc77534a6e_aecd055e
16d5a7e41944        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 11 minutes ago      Up 11 minutes                           k8s_POD.d8dbe16c_kube-apiserver-k8s-master-39229988-0_kube-system_1b3fae831a29391607f2e670f7f1e21a_122a8fd7
33b25ada02b6        gcr.io/google_containers/pause-amd64:3.0                 "/pause"                 11 minutes ago      Up 11 minutes                           k8s_POD.d8dbe16c_kube-addon-manager-k8s-master-39229988-0_kube-system_c0133a504dee133427d4802c1f2c3314_70fac706
5fce078d090b        gcr.io/google_containers/hyperkube-amd64:v1.4.5          "/hyperkube kubelet -"   12 minutes ago      Up 12 minutes                           jovial_hoover

I found the tons of the followed lines in a log of the hyperkube docker container:

5728 status_manager.go:450] Failed to update status for pod "_()": Get https://10.240.255.5:443/api/v1/namespaces/kube-system/pods/kube-apiserver-k8s-master-39229988-0: dial tcp 10.240.255.5:443: getsockopt: connection refused

If I try this request, I got this error:

wget https://10.240.255.5:443/api/v1/namespaces/kube-system/pods/kube-apiserver-k8s-master-39229988-0
--2016-11-16 08:36:25--  https://10.240.255.5/api/v1/namespaces/kube-system/pods/kube-apiserver-k8s-master-39229988-0
Connecting to 10.240.255.5:443... connected.
ERROR: cannot verify 10.240.255.5's certificate, issued by 'CN=ca':
  Unable to locally verify the issuer's authority.

Could you help me, what is wrong with my setup?
Thanks in advance.

[SorraTheOrc]

Did you perform step 9 in the walkthrough at https://github.com/Azure/acs-engine/blob/master/docs/kubernetes.md?

[SorraTheOrc]

Oh sorry, yes you did since I completely misread your error - sorry for the noise

[tudvari]

No problem, I need your help :)

[tudvari]

It seems to be, I found a partially solution: Dashboard stands on pending state. The problem was, my Service Principal doesn't have enough right to modify resources in Azure. After that the principal has more right, the pending services are started.

[colemickens]

The curl error is expected, the CA here is self-signed, not a trusted public CA, so you'll need to tell curl about the CA we use. Additionally, the apiserver is protected with client cert auth, so you'd need to also tell curl about the client cert/key to use.

But I'm assuming that curl was just a debug tool to figure out the state of the apiserver... So, if you've got it running, can we close this out? Thanks.

[tudvari]

Sure! We can close this issue, it seems to be working now. Thanks for your help.