Disable the kube-apiserver insecure port, and lead to health-check can't work

[fisherxu]

In my environment, because the insecure port 8080 of kube-apiserver expose all the API. For security reasons, the port is Disabled. But lead to health-check and pprof can't work. Can we consider opening a separate port like other component only for health-check and pprof， metrics？

/sig cluster-lifecycle
/cc @caesarxuchao @deads2k @lavalamp @liggitt @mml

[fisherxu]

/cc @nikhiljindal @smarterclayton @sttts

[deads2k]

But lead to health-check and pprof can't work. Can we consider opening a separate port like other component only for health-check and pprof， metrics？

Why not update your health check to use the secure port? The kubelet supports it and the default authorization rules allow it.

[fisherxu]

@deads2k Thanks for quick reply, since other components like kubelet all have the insecure port to do health-check and pprof, so we want to be consistent.Why can't consider open an separate port like other components to do health and metric，8080 only for insecure api for k8s itself. I think this make the architecture more clearer :)

[deads2k]

@deads2k Thanks for quick reply, since other components like kubelet all have the insecure port to do health-check and pprof, so we want to be consistent.Why can't consider open an separate port like other components to do health and metric，8080 only for insecure api for k8s itself. I think this make the architecture more clearer :)

Nothing should use an unsecured port. All the others should be fixed to remove that port. Movement on the kube-apiserver is towards securing all ports, not opening specific holes. Default authorization (rbac) will allow access to healthz, but not to metrics. Other authorizers should behave likewise. Anything accessing metrics should be authenticatable.

[fisherxu]

Default authorization (rbac) will allow access to healthz, but not to metrics. Other authorizers should behave likewise. Anything accessing metrics should be authenticatable.

I really agree with you, accessing metrics should be authenticatable. BTW, when this plan will be implemented?

[fisherxu]

Still another question, I deploy kube-apiserver as a container, if disabled insecure port, kubelet probe will can't work, have any idea how to deal with it and in your cluster how to do this?

[deads2k]

Still another question, I deploy kube-apiserver as a container, if disabled insecure port, kubelet probe will can't work, have any idea how to deal with it and in your cluster how to do this?

Readiness probes allow the specification of a scheme, which can be https.

[idouba]

@deads2k Yes, HTTPProber can specify HTTPS schema, but disable security check by this:
func New() HTTPProber { tlsConfig := &tls.Config{InsecureSkipVerify: true} ... return httpProber{transport} }

[dims]

@idouba InsecureSkipVerify seems ok for a health check. can you please explain what you mean a bit more?

[fisherxu]

@deads2k @idouba @dims Thanks very much for reploy.

Readiness probes allow the specification of a scheme, which can be https.

Our kube-apiserver securePort will verify the client's certificate. InsecureSkipVerify ok for a health check？

[dims]

@fisherxu i don't believe /healthz needs the client cert. Can you please try and see if it works? yes on the client side InsecureSkipVerify should be enough

[fisherxu]

@dims I try curl -k https://masterip:6443/healthz and it return Unauthorized. Maybe we can make probe support the certificate？

[deads2k]

@dims I try curl -k https://masterip:6443/healthz and it return Unauthorized. Maybe we can make probe support the certificate？

Can you show the complete -v output from curl? It can return 403's until the bootstrap roles and bindings are created, but we bind system:discovery to system:unauthenticated. If you're using a webhook or some other authorization scheme, you'll want to do the same. /healthz doesn't reveal sensitive data; the reasons are redacted.

[fisherxu]

root@SZV1000272344:~# curl -k -v https://masterip:6443/healthz

• Trying masterip...
• Connected to masterip(masterip) port 6443 (#0)
• found 173 certificates in /etc/ssl/certs/ca-certificates.crt
• found 694 certificates in /etc/ssl/certs
• ALPN, offering http/1.1
• SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
• server certificate verification SKIPPED
• server certificate status verification SKIPPED
• common name: kube-apiserver (matched)
• server certificate expiration date OK
• server certificate activation date OK
• certificate public key: RSA
• certificate version: Need real cluster integration tests #3
• subject: CN=kube-apiserver
• start date: Wed, 26 Apr 2017 03:05:08 GMT
• expire date: Thu, 26 Apr 2018 03:05:08 GMT
• issuer: CN=kubernetes
• compression: NULL
• ALPN, server accepted to use http/1.1

GET /healthz HTTP/1.1
Host: masterip:6443
User-Agent: curl/7.47.0
Accept: /

< HTTP/1.1 401 Unauthorized
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Thu, 24 Aug 2017 13:40:56 GMT
< Content-Length: 13
<
Unauthorized

• Connection #0 to host 100.109.161.75 left intact

it return 401 @deads2k @dims @idouba