Adds liveness/readiness probes. Removes unused manifest.

[absoludity]

Description of the change

This PR enables the liveness and readiness checks so that they are used to mark the pod as ready and/or restart the pod when it's not responding.

I've just re-used the http version of the configured plugins endpoint, since it is cheap and a good candidate.

I also removed the now unused developer manifest for the kubeapps apis setup (since it was overwriting the changes in the main values :) ).

Benefits

The pod can be restarted by Kubernetes as needed.

Possible drawbacks

None.

Applicable issues

• fixes Add liveness and readiness probes for kubeapps-apis #3284

Additional information

We should add proper logging - I'll create an issue for that.

[antgamdia]

Great! Thanks for removing the unused files. I just have a question on the livenessProbe, but not a big deal, +1-ing!

[antgamdia]

Just wondering: if no plugin is enabled, wouldn't it rather return a 404-alike, so k8s'd say: "it's not live" ?

I mean, for the readiness probe I think this behavior is ok, but for the liveness, I'm not sure, what do you think?

[absoludity]

The configured-plugins endpoint returns a list of configured plugins, so there's no 404 when none are configured:

curl http://localhost:8080/core/plugins/v1alpha1/configured-plugins -v
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /core/plugins/v1alpha1/configured-plugins HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< Grpc-Metadata-Content-Type: application/grpc
< Date: Fri, 27 Aug 2021 07:51:03 GMT
< Content-Length: 2
< 
* Connection #0 to host localhost left intact
{}

[antgamdia]

Great! Thanks for checking!