Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RBAC proxy to metrics #424

Merged
merged 1 commit into from
Apr 29, 2021

Conversation

saschagrunert
Copy link
Member

What type of PR is this?

/kind feature

What this PR does / why we need it:

We now use the kube-rbac-proxy to provide two metrics endpoints. Those
get exposed through the metrics-spod and metrics-controller-runtime
services, serving an authorized self-signed TLS connection.

To be able to retrieve the metrics, it is now required to deploy a
ClusterRole and ClusterRoleBinding for a ServiceAccount. For
example:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: spo-metrics-client
rules:
- nonResourceURLs:
  - /metrics
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: spo-metrics-client
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: spo-metrics-client
subjects:
- kind: ServiceAccount
  name: default
  namespace: security-profiles-operator

The above example is also part of the default deployment, and can be
tested via kubectl run within the security-profiles-operator
namespace by running:

> kubectl run --rm -it pod --image=alpine -- sh -c \
    'apk update && apk add bash curl &&
     curl -vsk -H "Authorization: Bearer `cat /var/run/secrets/kubernetes.io/serviceaccount/token`"
     https://metrics-spod/metrics'
…
*   Trying 10.0.0.222:443...
* Connected to metrics-spod (10.0.0.222) port 443 (#0)
…
> authorization: Bearer xxx
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< content-type: text/plain; version=0.0.4; charset=utf-8
< date: Wed, 28 Apr 2021 12:00:46 GMT
…
security_profiles_operator_seccomp_profile{operation="update"} 1

Documentation and e2e testing will follow in further patches.

Which issue(s) this PR fixes:

Refers to #422

Does this PR have test?

None

Special notes for your reviewer:

None

Does this PR introduce a user-facing change?

Deploying kube-rbac-proxy sidecar in SPOD for exposing metrics via the new `metrics-spod` and `metrics-controller-runtime` services.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 28, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 28, 2021
Copy link
Contributor

@JAORMX JAORMX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rbac-proxy isn't using TLS, is it?

internal/pkg/manager/spod/bindata/spod.go Outdated Show resolved Hide resolved
@saschagrunert
Copy link
Member Author

the rbac-proxy isn't using TLS, is it?

It is using a self-signed one right now and we also can provide an external one.

@saschagrunert saschagrunert force-pushed the rbac-proxy branch 3 times, most recently from d3702b4 to f22692d Compare April 28, 2021 13:05
@saschagrunert saschagrunert changed the title Add RBAC proxy to metrics WIP: Add RBAC proxy to metrics Apr 28, 2021
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 28, 2021
@saschagrunert
Copy link
Member Author

/retest

We now use the kube-rbac-proxy to provide two metrics endpoints. Those
get exposed through the `metrics-spod` and `metrics-controller-runtime`
services, serving an authorized self-signed TLS connection.

To be able to retrieve the metrics, it is now required to deploy a
`ClusterRole` and `ClusterRoleBinding` for a `ServiceAccount`. For
example:

```yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: spo-metrics-client
rules:
- nonResourceURLs:
  - /metrics
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: spo-metrics-client
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: spo-metrics-client
subjects:
- kind: ServiceAccount
  name: default
  namespace: security-profiles-operator
```

The above example is also part of the default deployment, and can be
tested via `kubectl run` within the `security-profiles-operator`
namespace by running:

```
> kubectl run --rm -it pod --image=alpine -- sh -c \
    'apk update && apk add bash curl &&
     curl -vsk -H "Authorization: Bearer `cat /var/run/secrets/kubernetes.io/serviceaccount/token`"
     https://metrics-spod/metrics'
…
*   Trying 10.0.0.222:443...
* Connected to metrics-spod (10.0.0.222) port 443 (#0)
…
> authorization: Bearer xxx
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 200
< content-type: text/plain; version=0.0.4; charset=utf-8
< date: Wed, 28 Apr 2021 12:00:46 GMT
…
security_profiles_operator_seccomp_profile{operation="update"} 1
```

Documentation and e2e testing will follow in further patches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
@saschagrunert saschagrunert changed the title WIP: Add RBAC proxy to metrics Add RBAC proxy to metrics Apr 28, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 28, 2021
deploy/base/service.yaml Show resolved Hide resolved
Copy link
Contributor

@JAORMX JAORMX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 29, 2021
@k8s-ci-robot k8s-ci-robot merged commit a157bda into kubernetes-sigs:master Apr 29, 2021
@saschagrunert saschagrunert deleted the rbac-proxy branch April 29, 2021 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants