Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet #778

bond95 · 2020-09-07T08:54:09Z

Follow up of this PR: #751

Basically it's fixing same issue as for SDN. To avoid deploying metric's DeamonSet before OVN, metrics was moved to OVN DeamonSet.

openshift-ci-robot · 2020-09-07T08:54:33Z

Hi @bond95. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

juanluisvaladas · 2020-09-08T08:08:52Z

/retest

juanluisvaladas · 2020-09-08T08:58:18Z

/ok-to-test

danwinship · 2020-09-08T11:45:28Z

bindata/network/ovn-kubernetes/ovnkube-node.yaml

@@ -286,63 +343,6 @@ spec:
      - name: ovn-cert
        secret:
          secretName: ovn-cert


I believe the volume entry in both ovnkube-node.yaml and ovnkube-master.yaml needs to be made optional: true (as it is in bindata/network/openshift-sdn/sdn.yaml), because the secret can't be created until after the network is already up, but if the volume is required then the network won't come up without the secret having already been created.

Tests are failing precisely because of this

juanluisvaladas · 2020-09-10T11:02:09Z

@bond95 can you add this to the PR as well?
#786

bond95 · 2020-09-14T13:10:26Z

/retest

juanluisvaladas · 2020-09-14T17:22:44Z

/lgtm

juanluisvaladas · 2020-09-15T09:58:33Z

/retest

bond95 · 2020-09-15T16:03:47Z

/retest

juanluisvaladas · 2020-09-30T13:50:42Z

/lgtm

juanluisvaladas · 2020-09-30T14:53:00Z

bindata/network/ovn-kubernetes/ovnkube-master.yaml

+                -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
+                "https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}/api/v1/namespaces/ovn-kubernetes/services/ovnkube-master" |
+                  python -c 'import json,sys; print(json.load(sys.stdin)["metadata"]["creationTimestamp"])'
+            )


replace this line with
) || :

So that we don't get the error treated as a fatal. And the same with the node one.

juanluisvaladas · 2020-09-30T18:00:05Z

/retest

juanluisvaladas · 2020-09-30T18:18:53Z

@abhat this includes PRs #751 and #786 . I've verified it works on a cluster manually. Can you PTAL?

Logs of the cluster where we see it working as intended

$ oc logs  -c kube-rbac-proxy ovnkube-node-5m24w 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.6/json/__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I0930 17:35:03.208820    6882 main.go:188] Valid token audiences: 
I0930 17:35:03.208920    6882 main.go:261] Reading certificate files
I0930 17:35:03.209213    6882 main.go:294] Starting TCP socket on :9103
I0930 17:35:03.210421    6882 main.go:301] Listening securely on :9103
2020/09/30 17:35:04 http: TLS handshake error from 10.0.32.4:38802: remote error: tls: bad certificate
2020/09/30 17:35:28 http: TLS handshake error from 10.0.32.3:41002: remote error: tls: bad certificate
2020/09/30 17:35:34 http: TLS handshake error from 10.0.32.4:39298: remote error: tls: bad certificate
2020/09/30 17:35:58 http: TLS handshake error from 10.0.32.3:41228: remote error: tls: bad certificate
2020/09/30 17:36:04 http: TLS handshake error from 10.0.32.4:39702: remote error: tls: bad certificate
2020/09/30 17:36:28 http: TLS handshake error from 10.0.32.3:41414: remote error: tls: bad certificate
2020/09/30 17:36:34 http: TLS handshake error from 10.0.32.4:40104: remote error: tls: bad certificate

juanluisvaladas · 2020-09-30T21:11:19Z

/retest

openshift-bot · 2020-10-02T09:20:15Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-10-02T09:46:14Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-10-02T11:17:10Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-10-02T11:58:03Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-10-02T12:22:14Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-10-02T13:01:13Z

/retest

Please review the full test history for this PR and help us cut down flakes.

juanluisvaladas · 2020-10-02T13:18:47Z

/bugzilla refresh

openshift-ci-robot · 2020-10-02T13:18:49Z

@juanluisvaladas: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

juanluisvaladas · 2020-10-02T13:20:00Z

/retitle Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet

juanluisvaladas · 2020-10-02T13:20:09Z

/bugzilla refresh

openshift-ci-robot · 2020-10-02T13:20:10Z

@juanluisvaladas: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-10-02T13:20:24Z

@bond95: This pull request references Bugzilla bug 1884430, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.6.0) matches configured target release for branch (4.6.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

juanluisvaladas · 2020-10-02T13:22:25Z

/test e2e-vsphere-ovn

juanluisvaladas · 2020-10-02T13:24:17Z

/help

tssurya · 2020-10-02T13:29:58Z

@juanluisvaladas : can we please make sure we don't break that alerts-job thing again ? We have already bounced around this label change quiet a few times:

juanluisvaladas · 2020-10-02T13:39:17Z

@tssurya what needs to be changed? I see tests are passing so it should be fine

openshift-bot · 2020-10-02T14:32:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-10-02T14:45:16Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2020-10-02T14:59:59Z

@bond95: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-gcp-upgrade	`b7f8cc4`	link	`/test e2e-gcp-upgrade`
ci/prow/e2e-vsphere	`661f1d5`	link	`/test e2e-vsphere`
ci/prow/e2e-openstack	`518118b`	link	`/test e2e-openstack`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-bot · 2020-10-02T15:11:17Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-ci-robot · 2020-10-02T16:25:38Z

@bond95: All pull requests linked via external trackers have merged:

openshift/cluster-network-operator#778

Bugzilla bug 1884430 has been moved to the MODIFIED state.

In response to this:

Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

With PR openshift#778 (commit 518118b) we changed the service monitor definition which broke the alert definition for OVN.

With PR openshift#778 (commit 518118b) we changed the service monitor definition, and the daemonset serviceaccount. With this PR we fix the serverName in the serviceMonitors, the serviceAccount permissions and the prometheusRule alert expression.

openshift-ci-robot requested review from abhat and aojea September 7, 2020 08:54

openshift-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 7, 2020

bond95 force-pushed the move-rbac-proxy branch from 080d5e9 to b7f8cc4 Compare September 8, 2020 08:10

openshift-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 8, 2020

danwinship reviewed Sep 8, 2020

View reviewed changes

juanluisvaladas mentioned this pull request Sep 13, 2020

Bug 1883903:Add retries to SDN's RBAC proxy #786

Merged

bond95 force-pushed the move-rbac-proxy branch from b7f8cc4 to 661f1d5 Compare September 14, 2020 10:11

openshift-ci-robot assigned juanluisvaladas Sep 14, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 14, 2020

bond95 force-pushed the move-rbac-proxy branch from 661f1d5 to 20d9f23 Compare September 30, 2020 13:48

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Sep 30, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 30, 2020

juanluisvaladas suggested changes Sep 30, 2020

View reviewed changes

Moved kube-rbac-proxy to general ovnkube DeamonSet

518118b

bond95 force-pushed the move-rbac-proxy branch from 20d9f23 to 518118b Compare September 30, 2020 15:33

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Sep 30, 2020

openshift-ci-robot changed the title ~~Bug 1884430 Move kube-rbac-proxy to general ovnkube DeamonSet~~ Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet Oct 2, 2020

openshift-ci-robot added bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Oct 2, 2020

openshift-merge-robot merged commit af1f2ed into openshift:master Oct 2, 2020

juanluisvaladas pushed a commit to juanluisvaladas/cluster-network-operator that referenced this pull request Oct 5, 2020

OVN: Fix master control plane job name

cfba329

With PR openshift#778 (commit 518118b) we changed the service monitor definition which broke the alert definition for OVN.

juanluisvaladas mentioned this pull request Oct 5, 2020

Bug 1885165: Fix ovnkube metrics #824

Merged

juanluisvaladas pushed a commit to juanluisvaladas/cluster-network-operator that referenced this pull request Oct 5, 2020

OVN: Fix master control plane job name

d18d1c8

With PR openshift#778 (commit 518118b) we changed the service monitor definition which broke the alert definition for OVN.

Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet #778

Bug 1884430: Move kube-rbac-proxy to general ovnkube DeamonSet #778

Conversation

bond95 commented Sep 7, 2020

openshift-ci-robot commented Sep 7, 2020

juanluisvaladas commented Sep 8, 2020

juanluisvaladas commented Sep 8, 2020

danwinship Sep 8, 2020

Choose a reason for hiding this comment

juanluisvaladas Sep 8, 2020

Choose a reason for hiding this comment

juanluisvaladas commented Sep 10, 2020

bond95 commented Sep 14, 2020

juanluisvaladas commented Sep 14, 2020

juanluisvaladas commented Sep 15, 2020

bond95 commented Sep 15, 2020

juanluisvaladas commented Sep 30, 2020

juanluisvaladas Sep 30, 2020

Choose a reason for hiding this comment

juanluisvaladas commented Sep 30, 2020

juanluisvaladas commented Sep 30, 2020 • edited

juanluisvaladas commented Sep 30, 2020

openshift-bot commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

juanluisvaladas commented Oct 2, 2020

openshift-ci-robot commented Oct 2, 2020

juanluisvaladas commented Oct 2, 2020

juanluisvaladas commented Oct 2, 2020

openshift-ci-robot commented Oct 2, 2020

openshift-ci-robot commented Oct 2, 2020

juanluisvaladas commented Oct 2, 2020

juanluisvaladas commented Oct 2, 2020

tssurya commented Oct 2, 2020

juanluisvaladas commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

openshift-bot commented Oct 2, 2020

openshift-ci-robot commented Oct 2, 2020 • edited

openshift-bot commented Oct 2, 2020

openshift-ci-robot commented Oct 2, 2020

juanluisvaladas commented Sep 30, 2020 •

edited

openshift-ci-robot commented Oct 2, 2020 •

edited