WINC-643: Normalize BYOH and Machine logs and events #553

aravindhp · 2021-07-30T21:34:28Z

Change ConfigMap controller log name to "configmap" to be consistent with other controllers
Change the Windows Machine controller log name to reduce stuttering
Add logs when deconfiguring a BYOH instance
Add events for BYOH instances

aravindhp · 2021-07-30T21:42:17Z

/retitle WIP: WINC-643: [wmco] Normalize BYOH and Machine logs and events

aravindhp · 2021-07-30T21:42:34Z

/approve cancel
/cc @openshift/openshift-team-windows-containers

aravindhp · 2021-07-30T21:48:44Z

Production log output:

BYOH configuration:

{"level":"info","ts":1627675159.387272,"logger":"controllers.configmap","msg":"processing","instances in":"windows-instances"}
{"level":"info","ts":1627675189.5466676,"logger":"wc 10.0.128.8","msg":"configuring"}
{"level":"info","ts":1627675189.7167535,"logger":"wc 10.0.128.8","msg":"transferring files"}
{"level":"info","ts":1627675225.300282,"logger":"wc 10.0.128.8","msg":"configured","service":"windows_exporter","args":"--collectors.enabled cpu,cs,logical_disk,net,os,service,system,textfile,container,memory,cpu_info\""}
{"level":"info","ts":1627675230.3788443,"logger":"wc 10.0.128.8","msg":"configured kubelet","cmd":"C:\\k\\\\wmcb.exe initialize-kubelet --ignition-file C:\\Windows\\Temp\\worker.ign --kubelet-path C:\\k\\kubelet.exe","output":"Bootstrapping completed successfully"}
{"level":"info","ts":1627675260.5043035,"logger":"wc 10.0.128.8","msg":"configure","service":"hybrid-overlay-node","args":"--node winworker-r6f6p --k8s-kubeconfig c:\\k\\kubeconfig --windows-service --logfile C:\\var\\log\\hybrid-overlay\\hybrid-overlay.log\" depend= kubelet"}
{"level":"info","ts":1627675382.4569006,"logger":"wc 10.0.128.8","msg":"configured","service":"hybrid-overlay-node","args":"--node winworker-r6f6p --k8s-kubeconfig c:\\k\\kubeconfig --windows-service --logfile C:\\var\\log\\hybrid-overlay\\hybrid-overlay.log\" depend= kubelet"}
{"level":"info","ts":1627675429.4616241,"logger":"wc 10.0.128.8","msg":"configured kubelet for CNI","cmd":"C:\\k\\wmcb.exe configure-cni --cni-dir=\"C:\\k\\cni\\ --cni-config=\"C:\\k\\cni\\config\\cni.conf","output":"CNI configuration completed successfully"}
{"level":"info","ts":1627675434.937472,"logger":"wc 10.0.128.8","msg":"configured","service":"kube-proxy","args":"--windows-service --v=4 --proxy-mode=kernelspace --feature-gates=WinOverlay=true --hostname-override=winworker-r6f6p --kubeconfig=c:\\k\\kubeconfig --cluster-cidr=10.132.1.0/24 --log-dir=C:\\var\\log\\kube-proxy\\ --logtostderr=false --network-name=OVNKubernetesHybridOverlayNetwork --source-vip=10.132.1.217 --enable-dsr=false --feature-gates=IPv6DualStack=false\" depend= hybrid-overlay-node"}
{"level":"info","ts":1627675450.0061767,"logger":"nc 10.0.128.8","msg":"instance has been configured as a worker node","version":"3.0.0+"}
{"level":"info","ts":1627675450.1448567,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}
{"level":"info","ts":1627675450.145549,"logger":"controllers.configmap","msg":"processing","instances in":"windows-instances"}
{"level":"info","ts":1627675450.1597724,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}

BYOH deconfiguration:

{"level":"info","ts":1627675469.9359932,"logger":"controllers.configmap","msg":"processing","instances in":"windows-instances"}
{"level":"info","ts":1627675480.175665,"logger":"wc 10.0.128.8","msg":"deconfiguring"}
{"level":"info","ts":1627675510.521378,"logger":"wc 10.0.128.8","msg":"deconfigured","service":"windows_exporter"}
{"level":"info","ts":1627675540.693095,"logger":"wc 10.0.128.8","msg":"deconfigured","service":"kube-proxy"}
{"level":"info","ts":1627675570.934402,"logger":"wc 10.0.128.8","msg":"deconfigured","service":"hybrid-overlay-node"}
{"level":"info","ts":1627675601.105676,"logger":"wc 10.0.128.8","msg":"deconfigured","service":"kubelet"}
{"level":"info","ts":1627675601.1057017,"logger":"wc 10.0.128.8","msg":"deleting directories"}
{"level":"info","ts":1627675601.3327527,"logger":"controllers.configmap","msg":"instance has been deconfigured","node":"winworker-r6f6p"}
{"level":"info","ts":1627675601.3702445,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}
{"level":"info","ts":1627675601.3710914,"logger":"controllers.configmap","msg":"processing","instances in":"windows-instances"}
{"level":"info","ts":1627675601.3956797,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}

Machine configuration:

{"level":"info","ts":1627676656.6044638,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}
{"level":"info","ts":1627676656.6167266,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}
{"level":"info","ts":1627676656.89487,"logger":"controller.windowsmachine","msg":"processing","":"openshift-machine-api/winmach-qmp6z","address":"10.0.128.9"}
{"level":"info","ts":1627676875.5816388,"logger":"wc 10.0.128.9","msg":"configuring"}
{"level":"info","ts":1627676875.776952,"logger":"wc 10.0.128.9","msg":"transferring files"}
{"level":"info","ts":1627676908.6395252,"logger":"wc 10.0.128.9","msg":"configured","service":"windows_exporter","args":"--collectors.enabled cpu,cs,logical_disk,net,os,service,system,textfile,container,memory,cpu_info\""}
{"level":"info","ts":1627676913.8399482,"logger":"wc 10.0.128.9","msg":"configured kubelet","cmd":"C:\\k\\\\wmcb.exe initialize-kubelet --ignition-file C:\\Windows\\Temp\\worker.ign --kubelet-path C:\\k\\kubelet.exe","output":"Bootstrapping completed successfully"}
{"level":"info","ts":1627676943.897754,"logger":"wc 10.0.128.9","msg":"configure","service":"hybrid-overlay-node","args":"--node winmach-qmp6z --k8s-kubeconfig c:\\k\\kubeconfig --windows-service --logfile C:\\var\\log\\hybrid-overlay\\hybrid-overlay.log\" depend= kubelet"}
{"level":"info","ts":1627677066.127466,"logger":"wc 10.0.128.9","msg":"configured","service":"hybrid-overlay-node","args":"--node winmach-qmp6z --k8s-kubeconfig c:\\k\\kubeconfig --windows-service --logfile C:\\var\\log\\hybrid-overlay\\hybrid-overlay.log\" depend= kubelet"}
{"level":"info","ts":1627677113.0583558,"logger":"wc 10.0.128.9","msg":"configured kubelet for CNI","cmd":"C:\\k\\wmcb.exe configure-cni --cni-dir=\"C:\\k\\cni\\ --cni-config=\"C:\\k\\cni\\config\\cni.conf","output":"CNI configuration completed successfully"}
{"level":"info","ts":1627677118.7240837,"logger":"wc 10.0.128.9","msg":"configured","service":"kube-proxy","args":"--windows-service --v=4 --proxy-mode=kernelspace --feature-gates=WinOverlay=true --hostname-override=winmach-qmp6z --kubeconfig=c:\\k\\kubeconfig --cluster-cidr=10.132.2.0/24 --log-dir=C:\\var\\log\\kube-proxy\\ --logtostderr=false --network-name=OVNKubernetesHybridOverlayNetwork --source-vip=10.132.2.76 --enable-dsr=false --feature-gates=IPv6DualStack=false\" depend= hybrid-overlay-node"}
{"level":"info","ts":1627677133.7587388,"logger":"nc 10.0.128.9","msg":"instance has been configured as a worker node","version":"3.0.0+"}
{"level":"info","ts":1627677133.790622,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}
{"level":"info","ts":1627677133.8032951,"logger":"metrics","msg":"Prometheus configured","endpoints":"windows-exporter","port":9182,"name":"metrics"}

mansikulkarni96 · 2021-08-02T22:26:05Z

made a pass on all the logs, LGTM.

aravindhp · 2021-08-04T01:42:22Z

/retitle WINC-643: Normalize BYOH and Machine logs and events

aravindhp · 2021-08-04T01:48:38Z

/cherry-pick community-4.8

openshift-cherrypick-robot · 2021-08-04T01:48:39Z

@aravindhp: once the present PR merges, I will cherry-pick it on top of community-4.8 in a new PR and assign it to you.

In response to this:

/cherry-pick community-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aravindhp · 2021-08-04T01:48:54Z

/cherry-pick release-4.8

openshift-cherrypick-robot · 2021-08-04T01:48:55Z

@aravindhp: once the present PR merges, I will cherry-pick it on top of release-4.8 in a new PR and assign it to you.

In response to this:

/cherry-pick release-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

controllers/configmap_controller.go

sebsoto · 2021-08-04T17:10:11Z

/approve
/hold
can be removed after revert

pkg/windows/windows.go

- Change ConfigMap controller log name to "configmap" to be consistent with other controllers - Change the Windows Machine controller log name to reduce stuttering - Add logs when deconfiguring a BYOH instance - Add events for BYOH instances

jrvaldes

/lgtm

openshift-ci · 2021-08-04T18:07:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jrvaldes, sebsoto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sebsoto]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aravindhp · 2021-08-04T23:58:09Z

/hold cancel

openshift-bot · 2021-08-05T00:06:53Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2021-08-05T00:54:55Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-cherrypick-robot · 2021-08-05T05:08:34Z

@aravindhp: new pull request created: #565

In response to this:

/cherry-pick community-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-cherrypick-robot · 2021-08-05T05:09:08Z

@aravindhp: #553 failed to apply on top of branch "release-4.8":

Applying: Normalize BYOH and Machine logs and events
Using index info to reconstruct a base tree...
M	controllers/configmap_controller.go
M	controllers/controllers.go
M	controllers/windowsmachine_controller.go
M	pkg/nodeconfig/nodeconfig.go
M	pkg/windows/windows.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/windows/windows.go
CONFLICT (content): Merge conflict in pkg/windows/windows.go
Auto-merging pkg/nodeconfig/nodeconfig.go
CONFLICT (content): Merge conflict in pkg/nodeconfig/nodeconfig.go
Auto-merging controllers/windowsmachine_controller.go
CONFLICT (content): Merge conflict in controllers/windowsmachine_controller.go
Auto-merging controllers/controllers.go
CONFLICT (content): Merge conflict in controllers/controllers.go
Auto-merging controllers/configmap_controller.go
CONFLICT (content): Merge conflict in controllers/configmap_controller.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Normalize BYOH and Machine logs and events
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aravindhp · 2021-08-11T21:59:11Z

@mansikulkarni96 tagging you here as a reminder to include this PR when you are backporting to release-4.8.

mansikulkarni96 · 2021-08-17T16:53:21Z

/cherry-pick release-4.8

openshift-cherrypick-robot · 2021-08-17T16:53:57Z

@mansikulkarni96: new pull request created: #608

In response to this:

/cherry-pick release-4.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jul 30, 2021

openshift-ci bot requested a review from sebsoto July 30, 2021 21:34

aravindhp force-pushed the WINC-643-byoh-logging branch from 39dacb0 to 8be8c1a Compare July 30, 2021 21:41

openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2021

openshift-ci bot changed the title ~~WINC-643: [wmco] Normalize BYOH and Machine logs and events~~ WIP: WINC-643: [wmco] Normalize BYOH and Machine logs and events Jul 30, 2021

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 30, 2021

openshift-ci bot requested a review from a team July 30, 2021 21:42

openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 30, 2021

openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 2, 2021

aravindhp force-pushed the WINC-643-byoh-logging branch from 8be8c1a to 6b756db Compare August 4, 2021 01:41

openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 4, 2021

openshift-ci bot changed the title ~~WIP: WINC-643: [wmco] Normalize BYOH and Machine logs and events~~ WINC-643: Normalize BYOH and Machine logs and events Aug 4, 2021

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 4, 2021

sebsoto requested changes Aug 4, 2021

View reviewed changes

controllers/configmap_controller.go Show resolved Hide resolved

controllers/configmap_controller.go Outdated Show resolved Hide resolved

openshift-ci bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Aug 4, 2021

jrvaldes reviewed Aug 4, 2021

View reviewed changes

pkg/windows/windows.go Outdated Show resolved Hide resolved

aravindhp force-pushed the WINC-643-byoh-logging branch from 6b756db to a96245a Compare August 4, 2021 17:34

jrvaldes approved these changes Aug 4, 2021

View reviewed changes

openshift-ci bot assigned jrvaldes Aug 4, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 4, 2021

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 4, 2021

openshift-ci bot merged commit d372356 into openshift:master Aug 5, 2021

openshift-cherrypick-robot mentioned this pull request Aug 5, 2021

[community-4.8] WINC-643: Normalize BYOH and Machine logs and events #565

Merged

aravindhp deleted the WINC-643-byoh-logging branch August 11, 2021 21:58

openshift-cherrypick-robot mentioned this pull request Aug 17, 2021

[release-4.8] WINC-643: Normalize BYOH and Machine logs and events #608

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WINC-643: Normalize BYOH and Machine logs and events #553

WINC-643: Normalize BYOH and Machine logs and events #553

aravindhp commented Jul 30, 2021

aravindhp commented Jul 30, 2021 •

edited by openshift-ci bot

aravindhp commented Jul 30, 2021

aravindhp commented Jul 30, 2021

mansikulkarni96 commented Aug 2, 2021

aravindhp commented Aug 4, 2021 •

edited by openshift-ci bot

aravindhp commented Aug 4, 2021

openshift-cherrypick-robot commented Aug 4, 2021

aravindhp commented Aug 4, 2021

openshift-cherrypick-robot commented Aug 4, 2021

sebsoto commented Aug 4, 2021

jrvaldes left a comment

openshift-ci bot commented Aug 4, 2021

aravindhp commented Aug 4, 2021

openshift-bot commented Aug 5, 2021

openshift-bot commented Aug 5, 2021

openshift-cherrypick-robot commented Aug 5, 2021

openshift-cherrypick-robot commented Aug 5, 2021

aravindhp commented Aug 11, 2021

mansikulkarni96 commented Aug 17, 2021

openshift-cherrypick-robot commented Aug 17, 2021

WINC-643: Normalize BYOH and Machine logs and events #553

WINC-643: Normalize BYOH and Machine logs and events #553

Conversation

aravindhp commented Jul 30, 2021

aravindhp commented Jul 30, 2021 • edited by openshift-ci bot

aravindhp commented Jul 30, 2021

aravindhp commented Jul 30, 2021

mansikulkarni96 commented Aug 2, 2021

aravindhp commented Aug 4, 2021 • edited by openshift-ci bot

aravindhp commented Aug 4, 2021

openshift-cherrypick-robot commented Aug 4, 2021

aravindhp commented Aug 4, 2021

openshift-cherrypick-robot commented Aug 4, 2021

sebsoto commented Aug 4, 2021

jrvaldes left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Aug 4, 2021

aravindhp commented Aug 4, 2021

openshift-bot commented Aug 5, 2021

openshift-bot commented Aug 5, 2021

openshift-cherrypick-robot commented Aug 5, 2021

openshift-cherrypick-robot commented Aug 5, 2021

aravindhp commented Aug 11, 2021

mansikulkarni96 commented Aug 17, 2021

openshift-cherrypick-robot commented Aug 17, 2021

aravindhp commented Jul 30, 2021 •

edited by openshift-ci bot

aravindhp commented Aug 4, 2021 •

edited by openshift-ci bot