Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate pkg/kubelet/cm/memorymanager to structured logging #99974

Merged
merged 1 commit into from Mar 16, 2021

Conversation

knabben
Copy link
Member

@knabben knabben commented Mar 9, 2021

What type of PR is this?

/kind feature
/sig instrumentation
/sig node

What this PR does / why we need it:

Migrate pkg/kubelet/cm/memorymanager to structured logging

Which issue(s) this PR fixes:

Refs #98976

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

@k8s-ci-robot k8s-ci-robot added release-note-none cncf-cla: yes kind/feature size/M sig/instrumentation sig/node needs-triage needs-priority labels Mar 9, 2021
@k8s-ci-robot k8s-ci-robot requested review from ehashman and matthyx Mar 9, 2021
@ehashman ehashman added this to Triage in SIG Node PR Triage Mar 9, 2021
pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
return false
}

if len(nodeState1.MemoryMap) != len(nodeState2.MemoryMap) {
klog.Errorf("[memorymanager] node states memory map have different length: %d != %d", len(nodeState1.MemoryMap), len(nodeState2.MemoryMap))
klog.ErrorS(nil, "[memorymanager] Node states memory map have different length.", "len(state1.MemoryMap)", len(nodeState1.MemoryMap), "len(state2.MemoryMap)", len(nodeState2.MemoryMap))
Copy link
Contributor

@adisky adisky Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
klog.ErrorS(nil, "[memorymanager] Node states memory map have different length.", "len(state1.MemoryMap)", len(nodeState1.MemoryMap), "len(state2.MemoryMap)", len(nodeState2.MemoryMap))
klog.ErrorS(nil, "[memorymanager] Node states memory map have different length", "len1", len(nodeState1.MemoryMap), "len2", len(nodeState2.MemoryMap))

Copy link
Contributor

@adisky adisky Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure here, keys could be state1 or state2 or lengthState1 lengthState2

Copy link
Member Author

@knabben knabben Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have this doubt, maybe x1 and x2 is enough for the context

return false
}

if nodeState1.NumberOfAssignments != nodeState2.NumberOfAssignments {
klog.Errorf("[memorymanager] node states number of assignments are different: %d != %d", nodeState1.NumberOfAssignments, nodeState2.NumberOfAssignments)
klog.ErrorS(nil,"[memorymanager] Node states number of assignments are different.", "state1.NumberOfAssignments", nodeState1.NumberOfAssignments, "state2.NumberOfAssignments", nodeState2.NumberOfAssignments)
Copy link
Contributor

@adisky adisky Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
klog.ErrorS(nil,"[memorymanager] Node states number of assignments are different.", "state1.NumberOfAssignments", nodeState1.NumberOfAssignments, "state2.NumberOfAssignments", nodeState2.NumberOfAssignments)
klog.ErrorS(nil,"[memorymanager] Node states number of assignments are different.", "assignment1", nodeState1.NumberOfAssignments, "assignment2", nodeState2.NumberOfAssignments)

@adisky
Copy link
Contributor

@adisky adisky commented Mar 9, 2021

/triage accepted
/priority important-soon

@k8s-ci-robot k8s-ci-robot added triage/accepted priority/important-soon and removed needs-triage needs-priority labels Mar 9, 2021
return NewPolicyNone()
}

func (m *fakeManager) Allocate(pod *v1.Pod, container *v1.Container) error {
klog.Infof("[fake memorymanager] Allocate (pod: %s, container: %s", pod.Name, container.Name)
klog.InfoS("[fake memorymanager] Allocate", "podName", pod.Name, "container", container.Name)
Copy link
Contributor

@lala123912 lala123912 Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

@knabben knabben Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these are only strings

pkg/kubelet/cm/memorymanager/fake_memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/fake_memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/fake_memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/state/state_checkpoint.go Outdated Show resolved Hide resolved
@ehashman ehashman moved this from Triage to Waiting on Author in SIG Node PR Triage Mar 10, 2021
Copy link
Member

@kikisdeliveryservice kikisdeliveryservice left a comment

this is a user facing change, please update pr description so this gets a release note. :)

@k8s-ci-robot k8s-ci-robot added the release-note label Mar 11, 2021
@k8s-ci-robot k8s-ci-robot removed the release-note-none label Mar 11, 2021
@kikisdeliveryservice kikisdeliveryservice dismissed their stale review Mar 12, 2021

note added :)

Copy link
Member

@kikisdeliveryservice kikisdeliveryservice left a comment

Couple things but q: if we are deleting [memorymanager] should it also be deleted from fmt.Errorf("[memorymanager] failed to get the default NUMA affinity, no NUMA nodes with enough memory is available") in the same file (pkg/kubelet/cm/memorymanager/policy_static.go )?
cc: @ehashman

pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
@knabben
Copy link
Member Author

@knabben knabben commented Mar 12, 2021

/retest

@kikisdeliveryservice
Copy link
Member

@kikisdeliveryservice kikisdeliveryservice commented Mar 12, 2021

/test pull-kubernetes-unit

Copy link
Member

@ehashman ehashman left a comment

One left :)

pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
@ehashman
Copy link
Member

@ehashman ehashman commented Mar 12, 2021

/milestone v1.21

@k8s-ci-robot k8s-ci-robot added this to the v1.21 milestone Mar 12, 2021
@knabben
Copy link
Member Author

@knabben knabben commented Mar 13, 2021

/retest

pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/state/state_mem.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/memory_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
@@ -228,13 +228,13 @@ func (m *manager) RemoveContainer(containerID string) error {
// if error appears it means container entry already does not exist under the container map
podUID, containerName, err := m.containerMap.GetContainerRef(containerID)
if err != nil {
klog.Warningf("[memorymanager] Failed to get container %s from container map error: %v", containerID, err)
klog.ErrorS(err, "Failed to get container from container map", "containerID", containerID)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have quite a few Warningf->ErrorS - shouldn't they be InfoS??

Can you please check throughout

Copy link
Member Author

@knabben knabben Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think is correct to migrate Warningf to Info, but in this case should not be this an error log level?

Copy link
Member Author

@knabben knabben Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving it to klog.InfoS(..., "err", err), just looks odd to me

Copy link
Member

@ehashman ehashman Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

@knabben knabben Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ehashman yes, the point is the Warning is already being wrong here.. but changed according to the migration guide

pkg/kubelet/cm/memorymanager/state/state_checkpoint.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/policy_static.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/state/state_checkpoint.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/memorymanager/state/state_mem.go Outdated Show resolved Hide resolved
@kikisdeliveryservice
Copy link
Member

@kikisdeliveryservice kikisdeliveryservice commented Mar 15, 2021

Thanks for all of our work on this @knabben and for updating with changes so quickly 👍

/lgtm

@kikisdeliveryservice kikisdeliveryservice moved this from Waiting on Author to Needs Approver in Structured Logging Migration for Kubelet, 1.21 Mar 15, 2021
@k8s-ci-robot k8s-ci-robot added the lgtm label Mar 15, 2021
@kikisdeliveryservice kikisdeliveryservice moved this from Waiting on Author to Needs Approver in SIG Node PR Triage Mar 15, 2021
@k8s-ci-robot
Copy link
Contributor

@k8s-ci-robot k8s-ci-robot commented Mar 16, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: knabben, mrunalp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved label Mar 16, 2021
@mrunalp mrunalp moved this from Needs Approver to Done in SIG Node PR Triage Mar 16, 2021
@k8s-ci-robot k8s-ci-robot merged commit e5309ef into kubernetes:master Mar 16, 2021
14 checks passed
Structured Logging Migration for Kubelet, 1.21 automation moved this from Needs Approver to Done Mar 16, 2021
@k8s-ci-robot k8s-ci-robot added release-note-none and removed release-note labels Mar 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved area/kubelet cncf-cla: yes kind/feature lgtm priority/important-soon release-note-none sig/instrumentation sig/node size/M triage/accepted
Development

Successfully merging this pull request may close these issues.

None yet

8 participants