Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated - topologymanager: Add Merge method to Policy #85798

Merged
merged 33 commits into from
Jan 17, 2020
Merged

Updated - topologymanager: Add Merge method to Policy #85798

merged 33 commits into from
Jan 17, 2020

Conversation

nolancon
Copy link

@nolancon nolancon commented Dec 2, 2019

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind api-change
/kind bug
/kind cleanup
/kind deprecation
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:
This PR is an updated version of #81586:
"In the current design the Topology Manager generates the merged
hints and select the best hint and the policy indicates whether
or not to admit the pod.

This patch adds Merge abstraction to Topology Manager policies which
allow the them to control on how to merge the permutation hints.
It also update the strict policy to admit only if all resources
aligned to same NUMA."

Now that #84721 has been merged, the changes proposed here are the original commits by @adrianchiris, but rebased to accommodate easier reviewing.

Related Issue: #83478

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/kubelet sig/node Categorizes an issue or PR as relevant to SIG Node. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 2, 2019
@nolancon
Copy link
Author

nolancon commented Dec 2, 2019

/cc @adrianchiris @klueska @lmdaly

@k8s-ci-robot
Copy link
Contributor

@nolancon: GitHub didn't allow me to request PR reviews from the following users: adrianchiris.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @adrianchiris @klueska @lmdaly

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

adrianchiris
adrianchiris suggested changes Dec 2, 2019
View reviewed changes
pkg/kubelet/cm/topologymanager/topology_manager.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/topologymanager/policy_best_effort.go Outdated
@@ -105,21 +124,21 @@ func mergeProvidersHints(policy Policy, numaNodes []int, providersHints []map[st
// If hints is nil, insert a single, preferred any-numa hint into allProviderHints.
if len(hints) == 0 {
klog.Infof("[topologymanager] Hint Provider has no preference for NUMA affinity with any resource")
allProviderHints = append(allProviderHints, []TopologyHint{{nil, true}})
allProviderHints = append(allProviderHints, []TopologyHint{{defaultAffinity, true}})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of history here:
The reason we put defaultAffinity here instead of nil is simply because nil was introduced when adding SingleNumaNode policy since we had one merge logic that dealt with all policies.

since Single NUMA policy has now its own logic, this is no longer needed, thus we restore the original value.
Same applies for L#135,141

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we should keep these semantics here. It makes it clear what the difference is between having a nil Affinity field, an empty Affinity field, and a full Afinity field.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, makes sense - I will update this.

pkg/kubelet/cm/topologymanager/policy_best_effort.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/topologymanager/policy_best_effort.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/topologymanager/policy_restricted.go Outdated Show resolved Hide resolved
pkg/kubelet/cm/topologymanager/policy_single_numa_node.go Outdated Show resolved Hide resolved
@adrianchiris adrianchiris mentioned this pull request Dec 2, 2019
Closed
@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 2, 2019
nolancon and others added 14 commits January 16, 2020 08:13
Move test case 'HintProvider returns empty non-nil map[string][]Topol…
57661ee 
…ogyHint' into specific policy tests.
Move test case "HintProvider returns -nil map[string][]TopologyHint f…
599217d 
…rom provider" into specific policy tests
Move test case "HintProvider returns empty non-nil map[string][]Topol…
baeff9e 
…ogyHint from

provider" into specific policy tests.
Move test case "Single TopologyHint with Preferred as true and
6460ef6 
NUMANodeAffinity as nil" into specific policy tests.
Move test case "Single TopologyHint with Preferred as false and
401a2bb 
NUMANodeAffinity as nil" into specific policy tests.
Move test case "Two providers, 1 hint each, no common mask" into
f639da7 
specific policy tests.
Move test case "Two providers, 1 hint each, same mask, 1 preferred, 1
a38a256 
not 1/2" into specific policy test.
Move test case "Two providers, 1 hint each, same mask, 1 preferred, 1
681c42b 
not 2/2" into specific policy tests
Move test case "Two providers, 1 with 2 hints, 1 with single
8b3f6e6 
non-preferred hint matching" into specific policy tests
Update "Single NUMA hint generation" expected affinity to nil
92eb7cd
Use reflect.DeepEqual check in policy_test.go
5e23517
@klueska
Cleanup use of defaultAffinity in mergePermutation of TopologyManager
94489c1
@klueska
Rename TopologyManager test TestPolicyBestEffortMerge for consistency
2905fff
@klueska
Update TopologyManager single-numa-node logic to handle "don't cares"
7069b1d 
The logic has been updated to match the logic of the best-effort policy
except in two places:

1) The hint filtering frunction has been updated to allow "don't care"
hints encoded with a `nil` affinity mask, to pass through the filter in
addition to hints that have just a single NUMA bit set.

2) After calculating the `bestHint` we transform "don't care" affinities
encoded as having all NUMA bits set in their affinity masks into "don't
care" affinities encoded as `nil`.
@k8s-ci-robot k8s-ci-robot added do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. and removed do-not-merge/contains-merge-commits Indicates a PR which contains merge commits. labels Jan 16, 2020
@nolancon nolancon changed the title [WIP] Updated - topologymanager: Add Merge method to Policy Updated - topologymanager: Add Merge method to Policy Jan 16, 2020
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 16, 2020
@klueska
Copy link
Contributor

klueska commented Jan 17, 2020

/approve
/lgtm

I'm a big fan of being able to walk commit by commit through a PR and see all the final changes build up one-by one (instead of having a change in an earlier commit be overwritten by a later one). A good example of this is the breakdown for the moving of the tests in policy_test.go. It would be nice if the other commits built on each other similarly, but we've iterated on this one quite alot, and I'm happy with the final result, so let's go ahead and merge this one. In the future, let's see if we can break them down as described all the way through.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 17, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: klueska, nolancon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 17, 2020
@nolancon
Copy link
Author

/kind cleanup

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Jan 17, 2020
@k8s-ci-robot k8s-ci-robot merged commit 50f9ea7 into kubernetes:master Jan 17, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.18 milestone Jan 17, 2020
This was referenced Jan 21, 2020
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@klueska klueska klueska requested changes

@adrianchiris adrianchiris adrianchiris requested changes

@krmayankk krmayankk Awaiting requested review from krmayankk

@thockin thockin Awaiting requested review from thockin

@lmdaly lmdaly Awaiting requested review from lmdaly

Assignees

@klueska klueska

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubelet cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note-none Denotes a PR that doesn't merit a release note. sig/node Categorizes an issue or PR as relevant to SIG Node. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
v1.18
Development

Successfully merging this pull request may close these issues.
4 participants
@nolancon @k8s-ci-robot @klueska @adrianchiris