Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Register consecutive priorities in batch #1331

Merged
merged 2 commits into from Oct 15, 2020

Conversation

Dyanngg
Copy link
Contributor

@Dyanngg Dyanngg commented Oct 1, 2020

This PR aims to improve the efficiency of registering Antrea-native policy priorities in the OpenFlow priority space. Fundamental idea is to take advantage of the fact that, when the agent receives internal NP created for those policies, it already knows the number of absolute consecutive Priorities that needs to be registered. Those Priorities will have the same Tier and Policy priority, and consecutive rule priorities as they correspond to the 1st, 2nd, and etc. rules of the ingress/egress rule of that policy.

Thus, when registering priorities, we can break these priorities into zones (each zone of same Tier and Policy priority), and register each zone in batch. This eliminates the problematic scenario where asynchronously registering a series of priorities cause existing priorities to be reassigned each time.

The priorityAssigner module is refactored to accommodate this change. The following illustrations demonstrates the new algorithm that it uses to insert consecutive priorities. For each diagram, the first row depicts the current ofPriority space, and where the initial heuristic function puts in the inserting priorities.

Case 1 The spaces computed by the heuristic function is unoccupied, AND the priorities immediately lower than those being inserted, if exists, are registered below these ofPriorities, AND the priorities immediately higher than those being inserted, if exists, are registered above these ofPriorities. In this case, we simply assign the ofPriorities returned by the heuristic function.
image

Case 2 The ofPriorities returned by the heuristic function overlaps with existing priorities/are out of place. If the priorities to be registered overlaps with lower priorities/are lower than the lower priorities, and the gap between lowerBound for insertion and upperBound for insertion is very large, then we insert these priorities above the lowerBound, offsetted by a constant zoneOffset. This offset gives buffer in case priorities are again created in between those zone. Vice versa for priorities overlaps with/higher than higher priorities.
image

Case 3 The ofPriorities returned by the heuristic function overlaps with existing priorities/are out of place, and the gap between lowerBound for insertion and upperBound for insertion is tight. In that case, we simply register the priorities in the middle of the gap.
image

Case 4 The gap between lowerBound for insertion and upperBound for insertion is not big enough for insertion. In this case, we push lower priorities and/or higher priorities towards each side in attempt to make enough room for the registering priorities, while in the meantime try to reassign as few existing priorities as possible. For example, in the following diagram, the yellow Priorities will be kept where they were to save reassignments.
image

@antrea-bot
Copy link
Collaborator

Thanks for your PR.
Unit tests and code linters are run automatically every time the PR is updated.
E2e, conformance and network policy tests can only be triggered by a member of the vmware-tanzu organization. Regular contributors to the project should join the org.

The following commands are available:

  • /test-e2e: to trigger e2e tests.
  • /skip-e2e: to skip e2e tests.
  • /test-conformance: to trigger conformance tests.
  • /skip-conformance: to skip conformance tests.
  • /test-whole-conformance: to trigger all conformance tests on linux.
  • /skip-whole-conformance: to skip all conformance tests on linux.
  • /test-networkpolicy: to trigger networkpolicy tests.
  • /skip-networkpolicy: to skip networkpolicy tests.
  • /test-windows-conformance: to trigger windows conformance tests.
  • /skip-windows-conformance: to skip windows conformance tests.
  • /test-windows-networkpolicy: to trigger windows networkpolicy tests.
  • /skip-windows-networkpolicy: to skip windows networkpolicy tests.
  • /test-hw-offload: to trigger ovs hardware offload test.
  • /skip-hw-offload: to skip ovs hardware offload test.
  • /test-all: to trigger all tests (except whole conformance).
  • /skip-all: to skip all tests (except whole conformance).

@codecov-commenter
Copy link

codecov-commenter commented Oct 1, 2020

Codecov Report

Merging #1331 into master will decrease coverage by 0.10%.
The diff coverage is 85.21%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1331      +/-   ##
==========================================
- Coverage   64.60%   64.49%   -0.11%     
==========================================
  Files         157      159       +2     
  Lines       12626    12667      +41     
==========================================
+ Hits         8157     8170      +13     
- Misses       3622     3644      +22     
- Partials      847      853       +6     
Flag Coverage Δ
#integration-tests 44.76% <0.00%> (-0.04%) ⬇️
#kind-e2e-tests 50.74% <45.77%> (+0.05%) ⬆️
#unit-tests 42.11% <84.78%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/agent/types/networkpolicy.go 88.23% <50.00%> (+19.00%) ⬆️
pkg/agent/controller/networkpolicy/reconciler.go 69.03% <64.70%> (+0.59%) ⬆️
pkg/agent/controller/networkpolicy/priority.go 87.85% <88.39%> (-2.15%) ⬇️
pkg/agent/controller/networkpolicy/cache.go 84.98% <100.00%> (+0.39%) ⬆️
pkg/controller/networkpolicy/tier.go 90.00% <0.00%> (-10.00%) ⬇️
...kg/controller/networkpolicy/store/networkpolicy.go 77.96% <0.00%> (-3.39%) ⬇️
pkg/monitor/controller.go 48.52% <0.00%> (-2.95%) ⬇️
pkg/antctl/command_list.go 31.48% <0.00%> (-1.22%) ⬇️
pkg/antctl/command_definition.go 41.96% <0.00%> (-0.44%) ⬇️
...ntroller/networkpolicy/networkpolicy_controller.go 80.69% <0.00%> (-0.14%) ⬇️
... and 7 more

@Dyanngg Dyanngg force-pushed the batch-priority branch 2 times, most recently from 56bd497 to c832a83 Compare October 1, 2020 23:25
Copy link
Contributor

@antoninbas antoninbas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to make sure that your unit tests cover all the cases described in the PR description (thanks for the comprehensive description BTW!) ?

pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/types/networkpolicy.go Outdated Show resolved Hide resolved
pkg/agent/types/networkpolicy.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority_test.go Outdated Show resolved Hide resolved
@Dyanngg
Copy link
Contributor Author

Dyanngg commented Oct 6, 2020

Just want to make sure that your unit tests cover all the cases described in the PR description (thanks for the comprehensive description BTW!) ?

I have updated UT and I believe it now covers all the cases mentioned in the PR description. Also, I realized that the original PR description for case 4 is inaccurate, I have updated it accordingly as well.

@Dyanngg
Copy link
Contributor Author

Dyanngg commented Oct 6, 2020

/test-all

Copy link
Contributor

@antoninbas antoninbas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but we should get another pair of eyes on this

pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
for i := target; i >= 0; i-- {
if cost, exists := costMap[i]; exists && cost.cost < minCost {
// make sure that the reassign range makes up for all Priorities to be registered.
if int(cost.upperBound-cost.lowerBound)+1 == numNewPriorities+cost.cost {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this guaranteed by your algorithm above? If yes, wouldn't it make sense to log something or return an error if this doesn't hold?

Copy link
Contributor Author

@Dyanngg Dyanngg Oct 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it is not guaranteed by the algorithm above and there should not be an error thrown. Consider a case where we need to find 3 empty spots below lowerBound and/or above upperBound. In the first pass, the algorithm tries to fill the table on the cost and where the lowerBound would be, if we were to use 1, 2 and 3 lower empty spots. For 1 and 2, when a cost is found, the upperBound will be temporarily set to the current upperBound which has not shifted at all. Those costs however only makes sense if in the second pass, the algorithm is able to find cost of getting 2 and 1 empty spots from above, in which case the total cost and new upperBound will be updated.
In other words, when int(cost.upperBound-cost.lowerBound)+1 != numNewPriorities+cost.cost, it simply means that this combination of (use x empty slots below, use y empty slots above) is not feasible based on the current map, but it's not an error.

pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
pkg/agent/controller/networkpolicy/priority.go Outdated Show resolved Hide resolved
@Dyanngg
Copy link
Contributor Author

Dyanngg commented Oct 8, 2020

/test-all

@codecov-io
Copy link

codecov-io commented Oct 8, 2020

Codecov Report

Merging #1331 into master will increase coverage by 0.00%.
The diff coverage is 85.21%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1331   +/-   ##
=======================================
  Coverage   64.32%   64.32%           
=======================================
  Files         159      159           
  Lines       12674    12684   +10     
=======================================
+ Hits         8152     8159    +7     
+ Misses       3668     3667    -1     
- Partials      854      858    +4     
Flag Coverage Δ
#integration-tests 44.86% <0.00%> (-0.04%) ⬇️
#kind-e2e-tests 50.44% <45.77%> (+0.14%) ⬆️
#unit-tests 42.02% <84.78%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/agent/types/networkpolicy.go 88.23% <50.00%> (+19.00%) ⬆️
pkg/agent/controller/networkpolicy/reconciler.go 69.03% <64.70%> (+0.59%) ⬆️
pkg/agent/controller/networkpolicy/priority.go 87.85% <88.39%> (-2.15%) ⬇️
pkg/agent/controller/networkpolicy/cache.go 84.98% <100.00%> (+0.39%) ⬆️
pkg/controller/traceflow/controller.go 61.29% <0.00%> (-1.30%) ⬇️
...ntroller/networkpolicy/networkpolicy_controller.go 80.69% <0.00%> (-0.14%) ⬇️

@Dyanngg
Copy link
Contributor Author

Dyanngg commented Oct 13, 2020

/test-all

@Dyanngg Dyanngg merged commit 70d3e65 into antrea-io:master Oct 15, 2020
@Dyanngg Dyanngg deleted the batch-priority branch October 29, 2020 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants