perf: [WIN-NPM] apply ipsets in background #1875
Conversation
huntergregory
force-pushed
the
hgregory/03-28-apply-ipsets-in-background
branch
from
4f2da6a
to
038137d
Compare
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
huntergregory
force-pushed
the
hgregory/03-28-apply-ipsets-in-background
branch
from
ff7062d
to
a6eaf90
Compare
npm/azure-npm.yaml
Outdated
| @@ -148,6 +148,9 @@ metadata: | |||
| data: | |||
| azure-npm.json: | | |||
| { | |||
| "WindowsNetworkName": "azure", | |||
There was a problem hiding this comment.
We can keep this network name for simplicity.
npm/config/config.go
Outdated
| WindowsNetworkName: util.AzureNetworkName, | ||
| ResyncPeriodInMinutes: defaultResyncPeriod, | ||
| WindowsNetworkName: util.AzureNetworkName, | ||
| ApplyDataPlaneMaxBatches: 0, |
There was a problem hiding this comment.
make sure these default are real numbers, if config map is not applied properly we should not be erroring out.
npm/pkg/dataplane/dataplane.go
Outdated
| @@ -464,3 +564,7 @@ func (dp *DataPlane) deleteIPSetsAndReferences(sets []*ipsets.TranslatedIPSet, n | |||
| } | |||
| return nil | |||
| } | |||
|
|
|||
| func (dp *DataPlane) configuredToApplyInBackground() bool { | |||
| return util.IsWindowsDP() && dp.ApplyDataPlaneMaxBatches > 0 && dp.ApplyDataPlaneInterval > 0 | |||
There was a problem hiding this comment.
not a fan of this, we should add a new flag which defines this new behavior.
Conformance and Cyclonus have succeeded for a few rounds (excluding Windows issues). Also finished multiple rounds of testing using these scripts: https://github.com/Azure/azure-container-networking/tree/master/test/scale |
npm/pkg/dataplane/dataplane.go
Outdated
| @@ -451,3 +547,7 @@ func (dp *DataPlane) deleteIPSetsAndReferences(sets []*ipsets.TranslatedIPSet, n | |||
| } | |||
| return nil | |||
| } | |||
|
|
|||
| func (dp *DataPlane) configuredToApplyInBackground() bool { | |||
| return util.IsWindowsDP() && dp.ApplyInBackground | |||
There was a problem hiding this comment.
Instead of checking for isWindowsDP everytime we need to applyDP, can we assign a global DP variable which we can set once in INIT and then only keep checking that value ?
| "PlaceAzureChainFirst": true | ||
| "PlaceAzureChainFirst": true, | ||
| "ApplyIPSetsOnNeed": false, | ||
| "ApplyInBackground": true |
Reason for Change:
Makes IPSet SysCalls 96% (23x) faster when there are no NetworkPolicies to process. Further optimized in #1900.
Issue Fixed:
First of three PRs addressing Windows limitations in #1614.
Requirements:
Notes:
Experiment
1.5K Pods, 10 NetworkPolicies
Results