perf: [WIN-NPM] add all cached NetworkPolicies to a Pod at once #1893
Conversation
huntergregory
changed the title
huntergregory
changed the title
huntergregory
force-pushed
the
hgregory/04-04-update-pod-quicker
branch
from
f2620f1
to
15857d9
Compare
huntergregory
force-pushed
the
hgregory/04-04-update-pod-quicker
branch
from
15857d9
to
9ba6ecc
Compare
npm/config/config.go
Outdated
| defaultResyncPeriod = 15 | ||
| defaultApplyMaxBatches = 100 | ||
| defaultApplyInterval = 500 | ||
| defaultMaxBatchedACLsPerPod = 100 |
There was a problem hiding this comment.
Lets make this smaller, if there is an issue with applying ACLs due to some weird limit of data type they are using underneath, we can end up in a limbo of not being able to apply a bunch of network policies at once.
Smaller means more network policies are guaranteed to get applied. Wdyt ?
There was a problem hiding this comment.
discussed offline to make this 30
| } | ||
|
|
||
| klog.Infof("[PolicyManager] finished applying all rules to endpoint for batch %d out of %d. endpoint ID: %s", i+1, len(ruleBatches), epToModifyID) | ||
| for policyKey := range policyKeys { |
There was a problem hiding this comment.
What if you applied a portion of policies and you error out ? you need to persist the state of policies you already applied on that endpoint so we dont end up with duplicate policies?
| "ListeningPort": 10091, | ||
| "ListeningAddress": "0.0.0.0", | ||
| "ApplyMaxBatches": 100, | ||
| "ResyncPeriodInMinutes": 15, |
There was a problem hiding this comment.
Add windows ds to this yaml.
There was a problem hiding this comment.
discussed offline that there may be script dependencies on either yaml
| } | ||
| } | ||
|
|
||
| return nil |
There was a problem hiding this comment.
return list of policyKeys you applied here
|
|
||
| if err := dp.policyMgr.AddAllPolicies(toAddPolicies, endpoint.id, endpoint.ip); err != nil { |
There was a problem hiding this comment.
make sure to update netpolrtef even if this errors out. because we could have applied a portion of the batch of policies
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Reason for Change:
Reduces time spent on ACL SysCalls by 99% at mid/high scale.
Specifically, when all of a Pod's NetworkPolicies have been cached. For example, for any Pod CRUD after NPM has reached steady state. Further optimized in #1900.
Issue Fixed:
Second of three PRs addressing Windows limitations in #1614.
Requirements:
Notes:
Experiment
500 NetworkPolicies applied to a Pod.
Results
MaxBatchedACLsPerPod=33): 1 min 10 seconds for all Pods (even adding ACLs to the 15th Pod).