fqdn: Delay ipcache upserts until policies have been updated #14084
Conversation
jrajahalme
added
kind/bug
jrajahalme
requested review from
nathanjsweet
and removed request for
a team
maintainer-s-little-helper
bot
added
backport/1.7
kind/backports
|
test-backport-1.7 |
|
Locally passed runtime FQDN tests. |
|
This is not a backport even though backport labels were automatically added :-) |
tgraf
force-pushed
the
pr/jrajahalme/1.7-policy-before-ipcache
branch
from
cc49269
to
a799ba7
Compare
|
Fixed up unit tests |
tgraf
force-pushed
the
pr/jrajahalme/1.7-policy-before-ipcache
branch
from
a799ba7
to
4b6e697
Compare
|
test-backport-1.7 |
Add a map for newly allocated identities to ipcache.AllocateCIDR functions that the caller can use to upsert the IPs to ipcache later, after affected endpoint policy maps have been updated. Use this new functionality on the DNS proxy code path, that makes sure that new policy map entries are in place before an IP received from a DNS server is placed in ipcache. This is really straightforward as the logic for waiting was already in place for delaying the forwarding of the DNS response. Policy update path is still allowing ipcache upserts at policy ingestion time rather than waiting for the policy maps to be updated. This means that new, more specific CIDRs (e.g., 10.0.0/24) in policies can still cause momentary drops on traffic currently using a less specific CIDR (e.g., 10.0/16). Similarly the DNS poller path still upserts to ipcache before policies have been updated. Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
jrajahalme
force-pushed
the
pr/jrajahalme/1.7-policy-before-ipcache
branch
from
4b6e697
to
3b1ef2b
Compare
|
test-backport-1.7 |
|
CI infra problems: |
|
test-backport-1.7 |
|
|
|
test-backport-1.7 |
1 similar comment
|
test-backport-1.7 |
… regenerated by endpoints. Move ipcache CIDR upserts and releases to the policy reaction queue, where upserts can be executed after regenerations have been completed, i.e. after endpoint policy maps have been updated. This way IP addresses are mapped to newly allocated identities only after endpoint policy maps are ready to classify them. Correspondingly, on deletes the to-be-deleted CIDR identities are first deleted from ipcache so that when they are deleted from endpoint policy maps they are no longer used in classification. Releases of CIDR identities must still be serialized with ipcache upserts via the policy reaction queue so that they are executed in the same order w.r.t. ipcache upserts as policy deletes and adds. Signed-off-by: Jarno Rajahalme <jarno@covalent.io>
jrajahalme
force-pushed
the
pr/jrajahalme/1.7-policy-before-ipcache
branch
from
d2911b0
to
86ef8aa
Compare
|
Simplified 2nd commit a lot, re-testing |
|
test-backport-1.7 |
|
Same failure in two different tests: Cilium agent in ImagePullBackoff in Upgrade Downgrade test. Seems like the docker rate limiting issue? |
|
@jrajahalme have you rebased? I believe we only merged the v1.7 backport for the image pull secrets mitigation on Friday afternoon. The CI run on this PR looks like it was likely run before that other PR was merged. |
|
Everything was green except the k8s tests which failed with imagepullbackoff due to docker ratelimits, so I will re-trigger to see if that rebases it to make CI pass. |
|
test-backport-1.8 EDIT: Typo, caused the failed "Runtime-4.9" job below which can be safely ignored. |
|
test-backport-1.7 |
|
The only failures are triggered because I accidentally ran the 1.8 backports jobs, which will guarantee to fail. All of the legitimate test checks were successful. Merging. |
Add a map for newly allocated identities to ipcache.AllocateCIDR
functions that the caller can use to insert the IPs to the IP cache later,
after affected endpoint policy maps have been updated.
Use this new functionality on the DNS proxy and policy update code
paths, that makes sure that new policy map entries are in place before
an IP received from a DNS server or a CIDR included in a policy is placed
in IP cache. This is really straightforward as the logic for waiting was
already in place.
The DNS poller path still inserts in to IP cache before policies
have been updated.
Signed-off-by: Jarno Rajahalme jarno@covalent.io