-
Notifications
You must be signed in to change notification settings - Fork 339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for EgressIP and ExternalTrafficPolicy=Local to coexist #4265
Conversation
✅ Deploy Preview for subtle-torrone-bb0c84 ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
ffba69d
to
f23039a
Compare
TODO: Add tests. |
c9f861d
to
c635ee0
Compare
271d731
to
28fb5ed
Compare
done! e2e and unit tests are done. |
@trozet PTAL this one is ready for reviews! |
I tested the secondary networks EIP test case on lgw and it works as expected with this PR with no changes: service traffic via eth1:
EIP traffic from pod to npclient:
I need to add an e2e test coverage for this. |
d48969d
to
f75c32d
Compare
d5105cb
to
ba0ef00
Compare
f5533f4
to
7301aeb
Compare
yayyyy I finally fixed all those 72 unit tests and my e2es are looking good |
@trozet I had to add client side db indexes for QoSRules because when two nodes were getting added at the same time in nonIC env, we were ending up inserting duplicate QoS Rules because nodes are handled in multiple threads and so two threads were trying to insert qos rules at the same time leading to multiple predicates found libovsdb error.
The only other feature using QoS rules on switch is EgressQoS in addition to EgressIPs. So introducing client side indexes means from this version onwards eqos feature will start creating qos rules in OVN with the new externalIDs that are unique per QoS Rule. That means when users upgrade from older versions to new versions with EgressQoS, there will be stale left overs which I don't plan to clean up in this PR. Anyways since EgressQoS feature is still not GA-ed and is only Dev Preview to the point where even the API may change in a breaking way including the name, I think its ok to start using db indexes for EQoS without worrying about updating older QoSRules i.e upgrades for QoS is not handled at the moment so stale QoS Rules will be left behind without the new externalIDs but that should be fine |
adding indexes didn't work actually. The reason here is because the race was in the cache level, so since 2 threads try to insert at same time, we hit some hiccupps.. I have instead used lock now (I added a new commit for dbindexes, but we can also move that out of this PR if that's too much) |
Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
This commit renames the AddressSetIPFamilyKey to IPFamilyKey so that in the next commit we can use it beyond just AddressSets. Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
This commit introduces a new LRP at 102 priority for EgressIP that matches on the 42 marked reply packets and simply "allows" that packet so that it prevents reply traffic from egressIP pods from ever hitting the reroute policies at priority 100 sample: 102 pkt.mark == 42 allow libovsdb updates/creates LRPs on startup so upgrades are handled automatically. also creating the policy is done one-time at startup via initClusterPolicies Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
7301aeb
to
176e88d
Compare
This commit introduces DBIndexing for QoSTable. Note that I have not added a qos_syncer or anything to cleanup older EQoS rules because this feature is not GA yet, I don't see a need to support upgrades and anyways we are planning to make major changes to this feature moving forward. Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
176e88d
to
ada07f5
Compare
@trozet : I have addressed your comments. PTAL Unintentionally I added the dbindexes commit for eqos: 07fa4d1 which is probably not really needed ATM; if you prefer to keep that commit as separate PR I can cut it out. |
ada07f5
to
40aa513
Compare
This commit introduces qos rules on every node's switch to mark packets emerging from EIP pods if they are reply packets. These are the packets that should not be reRouted to other node in the cluster. So basically we mark the SYNACK packets that are matched on by the LRP we saw in the previous commit. SYN packets are not marked as these are egress traffic from EIP pods. sample: from-lport 103 (ip4.src==$a4548040316634674295 && ct.trk && ct.rpl) mark=42 This let's these two features (ETP=local + EIP) to be used at the same time. Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
This commit adds e2e for primari EIPs and ETP=local cross feature testing Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
This commit adds cross feature testing for etp=local and etp=cluster with secondary NIC EIPs. Signed-off-by: Surya Seetharaman <suryaseetharaman.9@gmail.com>
40aa513
to
7527fcf
Compare
- What this PR does and why is it needed
This PR fixes the broken interaction between egressIPs and ETP=local features.
Currently if an EIP served pod is on a different node than the egressNode and
it is also a backend to the ETP=local service type, then the reply packet destined
for the ingress service traffic served by the EIP pod get's re-routed to the egressNode
using the reRoute policies we add for EIPs. This breaks the service traffic.
This PR implements the solution we did via https://issues.redhat.com/browse/FDP-42
to ensure these features work together seamlessly.
Solution:
Step1: Add a qos-rule to the node switches:
from-lport 103 (ip4.src == $a4548040316634674295 && ct.trk && ct.rpl) mark=42
where
a4548040316634674295
is the address set that contains all EIP pods.Step2: Add new LRP at 102 to match on mark and simply "allow" thus skipping the re-routes
LRP:
sample:
However we must do the mark changes ONLY for local zone pods not for remote zone pods, else it breaks traffic since mark doesn't persist outside the local node.
- Special notes for reviewers
- How to verify it
The
2024-04-25T14:09:40.7078121Z �[0mLoad Balancer Service Tests with MetalLB �[0m�[1mShould ensure load balancer service works when ETP=local and backend pods are also egressIP served pods�[0m
is running on both v4 and v6 lanes for LGW gateway mode and here the egressNode is different from the node where the pod livesAND
should work on secondary node interfaces for ETP=local and ETP=cluster when backend pods are also served by EgressIP
is running on SGW mode both v4 & v6- Description for the changelog
Support ETP=local with EIP
/label ci