[cni-cilium] New module

[konstantin-axenov]

Preflight Checklist

• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.

Use case. Why is this important?

Cilium seems as fine solution for most cases and its performance is rather good. Kube-router is quite slow if there are a lot of NetworkPolicy rules. We assume that switch to cilium can help us.

Proposed Solution

1. New module is available only for Static clusters
2. How to switch from flannel to cilium and back?
3. Can we get rid of kube-proxy?
4. Observability and diagnostics of cilium?
5. Kube-router can't be enabled if cilium is enabled
6. What about Network Policies? Does cilium really solves performance issues? Cilium Clusterwide Network Policy (CCNP) or a Cilium Network Policy (CNP) vs NetworkPolicy?
7. Does Deckhouse has required kernel version?
8. Can we use cilium for HA NAT instead of network-gateway + keepalived?
9. How does it combine Kubernetes' NetworkPolicy и CiliumNetworkPolicy? Does it intersect, override, or cancel each others rules?
10. How will the Cilium interact with our Unready Node patches.
11. Can we remove HostWithFailover mode from ingress-nginx and replace it with DSR and Services with trafficPolicy: Cluster.
12. Is netfilter's FORWARD chain's DROP policy respected in MetalLB setups?

Additional Information

No response

[xmulligan]

@konstantin-axenov Thanks for adding support for Cilium! Would you mind adding yourself to the Cilium Users doc? We need this for CNCF graduation https://github.com/cilium/cilium/blob/master/USERS.md

[konstantin-axenov]

@konstantin-axenov Thanks for adding support for Cilium! Would you mind adding yourself to the Cilium Users doc? We need this for CNCF graduation https://github.com/cilium/cilium/blob/master/USERS.md

Opened cilium/cilium#19804