Kubernetes in VPC - LB ip in pending state #8178
Description
ISSUE TYPE
- Bug Report
COMPONENT NAME
Kubernetes - VPC - Firewall - CCM
CLOUDSTACK VERSION
4.18.1
CONFIGURATION
- Advanced networking
- Kubernetes Cluster deployed in a VPC network
OS / ENVIRONMENT
- Ubuntu 22.04 kvm nodes
SUMMARY
I am creating a kubernetes cluster inside a VPC.
I am deploying ingres-nginx on this cluster.
All goes well except that the CCM (cloud container manager) is not able to reserve an external IP address for the loadbalancer ingress.
This seems to be because the CCM expects the network Firewall service to be available but VPC work with Network ACLs instead
(workaround was to manually assign a public ip and do loadbalancing towards the ingress nginx ports)
STEPS TO REPRODUCE
* create VPC
* create isolated network inside the VPC
* create a kubernetes cluster in that vpc network
* deploy ingress-nginx
* check cloud-container-manager logs or svc status to see that an external ip is not getting assigned
EXPECTED RESULTS
Expecting that CCM can assign an external IP.
ACTUAL RESULTS
The CCM cannot assign an external IP and it seems it is because it expects the Firewall service to be available in the VPC.
error processing service ingress-nginx/ingress-nginx-controller (will retry): failed to ensure load balancer: error creating new firewall rule for public IP 57eb71b2-283f-4ab7-a5b8-ff227817f7f0, proto tcp-proxy, port 80, allowed [0.0.0.0/0]: CloudStack API error 431 (CSExceptionErrorCode: 9999): There is no new provider for IP X.X.X.X of service Firewall!