F5 CIS in nodeport mode unable to create a VIP for "VirtualServer" crd

[kkothapelly]

Before you raise a new bug, please ensure you have visited the troubleshooting guide

• Visited the troubleshooting guide

Setup Details

FIC Version : Version: 0.1.8
CIS Version : 2.10.1
FIC Build: f5networks/f5-ipam-controller:latest
CIS Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP 17.0.0.1-0.0.4.ALL
AS3 Version: 3.39.0.7
Orchestration: Tanzu
Orchestration Version: 1.5.4
Additional Setup details: <Platform/CNI Plugins/ cluster nodes/ etc> : Antrea CNI

Description

When we deploy a F5 CIS controller in nodeport mode in Tanzu k8s cluster, we are able to deploy the k8s native L4 loadbalancer service. However, when we try to deploy a F5 crd "VirtualServer" to use the L7 applications, its unable to create the VIP object in the Big IP controller. K8s shows the Virtualserver CRD is created and IPAM assigns an IP address to the VS, however the object is not getting created in the BIG-IP partition.

Steps To Reproduce

1. Deploy F5 CIS controller in nodeport mode
2. deploy F5 IPAM controller
3. Deploy a F5 virtualserver object

Expected Result

Actual Result

Diagnostic Information

F5 CIS controller pod logs shows virtual server config is missing the serviceport, however we confirm that it exits but still we see the error. please find the below service,VS config and the error in pod logs:

root@photon-JB [ ~/f5/L4 ]# cat 2-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: f5-hello-world
  name: f5-hello-world
spec:
  ports:
    - name: f5-hello-world
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    app: f5-hello-world
  type: ClusterIP

root@photon-JB [ ~/f5/L4 ]# cat 3-vs.yaml
apiVersion: "cis.f5.com/v1"
kind: VirtualServer
metadata:
 name: cafe-virtual-server
 labels:
   f5cr: "true"
spec:
 host: cafe.tanzu.lab
 ipamLabel: Prod
 pools:
 - path: /coffee
   service: f5-hello-world
   servicePort: 8080

Pod logs:

2022/10/26 09:29:52 [DEBUG] [AS3] posting request to https://172.16.2.244/mgmt/shared/appsvcs/declare/bigip-partition
2022/10/26 09:29:53 [ERROR] [AS3] Raw response from Big-IP: map[code:422 declarationFullId: errors:[/bigip-partition/Shared/f5_hello_world_8080_default_cafe_tanzu_lab/members/0: should have required property 'servicePort'] message:declaration is invalid] {"$schema":"https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/3.38.0/as3-schema-3.38.0-3.json","class":"AS3","declaration":{"bigip-partition":{"Shared":{"class":"Application","coffee_lb_8080_default":{"class":"Pool","members":[{"addressDiscovery":"static","serverAddresses":["172.16.48.36"],"servicePort":32764},{"addressDiscovery":"static","serverAddresses":["172.16.48.44"],"servicePort":32764},{"addressDiscovery":"static","serverAddresses":["172.16.48.35"],"servicePort":32764},{"addressDiscovery":"static","serverAddresses":["172.16.48.40"],"servicePort":32764},{"addressDiscovery":"static","serverAddresses":["172.16.48.47"],"servicePort":32764},{"addressDiscovery":"static","serverAddresses":["172.16.48.49"],"servicePort":32764}]},"crd_172_16_48_146_80":{"source":"0.0.0.0/0","translateServerAddress":true,"translateServerPort":true,"class":"Service_HTTP","virtualAddresses":["172.16.48.146"],"virtualPort":80,"snat":"auto","policyEndpoint":"/bigip-partition/Shared/crd_172_16_48_146_80_cafe_tanzu_lab_policy"},"crd_172_16_48_146_80_cafe_tanzu_lab_policy":{"class":"Endpoint_Policy","rules":[{"name":"vs_cafe_tanzu_lab_coffee_f5_hello_world_8080_default_cafe_tanzu_lab","conditions":[{"type":"httpHeader","name":"host","event":"request","all":{"values":["cafe.tanzu.lab"],"operand":"equals"}},{"type":"httpUri","name":"1","event":"request","index":1,"pathSegment":{"values":["coffee"],"operand":"equals"}}],"actions":[{"type":"forward","event":"request","select":{"pool":{"use":"f5_hello_world_8080_default_cafe_tanzu_lab"}}}]}],"strategy":"first-match"},"f5_hello_world_8080_default_cafe_tanzu_lab":{"class":"Pool","members":[{"addressDiscovery":"static","serverAddresses":["172.16.48.36"]},{"addressDiscovery":"static","serverAddresses":["172.16.48.44"]},{"addressDiscovery":"static","serverAddresses":["172.16.48.35"]},{"addressDiscovery":"static","serverAddresses":["172.16.48.40"]},{"addressDiscovery":"static","serverAddresses":["172.16.48.47"]},{"addressDiscovery":"static","serverAddresses":["172.16.48.49"]}]},"template":"shared","vs_lb_svc_default_coffee_lb_172_16_48_145_8080":{"class":"Service_TCP","virtualAddresses":["172.16.48.145"],"virtualPort":8080,"snat":"auto","pool":"coffee_lb_8080_default","profileL4":"basic"}},"class":"Tenant","defaultRouteDomain":0},"class":"ADC","controls":{"class":"Controls","userAgent":"CIS/v2.10.1 K8S/v1.22.9+vmware.1"},"id":"urn:uuid:85626792-9ee7-46bb-8fc8-4ba708cfdc1d","label":"CIS Declaration","remark":"Auto-generated by CIS","schemaVersion":"3.38.0"}}
2022/10/26 09:29:53 [ERROR] [AS3] Big-IP Responded with code: 422
2022/10/26 09:29:53 [DEBUG] [AS3] Posting failed tenants configuration in 30s seconds
2022/10/26 09:29:53 [DEBUG] Updating VirtualServer Status with {172.16.48.146 Ok} for resource name:cafe-virtual-server , namespace: default
 

Observations (if any)

When we deploy F5 CIS controller in nodeportlocal mode, and create teh virtualserver object with the same configuration, it works fine. However, nodeportlocal mode does not support k8s native L4 LoadBalancer.

L4 is supported with nodeport only. so we wanted to deploy cis in nodeport mode and create both L4 and L7 services.

[kkothapelly]

This issue resolved when we change the service type to nodeport in the service config.

type: ClusterIP > type: Nodeport