CIS 2.6.1 share-nodes argument not working

[rgarcia89]

Setup Details

CIS Version : 2.6.1
Build: f5networks/k8s-bigip-ctlr:2.6.1
BIGIP Version: Big IP v16.1.1 (Build 0.0.16)
AS3 Version: 3.28
Agent Mode: AS3
Orchestration: K8S
Orchestration Version: 1.20.12
Pool Mode: Nodeport

Description

Setting --share-nodes=true in the CIS arguments is not making nodes to be created in the common partition.

Steps To Reproduce

1. Create a CIS deployment that passes the --share-nodes=true argument
2. Deploy as AS3 declaration that does not include the share-nodes argument for pool members
3. Check on the F5 which tenant the nodes are linked to

[trinaths]

@rgarcia89 - Please share complete CIS configuration and the as3 configMap, svc and deployments to validate this issue at your end.

[rgarcia89]

@trinaths
CIS declaration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8s-bigip-ctlr-lb-int-1-ams1
  namespace: f5
spec:
  # DO NOT INCREASE REPLICA COUNT
  replicas: 1
  progressDeadlineSeconds: 600
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: k8s-bigip-ctlr
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      name: k8s-bigip-ctlr
      labels:
        app: k8s-bigip-ctlr
      annotations:
        seccomp.security.alpha.kubernetes.io/defaultProfileName: unconfined
        seccomp.security.alpha.kubernetes.io/pod: unconfined
    spec:
      # Name of the Service Account bound to a Cluster Role with the required
      # permissions
      serviceAccountName: bigip-ctlr
      containers:
        - name: k8s-bigip-ctlr
          image: "f5networks/k8s-bigip-ctlr:2.6.1"
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /health
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 15
            periodSeconds: 15
            successThreshold: 1
            timeoutSeconds: 15
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /health
              port: 8080
              scheme: HTTP
            initialDelaySeconds: 30
            periodSeconds: 30
            successThreshold: 1
            timeoutSeconds: 15
          env:
            - name: BIGIP_USERNAME
              valueFrom:
                secretKeyRef:
                  # Replace with the name of the Secret containing your login
                  # credentials
                  name: bigip-login
                  key: username
            - name: BIGIP_PASSWORD
              valueFrom:
                secretKeyRef:
                  # Replace with the name of the Secret containing your login
                  # credentials
                  name: bigip-login
                  key: password
          command: ["/app/bin/k8s-bigip-ctlr"]
          args: [
              # See the k8s-bigip-ctlr documentation for information about
              # all config options
              # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest
              "--bigip-username=$(BIGIP_USERNAME)",
              "--bigip-password=$(BIGIP_PASSWORD)",
              "--bigip-url=https://lb-int-1",
              "--bigip-partition=k8s",
              "--pool-member-type=nodeport",
              "--manage-ingress=false",
              "--manage-configmaps=true",
              "--insecure=true",
              "--log-level=debug",
              "--log-as3-response=true",
              "--agent=as3",
              "--share-nodes=true",
              "--disable-teems",
              "--hubmode=true",
              "--node-label-selector=f5role=worker",
              "--use-secrets=false",
              "--namespace=f5"
            ]

AS3 declaration

kind: ConfigMap
apiVersion: v1
metadata:
  name: f5-as3-sensu-system
  namespace: f5
  labels:
    f5type: virtual-server
    as3: "true"
data:
  template: |
    {
        "class": "AS3",
        "action": "deploy",
        "persist": false,
        "declaration": {
            "class": "ADC",
            "schemaVersion": "3.28.0",
            "id": "f5-as3-sensu-system",
            "label": "f5-as3-sensu-system",
            "remark": "AS3 Declaration - NS sensu-system",
            "sensu-system_tenant": {
                "class": "Tenant",
                "grafana_app": {
                    "class": "Application",
                    "template": "generic",
                    "grafana_app": {
                        "class": "Service_HTTPS",
                        "remark": "grafana.dev.xyz",
                        "redirect80": true,
                        "virtualAddresses": [
                            {
                            "use": "grafana_serviceAddress"
                            }
                        ],
                        "virtualPort" : 443,
                        "allowVlans": [
                            "/Common/VL-lb-int"
                        ],
                        "serverTLS": { "bigip": "/Common/ssl_.xyz" },
                        "snat": "auto",
                        "pool": "grafana_pool"
                    },
                    "grafana_pool": {
                        "class": "Pool",
                        "loadBalancingMode": "round-robin",
                        "monitors": [
                            "tcp"
                        ],
                        "members": [
                            {
                                "servicePort": 80,
                                "serverAddresses": []
                            }
                        ]
                    },
                    "grafana_serviceAddress": {
                        "class": "Service_Address",
                        "virtualAddress": "10.1.1.1",
                        "arpEnabled": true,
                        "icmpEcho": "enable",
                        "routeAdvertisement": "all",
                        "trafficGroup":"/Common/traffic-1"
                    }
                }
            }
        }
    }

Rest should be straight forward.
Looking foward to hear from you

[mdditt2000]

@rgarcia89 @trinaths I believe CIS is working correctly. The use of --share-nodes=true in the CIS arguments is only for Ingress, Routes and CRDs as tested below

https://github.com/mdditt2000/kubernetes-1-19/tree/master/cis%202.7/github/2101

For ConfigMap add sharednodes to the AS3 declaration as shown below.

"members": [
                            {
                                "servicePort": 80,
                                "serverAddresses": [],
                                "shareNodes": true     ---------- Please add
                            }
                        ]
                    },

[rgarcia89]

@mdditt2000 well that is what I got recommended in the slack channel. That's why I was expecting it to work also for AS3.
Is there any special reason why it has not been implemented for that as well?

[trinaths]

This requires a document update. Created CONTCNTR-3052 for internal tracking.

[trinaths]

Document updated. Closing this issue.