KIC crashes due to stack overflow when broken config is applied in db mode

[pmalek]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When applying a broken config in DB mode KIC crashes due to stack overflow:

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc020f08398 stack=[0xc020f08000, 0xc040f08000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x1e66d61?, 0x2f27280?})
	/usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0xc00040be18 sp=0xc00040bde8 pc=0x438c9d
runtime.newstack()
	/usr/local/go/src/runtime/stack.go:1105 +0x5bd fp=0xc00040bfc8 sp=0xc00040be18 pc=0x45313d
runtime.morestack()
	/usr/local/go/src/runtime/asm_amd64.s:574 +0x8b fp=0xc00040bfd0 sp=0xc00040bfc8 pc=0x46d5cb

goroutine 546 [running]:
runtime.deductAssistCredit(0x10?)
	/usr/local/go/src/runtime/malloc.go:1201 +0x7b fp=0xc020f083a8 sp=0xc020f083a0 pc=0x40defb
runtime.mallocgc(0x10, 0x1cabde0, 0x1)
	/usr/local/go/src/runtime/malloc.go:932 +0xd0 fp=0xc020f08410 sp=0xc020f083a8 pc=0x40d6f0
runtime.convT(0x1cabde0, 0x40bb3d?)
	/usr/local/go/src/runtime/iface.go:331 +0x34 fp=0xc020f08448 sp=0xc020f08410 pc=0x40b614
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/deckerrors.ConfigConflictError.Is(...)
	/workspace/internal/dataplane/deckerrors/conflict.go:23
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/deckerrors.(*ConfigConflictError).Is(0x0?, {0x2120d60?, 0x2fd5f00?})
	<autogenerated>:1 +0x59 fp=0xc020f08488 sp=0xc020f08448 pc=0x177e159
errors.Is({0x2120d20, 0xc0023967f0}, {0x2120d60, 0x2fd5f00})
	/usr/local/go/src/errors/wrap.go:53 +0x156 fp=0xc020f08500 sp=0xc020f08488 pc=0x4779f6
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/sendconfig.ErrUpdateSkippedDueToBackoffStrategy.Is(...)
	/workspace/internal/dataplane/sendconfig/backoff_strategy.go:27
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/sendconfig.(*ErrUpdateSkippedDueToBackoffStrategy).Is(0x414705?, {0x2120d20?, 0xc0023967f0?})
	<autogenerated>:1 +0x6d fp=0xc020f08540 sp=0xc020f08500 pc=0x18fee6d
errors.Is({0x2120d60, 0x2fd5f00}, {0x2120d20, 0xc0023967f0})
	/usr/local/go/src/errors/wrap.go:53 +0x156 fp=0xc020f085b8 sp=0xc020f08540 pc=0x4779f6
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/deckerrors.ConfigConflictError.Is(...)
	/workspace/internal/dataplane/deckerrors/conflict.go:23
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/deckerrors.(*ConfigConflictError).Is(0x0?, {0x2120d60?, 0x2fd5f00?})
	<autogenerated>:1 +0x72 fp=0xc020f085f8 sp=0xc020f085b8 pc=0x177e172
errors.Is({0x2120d20, 0xc0023967e0}, {0x2120d60, 0x2fd5f00})
	/usr/local/go/src/errors/wrap.go:53 +0x156 fp=0xc020f08670 sp=0xc020f085f8 pc=0x4779f6
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/sendconfig.ErrUpdateSkippedDueToBackoffStrategy.Is(...)
	/workspace/internal/dataplane/sendconfig/backoff_strategy.go:27
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/sendconfig.(*ErrUpdateSkippedDueToBackoffStrategy).Is(0x414705?, {0x2120d20?, 0xc0023967e0?})
	<autogenerated>:1 +0x6d fp=0xc020f086b0 sp=0xc020f08670 pc=0x18fee6d
errors.Is({0x2120d60, 0x2fd5f00}, {0x2120d20, 0xc0023967e0})
	/usr/local/go/src/errors/wrap.go:53 +0x156 fp=0xc020f08728 sp=0xc020f086b0 pc=0x4779f6
github.com/kong/kubernetes-ingress-controller/v2/internal/dataplane/deckerrors.ConfigConflictError.Is(...)

Expected Behavior

No response

Steps To Reproduce

1. Deploy KIC with kong in dbmode
2. Apply the following config 

    
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: nginx
      name: nginx-sample-1
      namespace: default
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx-sample-1
      template:
        metadata:
          labels:
            app: nginx-sample-1
        spec:
          containers:
          - image: nginx
            name: nginx-sample-1
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 80
              name: http
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        konghq.com/methods: GET
        kubernetes.io/ingress.class: kong
      name: nginx-sample-1
      namespace: default
    spec:
      rules:
      - http:
          paths:
          - backend:
              service:
                name: nginx-sample-1
                port:
                  number: 80
            path: /sample/1
            pathType: ImplementationSpecific
    ---
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        ingress.kubernetes.io/service-upstream: 'true'
        konghq.com/plugins: nginx-sample-1 #,nginx-sample-1-rate # removing this breaks Kong but it does not validate on webhook
      name: nginx-sample-1
      namespace: default
    spec:
      ports:
      - name: http
        port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx-sample-1
    ---
    apiVersion: configuration.konghq.com/v1
    config:
      allow:
      - nginx-sample-1
      - app
      hide_groups_header: false
    kind: KongPlugin
    metadata:
      name: nginx-sample-1
      namespace: default
    plugin: acl
    ---
    apiVersion: configuration.konghq.com/v1
    config:
      allow:
      - nginx-sample-2
      - app
      hide_groups_header: false
    kind: KongPlugin
    metadata:
      name: nginx-sample-2
      namespace: default
    plugin: acl
    ---
    apiVersion: configuration.konghq.com/v1
    config:
      fault_tolerant: true
      hide_client_headers: false
      limit_by: consumer
      policy: local
      second: 2000
    kind: KongPlugin
    metadata:
      name: nginx-sample-1-rate
      namespace: default
    plugin: rate-limiting
    ---
    apiVersion: v1
    data:
      kongCredType: YmFzaWMtYXV0aA==
      password: NmVmNzI4ZGUtYmE2OC00ZTU5LWFjYjktNmU1MDJjMjhhZTBi
      username: Nzc0Zjg0NDYtNjQyNy00M2Y5LTk5NjItY2U3YWI4MDk3ZmU0
    kind: Secret
    metadata:
      name: app-cred
      namespace: default
    ---
    apiVersion: v1
    data:
      group: YXBw
      kongCredType: YWNs
    kind: Secret
    metadata:
      name: app-acl
      namespace: default
    ---
    apiVersion: configuration.konghq.com/v1
    config:
      fault_tolerant: true
      hide_client_headers: false
      limit_by: consumer
      policy: local
      second: 2000
    kind: KongPlugin
    metadata:
      name: nginx-sample-2-rate
      namespace: default
    plugin: rate-limiting
    ---
    apiVersion: configuration.konghq.com/v1
    credentials:
    - app-acl
    - app-cred
    kind: KongConsumer
    metadata:
      annotations:
        kubernetes.io/ingress.class: kong
        konghq.com/plugins: nginx-sample-1-rate,nginx-sample-2-rate # This is necessary for nginx-sample-1 Service to break Kong's config
      name: app
      namespace: default
    username: 774f8446-6427-43f9-9962-ce7ab8097fe4
    ```

3. Observe KIC crash due to stack overflow

Kong Ingress Controller version

No response

Kubernetes version

2.10

Anything else?

No response