When EMQX updates the load balancer, it resets the Connection Idle Timeout to the default value.

[qRainBirdp]

What happened?

I deployed an EMQX cluster in AWS EKS according to the official website documentation, and configured the listener type as load balancer. Here is the emqx.yaml file:

apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:
  name: emqx
  namespace: emqx
spec:
  image: emqx:5.1
  config:
    data: |
      sys_topics {
        sys_event_messages {client_subscribed = true, client_unsubscribed = true}
      }
      mqtt {
        shared_subscription = true
        shared_subscription_strategy = hash_clientid
      }
      authorization {
        sources = [
          {
            enable = true
            path = "/opt/emqx/etc/ht_acl.conf"
            type = file
          }
        ]
      }
  coreTemplate:
    spec:
      extraVolumes: 
        - name: emqx-acl-config
          configMap:
            name: emqx-acl-config
      extraVolumeMounts:
        - name: emqx-acl-config
          mountPath: /opt/emqx/etc/ht_acl.conf
          subPath: acl.conf
      volumeClaimTemplates:
        storageClassName: gp2
        resources:
          requests:
            storage: 5Gi
        accessModes:
          - ReadWriteOnce
      replicas: 3
  listenersServiceTemplate:
    spec:
      type: LoadBalancer
  dashboardServiceTemplate:
    spec:
      type: LoadBalancer

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: emqx-acl-config
  namespace: emqx
data:
  acl.conf: |
    {allow, {username, "mkb"}, all, ["$SYS/#", "#"]}.

    {allow, {username, {re, "^dashboard$"}}, subscribe, ["$SYS/#"]}.

    {allow, {ipaddr, "127.0.0.1"}, all, ["$SYS/#", "#"]}.

    {allow, all, subscribe, ["$SYS/#", {eq, "#"}]}.

    {allow, all}.

Then I modified the Connection Idle Timeout of the load balancer for the listener on AWS from the default 60s to 150s. However, when I checked again after two days, it had been changed back to the default value of 60s. Based on AWS and service logs, I suspect it was due to an update in EMQX, which reverted the Connection Idle Timeout value for the load balancer.

What did you expect to happen?

I hope that the value of Connection Idle Timeout will not be modified to the default value. Alternatively, EMQX could provide a configuration option for this parameter. However, based on the information I've found, it seems that AWS does not support YAML configuration for this parameter.

How can we reproduce it (as minimally and precisely as possible)?

No response

Anything else we need to know?

No response

EMQX version

emqx-operator-controller:2.2.17
emqx:5.1

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

Log files

arn:aws:iam::408919879510:role/eks-cluster ModifyLoadBalancerAttributes 2024-03-31T06:18:44.000Z

[Rory-Z]

Hi @qRainBirdp EMQX operator will not change AWS ELB configure, and also EMQX operator didn't have permission to change it.
But EMQX operator will update EMQX service in Kubernetes, I guess it is reason, you can set some annotation for AWS ELB configure, check this document: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/service/annotations/, you can set annotations in .spec.listenersServiceTemplate.metadata.annotations and .spec. dashboardServiceTemplate.metadata.annotations

[qRainBirdp]

There's no configuration regarding the Idle timeout in these annotations. I suspect that some behavior might have triggered the initialization of the ELB, which incidentally reset this parameter to its default value. We haven't encountered similar issues since then, so it's difficult to pinpoint the exact cause.