Difference between socket_timeout and keep_alive_interval

[aescobarsamcla]

Hello,
I'm using this library for my laravel project, it's fantastic. Anyway I also manage a mosquitto broker, in the mosquitto.log I see different behaviours between my mqtt client and other mqtt clients also connected to the broker.

The others PING every 1 minute, if I want to do the same I have to set keep_alive_interval as 60, that's ok.
But when I restart the broker (systemctl restart mosquitto) the others reconnect after a second as much, in the other hand my laravel mqtt client reconnects far away the 60 seconds.

I think that the others libraries (one of them use mqtt.js) differentiates between socket_tiemout (1s) and keep_alive_interval(60s) to get this behaviour.

This is my subscriber setting (I use the environment for some parameters, for example 8883 port instead 1883):

        'subscriber_connection' => [

            // The host and port to which the client shall connect.
            'host' => env('MQTT_HOST'),
            'port' => env('MQTT_PORT', 1883),

            // The MQTT protocol version used for the connection.
            'protocol' => MqttClient::MQTT_3_1,

            // A specific client id to be used for the connection. If omitted,
            // a random client id will be generated for each new connection.
            'client_id' => "LARAVEL".env('MQTT_CLIENT_ID').rand(),

            // Whether a clean session shall be used and requested by the client.
            // A clean session will let the broker forget about subscriptions and
            // queued messages when the client disconnects. Also, if available,
            // data of a previous session will be deleted when connecting.
            'use_clean_session' => env('MQTT_CLEAN_SESSION', false), // set to false because auto-reconnect is enabled

            // Whether logging shall be enabled. The default logger will be used
            // with the log level as configured.
            'enable_logging' => env('MQTT_ENABLE_LOGGING', false),

            // Which logging channel to use for logs produced by the MQTT client.
            // If left empty, the default log channel or stack is being used.
            'log_channel' => env('MQTT_LOG_CHANNEL', null),

            // Defines which repository implementation shall be used. Currently,
            // only a MemoryRepository is supported.
            'repository' => MemoryRepository::class,

            // Additional settings used for the connection to the broker.
            // All of these settings are entirely optional and have sane defaults.
            'connection_settings' => [

                // The TLS settings used for the connection. Must match the specified port.
                'tls' => [
                    'enabled' => env('MQTT_TLS_ENABLED', false),
                    'allow_self_signed_certificate' => env('MQTT_TLS_ALLOW_SELF_SIGNED_CERT', false),
                    'verify_peer' => env('MQTT_TLS_VERIFY_PEER', true),
                    'verify_peer_name' => env('MQTT_TLS_VERIFY_PEER_NAME', true),
                    'ca_file' => env('MQTT_TLS_CA_FILE'),
                    'ca_path' => env('MQTT_TLS_CA_PATH'),
                    'client_certificate_file' => env('MQTT_TLS_CLIENT_CERT_FILE'),
                    'client_certificate_key_file' => env('MQTT_TLS_CLIENT_CERT_KEY_FILE'),
                    'client_certificate_key_passphrase' => env('MQTT_TLS_CLIENT_CERT_KEY_PASSPHRASE'),
                ],

                // Credentials used for authentication and authorization.
                'auth' => [
                    'username' => env('MQTT_AUTH_USERNAME'),
                    'password' => env('MQTT_AUTH_PASSWORD'),
                ],

                // Can be used to declare a last will during connection. The last will
                // is published by the broker when the client disconnects abnormally
                // (e.g. in case of a disconnect).
                'last_will' => [
                    'topic' => env('MQTT_LAST_WILL_TOPIC'),
                    'message' => env('MQTT_LAST_WILL_MESSAGE'),
                    'quality_of_service' => env('MQTT_LAST_WILL_QUALITY_OF_SERVICE', 0),
                    'retain' => env('MQTT_LAST_WILL_RETAIN', false),
                ],

                // The timeouts (in seconds) used for the connection. Some of these settings
                // are only relevant when using the event loop of the MQTT client.
                'connect_timeout' => env('MQTT_CONNECT_TIMEOUT', 60),
                'socket_timeout' => env('MQTT_SOCKET_TIMEOUT', 5), // Reduced from 5 to 3 to 2 for faster disconnect detection
                'resend_timeout' => env('MQTT_RESEND_TIMEOUT', 5),

                // The interval (in seconds) in which the client will send a ping to the broker,
                // if no other message has been sent.
                'keep_alive_interval' => env('MQTT_KEEP_ALIVE_INTERVAL', 10), // Reverted to 10 seconds

                // Additional settings for the optional auto-reconnect. The delay between reconnect attempts is in seconds.
                'auto_reconnect' => [
                    'enabled' => env('MQTT_AUTO_RECONNECT_ENABLED', true), // changed from false
                    'max_reconnect_attempts' => env('MQTT_AUTO_RECONNECT_MAX_RECONNECT_ATTEMPTS', PHP_INT_MAX), // only relevant if 'enabled' is true, changed from 3 to infinite (-1)
                    'delay_between_reconnect_attempts' => env('MQTT_AUTO_RECONNECT_DELAY_BETWEEN_RECONNECT_ATTEMPTS', 0), // Changed back to 0 for immediate reconnection
                ],

            ],

        ],

Please, could you drive me how could I achieve the same behaviour as the other mqtt clients?

Thank you in advance

[Namoshek]

I think this could be related to php-mqtt/client#207 and the way we detect the state of the socket. Currently, this happens only through observation when a write or read operation fails because the socket is in a broken state.

However, according to the MQTT specification, it is expected that the broker does not cleanly disconnect its clients upon shutdown. The DISCONNECT package is purely to be sent from the client to the server, not the other way round. So there is no option for the server other than to simply close the socket on its side. And that seems tricky to discover on the client side, at least in PHP.

Can you give a bit more details about the way you use the client? And what other clients (that work as expected) are you referring to?

[aescobarsamcla]

Thanks for reply @Namoshek . I'll try clarify my request as best as possible, you could forgot the original request:

• Context:

  • I have a mosquitto broker installed on a cloud server. Me and other customers have other servers with mqtt clients that are connected as a subscribers to this broker. In my case I have a webserver using Laravel.
  • The customers are using non-php mqtt libraries as a mqtt clients.
  • I am using your library, that works very good.
  • They are pinging to the broker every 60 seconds and when broker restarts by systemctl restart mosquitto they auto-reconnect almost instantly.
  • In the other hand if I ping every keep_alive_interval seconds (for example 60), when broker restarts my subscriber auto-reconnects beyond the keep_alive_interval, i.e. it not auto-reconnects almost instantly, it last a lot of time.

• Question: How could I achieve the same behaviour than the other clients, pinging every 60s and auto-reconnecting almost instantly?

• Tips:

  • I've tried changing the values by several ways in connect_timeout, socket_timeout, resend_timeout and keep_alive_interval but no way to achieve ping every 60s and auto-reconnect after broker restart almost instantly.
  • One of the them is using the https://github.com/mqttjs library.

Sorry if I'm not clear. If you need more info please guide me of what you need.

[Namoshek]

Thanks for the clarification, it is basically what I understood from your initial request. The configuration of the client has the following meaning:

• connect_timeout is the number of seconds a connect attempt may take until the client aborts (see here)
• socket_timeout is the number of seconds used as timeout for the stream itself while it is connected (see here)
• resend_timeout is the number of seconds the client waits for an acknowledgement before resending messages to compensate messages lost in transmission (see here)
• keep_alive_interval is the number of seconds used to calculate until when a response is expected when a ping is sent

The keep_alive_interval is used at the application level, i.e. to detect disconnects at the level of the MQTT protocol. The connect_timeout and socket_timeout are used directly on the socket/stream that is used for the connection to the MQTT broker. How exactly those timeouts work is specific to the PHP implementation and I'm not really qualified to exactly describe how they work.

What I can tell is that a socket is a bidirectional connection between the client and the broker. If the broker closes the socket without sending an info that it has closed the socket, the client will receive this information with a delay based on implementation details of the operating system and other factors. It should be fairly real-time (since the loop() method actively polls the connection), but I cannot guarantee that for every setup.

Can you give a bit more details about your setup? Interesting details are:

• How is your application deployed (on a VM, in Docker, in Kubernetes, ...)?
• What operating system are you using (Windows, Linux, ...)?
• Are you connecting directly to the broker or via some kind of proxy?

[aescobarsamcla]

Thank you!

• About If the broker closes the socket without sending an info that it has closed the socket, ....: systemctl restart mosquitto is "clean" way of disconnection/connection.
• About How is your application deployed (on a VM, in Docker, in Kubernetes, ...)?: It's not running over any third layer.
• About What operating system are you using (Windows, Linux, ...)? -> Both Laravel and mosquitto broker run over Rocky Linux.
• About Are you connecting directly to the broker or via some kind of proxy?: Both laravel and mosquitto broker are installed in a cloud server, in a separate cloud servers but in the same hoster, so the communication is very direct.

[Namoshek]

About If the broker closes the socket without sending an info that it has closed the socket, ....: systemctl restart mosquitto is "clean" way of disconnection/connection.

No, there is no "clean" way for an MQTT broker to terminate the connection or tell the client beforehand that it will be disconnected. The DISCONNECT package is only sent by clients to the server, not the other way round. So the broker literally closes the socket and that's it. (Of course it can process all queued messages before, if that is the "clean" you refer to...)

However, I'm pretty sure the client is missing an EOF check in the read function. I've added such a check in php-mqtt/client#215 which is already released as v2.3.1, so I'm pretty confident that simply updating your dependencies should fix this issue.

[aescobarsamcla]

Thank you very much @Namoshek, now socket_timeout takes sense. I've tested the new release and works like a charm.
Congratulations!