Repository navigation
Description
Mosquitto library version: 2.0.21
Openssl version: 1.1.1z
Platform: Ubuntu x86_64
When I run the following program (with valid server certificate and client certificate/key loaded) while the MQTT server is already stopped, I observe the following behavior:
- on_disconnect callback gets invoked with rc = 14
- errno is set to EPROTO and mosquitto thread exists; below mosquitto code for reference
Lines 274 to 279 in c6c6850
case MOSQ_ERR_ERRNO: break; } if(errno == EPROTO){ return rc; } - If I again bring back MQTT up, it never re-connects back due to thread exit.
However, if I run the same program with MQTT server already started, I am able to connect to server. I also notice the client reconnects back even if I restart the MQTT server.
In short: If the initial connect fails due to transport error (e.g. MQTT server is not running), mosquitto thread exits, and subsequent reconnect never happens.
But, if initial connects succeeds, subsequent reconnects always happen.
Minimal reproducible example:
#include <atomic>
#include <csignal>
#include <cstring>
#include <iostream>
#include <mosquitto.h>
#include <string>
#include <thread>
std::atomic<bool> running{true};
void on_connect(struct mosquitto*, void*, int rc);
void on_publish(struct mosquitto*, void*, int mid);
void handle_signal(int signal);
void on_log(struct mosquitto*, void*, int level, const char* message);
void on_disconnect(struct mosquitto*, void*, int rc);
void on_connect(struct mosquitto*, void*, int rc)
{
if (rc == 0) {
std::cout << "Connected successfully to MQTT broker" << std::endl;
} else {
std::cout << "Failed to connect to MQTT broker. Error code: " << rc << std::endl;
}
}
void on_publish(struct mosquitto*, void*, int) {}
void on_log(struct mosquitto*, void*, int level, const char* message)
{
std::cout << "MQTT [" << level << "]: " << message << std::endl;
}
void on_disconnect(struct mosquitto*, void*, int rc)
{
if (rc == 0) {
std::cout << "Disconnected cleanly from MQTT broker" << std::endl;
} else {
std::cout << "Unexpected disconnect from MQTT broker. Error code: " << rc << std::endl;
}
}
void handle_signal(int signal)
{
std::cout << "Received signal " << signal << ", exiting..." << std::endl;
running = false;
}
int main(int argc, char* argv[])
{
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
const char* host = "127.0.0.1";
int port = 9000;
const char* client_id = "valid-device";
const char* topic = "test/topic";
const char* message = "Hello!";
int qos = 1;
std::string cert_path_prefix = "/tmp";
if (argc > 1) {
cert_path_prefix = argv[1];
}
mosquitto_lib_init();
struct mosquitto* mosq = mosquitto_new(client_id, true, nullptr);
if (!mosq) {
std::cerr << "Error: Failed to create mosquitto instance" << std::endl;
return 1;
}
std::string cafile = cert_path_prefix + "/rootCA.pem";
std::string certfile = cert_path_prefix + "/clientCert.pem";
std::string keyfile = cert_path_prefix + "/clientCertKey.pem";
int result = mosquitto_tls_set(mosq, cafile.c_str(), NULL, certfile.c_str(), keyfile.c_str(), NULL);
if (result != MOSQ_ERR_SUCCESS) {
std::cerr << "Error: Failed to set TLS options: " << mosquitto_strerror(result) << std::endl;
mosquitto_destroy(mosq);
mosquitto_lib_cleanup();
return 1;
}
mosquitto_connect_callback_set(mosq, on_connect);
mosquitto_publish_callback_set(mosq, on_publish);
mosquitto_log_callback_set(mosq, on_log);
mosquitto_disconnect_callback_set(mosq, on_disconnect);
result = mosquitto_loop_start(mosq);
if (result != MOSQ_ERR_SUCCESS) {
std::cerr << "Error: Failed to start network thread: " << mosquitto_strerror(result) << std::endl;
mosquitto_disconnect(mosq);
mosquitto_destroy(mosq);
mosquitto_lib_cleanup();
return 1;
}
std::cout << "Connecting to MQTT broker at " << host << ":" << port << std::endl;
result = mosquitto_connect_async(mosq, host, port, 60);
if (result != MOSQ_ERR_SUCCESS) {
std::cerr << "Warning: Initial connection to MQTT broker failed: " << mosquitto_strerror(result) << std::endl;
}
int message_counter = 0;
while (running) {
std::string full_message = message;
full_message += " [";
full_message += std::to_string(++message_counter);
full_message += "]";
int mid = 0;
result =
mosquitto_publish(mosq, &mid, topic, static_cast<int>(full_message.length()), full_message.c_str(), qos, false);
if (result != MOSQ_ERR_SUCCESS) {
std::cerr << "Error: Failed to publish message: " << mosquitto_strerror(result) << std::endl;
}
for (int i = 0; i < 50 && running; i++) {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
}
std::cout << "Cleaning up..." << std::endl;
mosquitto_disconnect(mosq);
mosquitto_loop_stop(mosq, true);
mosquitto_destroy(mosq);
mosquitto_lib_cleanup();
return 0;
}Logs from above program:
Connecting to MQTT broker at 127.0.0.1:9000
MQTT: Client valid-device sending CONNECT
MQTT: Client valid-device sending PUBLISH (d0, q1, r0, m1, 'test/topic', ... (39 bytes))
Unexpected disconnect from MQTT broker. Error code: 14
MQTT: mosq loop exit EPROTO, 14 (added by me in lib/loop.c while compiling mosquitto)
Error: Failed to publish message: The client is not currently connected.
Appreciate any guidance on same.
Thanks!
Activity
ralight commentedon May 1, 2025
Thanks very much for the detailed explanation and example code, it made it much easier to look into.
The problem is that the TLS error handling wasn't distinguishing between TLS errors and normal network errors. I've now got a fix for 2.0.22 that makes that change, so the ECONNREFUSED error code will be propagated correctly.
Fix client library thread exiting on first connection refused error
acode-x commentedon May 1, 2025
Thanks for looking into this issue.
I tried adding a log after below line to print the err (in lib/net_mosq.c - net__handle_ssl function)
mosquitto/lib/net_mosq.c
Line 951 in c6c6850
During the error scenario:
I get err = 5 i.e. SSL_ERROR_SYSCALL with error queue empty
ref: https://docs.openssl.org/1.1.1/man3/SSL_get_error/#return-values
The fix you've added works, but would like to understand if it ok to skip this error and continue with subsequent retries.
ralight commentedon May 1, 2025
My understanding is that when we get SSL_ERROR_SSL there was a specific SSL related error, like a protocol example for example. SSL_ERROR_SYSCALL is there to indicate that a different syscall error occurred that was outside of the scope of SSL.
For example, a short read/write syscall is already covered by openssl through the want read/write errors. Other errors like
connection lostwill happen in response to a syscall but aren't otherwise covered. That's the kind of error we're getting here.The fix isn't skipping the error, it's passed back to the normal mosquitto socket handling as if it was nothing to do with openssl - so if there was a lost connection the library will clean up the old connection and try again. From a technical point of view it should be fine.
acode-x commentedon May 1, 2025
Thank you so much for the detailed explanation! It really helped me understand the concept better.