Skip to content

client v2.0.21 exits mosquitto_thread if MQTT server is down during initial connect #3258

Closed

Description

@acode-x

Mosquitto library version: 2.0.21
Openssl version: 1.1.1z
Platform: Ubuntu x86_64

When I run the following program (with valid server certificate and client certificate/key loaded) while the MQTT server is already stopped, I observe the following behavior:

  1. on_disconnect callback gets invoked with rc = 14
  2. errno is set to EPROTO and mosquitto thread exists; below mosquitto code for reference

    mosquitto/lib/loop.c

    Lines 274 to 279 in c6c6850

    case MOSQ_ERR_ERRNO:
    break;
    }
    if(errno == EPROTO){
    return rc;
    }
  3. If I again bring back MQTT up, it never re-connects back due to thread exit.

However, if I run the same program with MQTT server already started, I am able to connect to server. I also notice the client reconnects back even if I restart the MQTT server.


In short: If the initial connect fails due to transport error (e.g. MQTT server is not running), mosquitto thread exits, and subsequent reconnect never happens.
But, if initial connects succeeds, subsequent reconnects always happen.


Minimal reproducible example:

#include <atomic>
#include <csignal>
#include <cstring>
#include <iostream>
#include <mosquitto.h>
#include <string>
#include <thread>

std::atomic<bool> running{true};

void on_connect(struct mosquitto*, void*, int rc);
void on_publish(struct mosquitto*, void*, int mid);
void handle_signal(int signal);
void on_log(struct mosquitto*, void*, int level, const char* message);
void on_disconnect(struct mosquitto*, void*, int rc);

void on_connect(struct mosquitto*, void*, int rc)
{
  if (rc == 0) {
    std::cout << "Connected successfully to MQTT broker" << std::endl;
  } else {
    std::cout << "Failed to connect to MQTT broker. Error code: " << rc << std::endl;
  }
}

void on_publish(struct mosquitto*, void*, int) {}

void on_log(struct mosquitto*, void*, int level, const char* message)
{
  std::cout << "MQTT [" << level << "]: " << message << std::endl;
}

void on_disconnect(struct mosquitto*, void*, int rc)
{
  if (rc == 0) {
    std::cout << "Disconnected cleanly from MQTT broker" << std::endl;
  } else {
    std::cout << "Unexpected disconnect from MQTT broker. Error code: " << rc << std::endl;
  }
}

void handle_signal(int signal)
{
  std::cout << "Received signal " << signal << ", exiting..." << std::endl;
  running = false;
}

int main(int argc, char* argv[])
{
  signal(SIGINT, handle_signal);
  signal(SIGTERM, handle_signal);

  const char* host = "127.0.0.1";
  int port = 9000;
  const char* client_id = "valid-device";
  const char* topic = "test/topic";
  const char* message = "Hello!";
  int qos = 1;

  std::string cert_path_prefix = "/tmp";

  if (argc > 1) {
    cert_path_prefix = argv[1];
  }

  mosquitto_lib_init();

  struct mosquitto* mosq = mosquitto_new(client_id, true, nullptr);
  if (!mosq) {
    std::cerr << "Error: Failed to create mosquitto instance" << std::endl;
    return 1;
  }

  std::string cafile = cert_path_prefix + "/rootCA.pem";
  std::string certfile = cert_path_prefix + "/clientCert.pem";
  std::string keyfile = cert_path_prefix + "/clientCertKey.pem";

  int result = mosquitto_tls_set(mosq, cafile.c_str(), NULL, certfile.c_str(), keyfile.c_str(), NULL);
  if (result != MOSQ_ERR_SUCCESS) {
    std::cerr << "Error: Failed to set TLS options: " << mosquitto_strerror(result) << std::endl;
    mosquitto_destroy(mosq);
    mosquitto_lib_cleanup();
    return 1;
  }

  mosquitto_connect_callback_set(mosq, on_connect);
  mosquitto_publish_callback_set(mosq, on_publish);
  mosquitto_log_callback_set(mosq, on_log);
  mosquitto_disconnect_callback_set(mosq, on_disconnect);

  result = mosquitto_loop_start(mosq);
  if (result != MOSQ_ERR_SUCCESS) {
    std::cerr << "Error: Failed to start network thread: " << mosquitto_strerror(result) << std::endl;
    mosquitto_disconnect(mosq);
    mosquitto_destroy(mosq);
    mosquitto_lib_cleanup();
    return 1;
  }

  std::cout << "Connecting to MQTT broker at " << host << ":" << port << std::endl;
  result = mosquitto_connect_async(mosq, host, port, 60);
  if (result != MOSQ_ERR_SUCCESS) {
    std::cerr << "Warning: Initial connection to MQTT broker failed: " << mosquitto_strerror(result) << std::endl;
  }

  int message_counter = 0;

  while (running) {
    std::string full_message = message;
    full_message += " [";
    full_message += std::to_string(++message_counter);
    full_message += "]";

    int mid = 0;
    result =
      mosquitto_publish(mosq, &mid, topic, static_cast<int>(full_message.length()), full_message.c_str(), qos, false);
    if (result != MOSQ_ERR_SUCCESS) {
      std::cerr << "Error: Failed to publish message: " << mosquitto_strerror(result) << std::endl;
    }

    for (int i = 0; i < 50 && running; i++) {
      std::this_thread::sleep_for(std::chrono::milliseconds(100));
    }
  }

  std::cout << "Cleaning up..." << std::endl;
  mosquitto_disconnect(mosq);
  mosquitto_loop_stop(mosq, true);
  mosquitto_destroy(mosq);
  mosquitto_lib_cleanup();

  return 0;
}

Logs from above program:

Connecting to MQTT broker at 127.0.0.1:9000
MQTT: Client valid-device sending CONNECT
MQTT: Client valid-device sending PUBLISH (d0, q1, r0, m1, 'test/topic', ... (39 bytes))
Unexpected disconnect from MQTT broker. Error code: 14
MQTT: mosq loop exit EPROTO, 14 (added by me in lib/loop.c while compiling mosquitto)
Error: Failed to publish message: The client is not currently connected.

Appreciate any guidance on same.

Thanks!

Activity

  1. ralight commented on May 1, 2025

    @ralight
    Contributor

    Thanks very much for the detailed explanation and example code, it made it much easier to look into.

    The problem is that the TLS error handling wasn't distinguishing between TLS errors and normal network errors. I've now got a fix for 2.0.22 that makes that change, so the ECONNREFUSED error code will be propagated correctly.

  2. added
    Status: CompletedNothing further to be done with this issue, it can be closed by the requestor or committer.
    and removed
    Status: AvailableNo one has claimed responsibility for resolving this issue.
    on May 1, 2025
  3. added theissue type on May 1, 2025
  4. added this to the 2.0.22 milestone on May 1, 2025
  5. added a commit that references this issue on May 1, 2025
    f781abc
  6. acode-x commented on May 1, 2025

    @acode-x
    Author

    Thanks for looking into this issue.

    I tried adding a log after below line to print the err (in lib/net_mosq.c - net__handle_ssl function)

    err = SSL_get_error(mosq->ssl, ret);

    During the error scenario:
    I get err = 5 i.e. SSL_ERROR_SYSCALL with error queue empty

    SSL_ERROR_SYSCALL
    
    Some non-recoverable, fatal I/O error occurred. 
    The OpenSSL error queue may contain more information on the error. 
    For socket I/O on Unix systems, consult errno for details.
    If this error occurs then no further I/O operations should be performed on the connection and SSL_shutdown() must not be called.
    
    This value can also be returned for other errors, check the error queue for details.
    

    ref: https://docs.openssl.org/1.1.1/man3/SSL_get_error/#return-values

    The fix you've added works, but would like to understand if it ok to skip this error and continue with subsequent retries.

  7. ralight commented on May 1, 2025

    @ralight
    Contributor

    My understanding is that when we get SSL_ERROR_SSL there was a specific SSL related error, like a protocol example for example. SSL_ERROR_SYSCALL is there to indicate that a different syscall error occurred that was outside of the scope of SSL.

    For example, a short read/write syscall is already covered by openssl through the want read/write errors. Other errors like connection lost will happen in response to a syscall but aren't otherwise covered. That's the kind of error we're getting here.

    The fix isn't skipping the error, it's passed back to the normal mosquitto socket handling as if it was nothing to do with openssl - so if there was a lost connection the library will clean up the old connection and try again. From a technical point of view it should be fine.

  8. acode-x commented on May 1, 2025

    @acode-x
    Author

    Thank you so much for the detailed explanation! It really helped me understand the concept better.

  9. locked as resolved and limited conversation to collaborators on Aug 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Component: libmosquittoStatus: CompletedNothing further to be done with this issue, it can be closed by the requestor or committer.

    Type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions