Support for ESP32 and W5500 based Secure Ethernet for HTTPS or MQTTS?

[winnergeorge]

I tried with W5500 connected with ESP32 and PubSubClient library for Secured MQTT

EthernetClient LANClient;
SSLClient secureClientEth(&LANClient);
PubSubClient mqttEthClient(secureClientEth);

EthernetLarge library is used. The code does get compiled however messages dont get published to the Broker. Anyone tried this for Ethernet and got success?

[RobertByrnes]

HI @winnergeorge, I don't have a W5500 here so I won't be able to help you directly: try these steps -

Based on your description, it seems that you're trying to connect to a MQTT broker using an Ethernet connection on an ESP32 with the W5500 Ethernet module and secured using the SSLClient library, but your messages are not being published. Here are a few steps you can take to troubleshoot this issue:

Check Your Connection: First, ensure that your device is correctly connected to the internet through the W5500 Ethernet module. You can test this by making a simple HTTP request or pinging a server. If you're unable to connect to the internet, you should double-check your wiring, power supply, and internet connection.

Verify SSL/TLS Connection: If your device is able to connect to the internet, the next step is to ensure that your SSL/TLS connection is correctly set up. SSLClient library needs a few parameters to make a connection, such as the certificate of the MQTT server. If these parameters aren't correctly set, you won't be able to establish a connection.

MQTT Connection: Once you've verified your SSL/TLS connection, you should check if you're able to establish a connection to your MQTT broker. You can verify this by checking the return code of the mqttEthClient.connect() function. If it returns false, then you're unable to connect to the MQTT broker.

Check QoS and Retain Flag: Make sure you're setting the correct Quality of Service (QoS) level and retain flag when publishing messages. If you're setting a QoS level higher than the broker supports or the retain flag incorrectly, your messages might not be published.

Debugging: The PubSubClient library provides a state() function that returns the current state of the client. You can use this to print the state of your client and potentially figure out what's going wrong. The SSLClient also provides various debug options that you can enable to see more detailed error messages.

Lastly, if all these steps fail, you might want to try using a different MQTT library to see if the issue lies within the PubSubClient library. There are many other libraries available, like the Adafruit_MQTT library or the AsyncMqttClient library, both of which also support secure MQTT connections.

If this works then please post the solution (and thanks GPT4!) and if it doesn't then you will have to supply a link to code to assist in making a determination. Good luck...

[winnergeorge]

Hi @RobertByrnes . I have two different Mosquitto brokers on separate AWS clients. One is plain MQTT and the other is MQTT with SSL.
Observation: 1) Messages are published to both Secure and Non Secure MQTT brokers from ESP32 WiFi and ESP32 with SIMCOM A7672 4G module.
2) Messages are published to Non Secure MQTT broker from ESP32 with W5500 Ethernet module, but not to the secure broker.
mqttEthClient.connect() function always returns FALSE in this case

Arduino Code:

#include <SPI.h>
#include <EthernetLarge.h> // Ethernet library v2 is required for proper operation
#include <PubSubClient.h>
#include <ArduinoJson.h>
#include "SSLClient.h"

// MQTT Broker details
const char* mqttServer = "MQTT_SSL_BROKER";
const int mqttPort = 8883;
const char* mqttTopic = "Test/Topic";
const char* mqttClientId = "Client_1234";

#define ETHERNET_RST_PIN 25 // Ethernet uses default SPI: MOSI -> 23, MISO -> 19, SCLK -> 18, CS -> 5
byte mac[] = {
0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED
};
const char* ca_cert =
"-----BEGIN CERTIFICATE-----\n"
"MYROOTCERTIFICATE\n"
"-----END CERTIFICATE-----\n";

EthernetClient LANClient;
SSLClient secureClientEth(&LANClient);
PubSubClient mqttEthClient(secureClientEth);

StaticJsonDocument<2048> JSONencoder;
char jsonBuffer[2048];

void setup() {

Serial.begin(115200); //Default Serial Monitor
secureClientEth.setCACert(ca_cert);
mqttEthClient.setBufferSize(2048);
mqttEthClient.setServer(mqttServer, mqttPort);

checkEthernetConnection();

}

void loop() {

JSONencoder["Data"] = "Hello from W5500";
serializeJson(JSONencoder, jsonBuffer);
Serial.println(jsonBuffer);

if (mqttEthClient.connect(mqttClientId)) {

// Publish data to MQTT broker
if (mqttEthClient.publish(mqttTopic, jsonBuffer)) 
{
  Serial.println("Data published to MQTT broker via Ethernet");
} else 

{
  Serial.println("Failed to publish data to MQTT broker via Ethernet");
}

}
mqttEthClient.loop();
delay(2000);
}

void checkEthernetConnection() {
Ethernet.init(5); // Chip Select pin
Serial.println("Testing Ethernet DHCP...plz wait for sometime");
if (Ethernet.begin(mac)) { // Dynamic IP setup
Serial.println("DHCP OK!");
}
else
{
return;
}
delay(5000); // give the Ethernet shield some time to initialize
Serial.print("Local IP : ");
Serial.println(Ethernet.localIP());
Serial.print("Subnet Mask : ");
Serial.println(Ethernet.subnetMask());
Serial.print("Gateway IP : ");
Serial.println(Ethernet.gatewayIP());
Serial.print("DNS Server : ");
Serial.println(Ethernet.dnsServerIP());
Serial.println("Ethernet Successfully Initialized");
}

[RobertByrnes]

Your network setup seems fine. This is evidenced by the ESP32 with W5500 being able to connect to a Non-Secure MQTT broker.
The issue is specific to establishing a secured MQTT connection via the W5500 Ethernet module.
You've already done the correct step by setting the CA cert via secureClientEth.setCACert(ca_cert);.
Here are some potential solutions and checks:

Certificate Mismatch ???
The CA certificate you've provided (ca_cert) might not match the one that the MQTT broker is expecting or using. Ensure that the CA certificate is correct.
If you have a domain for the MQTT server, ensure it matches the Common Name (CN) or the Subject Alt Name in the broker's certificate.

SSL Client Configuration:
The SSLClient library has different modes of operation. You might want to check if you need to set a client certificate or private key, especially if the broker expects mutual TLS.
Additionally, make sure the SSLClient library is correctly configured. You can enable its debugging to get more detailed error messages to help identify the problem. This can be done using secureClientEth.setDebugStream(&Serial);.
Memory Constraints:

SSL/TLS requires a fair bit of memory, especially when handling certificates. The ESP32 should have enough memory, but it's always worth ensuring that you aren't running into any memory allocation issues.
Increasing the buffer might help if large certificates are being used.
Network Firewall or Security Groups:

If using AWS, ensure that your Security Groups or Network ACLs allow for incoming connections on port 8883 (MQTT SSL port) from your device's IP.

MQTT Credentials:
Some MQTT brokers require username and password for authentication. Ensure that you provide them if required. For example:

if (mqttEthClient.connect(mqttClientId, mqttUser, mqttPassword)) {
    //... rest of the code
}

Persistent MQTT Session:
If your client was previously connected with a persistent session, the broker might be refusing a new connection. Try changing your mqttClientId to a different value to ensure it's not a session-related issue.

Broker Logs:
If possible, check the logs of your MQTT broker maybe...