Version 0.0.8 Release Notes- Beta - not recommended (yet) for critical applications
- Features
- Performance
- Comparison with AsyncMqttClient
- Installation
- Issues / Support
- IMPORTANT NOTE FOR PLATFORMIO USERS
- Using TLS
- STM32 targets
- Challenges of embedded MQTT
- Problems / errors in the AsyncMqttClient API
- List of 16 fatal bugs in AsyncMqttClient (with evidence)
- Example Sketches
- Full* MQTT v3.1.1 Qos0/1/2 compliance, session management and recovery
- Payload size limited only by available heap (~20kB on ESP8266, ~120kb on ESP32)
- TLS support (ESP8266 only)
- Compilable diagnostics and testing topics:
- dump
- reboot
- disconnect
- break selected QoS1/2 handshakes for session recovery testing
- Utility functions to view binary payloads and handle as:
- C-style string (
char*) std::string- Arduino
String intfrom any the above string types
- C-style string (
- Error-handling for:
- QoS failures*
- Subscribe failure
- "Killer" packets ( > free available heap) both inbound and outbound
NB *No device can fully comply with MQTT 3.1.1 QoS unless it has unlimited permanent storage to hold failed QoS messages across reboots. A future version will use LittleFS to perform this function. For more in depth explanation of the reasons, read Challenges of MQTT QoS on embedded systems which compares how the inevitable shortfall is handled by three different libraries:
PubSubClient- not at all: QoS0 only and 128-byte payload limitAsyncMqttClient- poorly / not at all: crashes on huge inward packets, limits outbound to 1-2kB and loses QoS1/2 recovery messages, fails to attempt session recovery and has no method of informing the user of either failure. Put simply, it doesn't workPangolin(this library) - as far as is technically possible, while informing the user of inevitable inherent failures.
Pangolin tries to preserve API compatibility with AsyncMqttClient but that library contains illogical and incorrect features, which means that you will have to make minor changes to two or three lines of code in your existing apps before using them with Pangolin.
Those have been "fixed" in this release of Pangolin only to make it quick and easy for users to move their apps to Pangolin with minimum effort and without compile errors, but they will change to "sensible" equivalents in the next release.
For an in-depth analysis (with evidence) of those problems, read API issues
Take a look at the first - very simple - example which exists in two forms: A "P_" version which uses Pangolin and an "A_" version which uses AsyncMqttClient. A side-by-side comparison of the two shows the very minimal changes required to get you up and running. (Also a single glance at the comparison of the two outputs gives a good overview of the major differences between Pangolin and AsyncMqttClient)
The remainder of the Examples can be compiled for either Pangolin or AsyncMqttClient to allow you to compare the two libraries in the few cases where AsynClientMqtt doesn't fail. See the Example Sketches on how to run them and get the best out of the code.
In the table below, CHG = those few API calls that have had to be changed just to "make sense" for the first release. These serve as a handy guide to the minimum you need to change in your current code.
DEP calls are "deprecated" which means they will change at the next release (this automatically includes those in the CHG category)
REM shows calls which will be completely removed at the next release for one of these reasons:
- They have no practical function
- They have confusing and/or misleading parameters
- Both of the above
NEW shows functions only available in Pangolin which provide functionality required for correct operation but which AsyncMqttClient does not provide.
Missing from the table below are the following, which live in the PANGO:: namespace and are designed to make a very simple task of correct payload handling.
void dumphex(uint8_t* mem, size_t len,uint8_t W);
char* payloadToCstring(uint8_t* data,size_t len);
int payloadToInt(uint8_t* data,size_t len);
std::string payloadToStdstring(uint8_t* data,size_t len);
String payloadToString(uint8_t* data,size_t len);TODO Update diagram
Full details, explanation and rationale for the changes here
This is the screenshot of the outputs of the two versions of the first example sketch, the first running AsyncMqttClient and the second running Pangolin
As you can see there are (at least) four major benefits of using Pangolin over AsyncMqttClient right from the start:
Pangolin handles any sized payload up to 1/2 the available heap both inbound and outbound. AsyncMqttClient cannot properly handle even medium-sized packets outbound and simply makes no attempt to handle large inbound packets, expecting the user to do it. Read more
AsyncMqttClient can only correctly handle a maximum outbound payload size* of the TCP MSS size*. "What's that?" I hear you ask. Exactly. It's an implementation-dependent value that AsyncMqttClient provides no way for you to discover. It changes according to the ArduinoIDE build settings.
If you choose the "Lower Memory" option it is 536, "Higher Bandwith" gives you 1460. Despite the fact that ArduinoIDE allocates 2x this size (e.g. 1072 vs 2920) it seems that AsyncMqttClient has "issues" dealing with anything over the buffer size.
Pangolin automatically fragments outbound packets and reassembles inbound packets of any size up to about 1/2 the free heap. User code simply gets a copy of the full packet - irrespective of its size - without any fuss and requiring zero code on his/her part.
It also provides a new function getMaxPayloadSize and a new onError callback that allows the users to safely and informatively handle any "killer packet" scenario without crashing.
Pangolin is simply faster, full stop. The exact amount depends to a great extent on the size / frequency / combination of inbound/outbound messages, but in certain instances can show huge improvements. See the examples for more detail.
AsyncMqttClient seems to assume all payloads are strings. Not only is not the case, but it provides no functions to assist the user in examining, parsing, converting or "unpacking" the payload.
Pangolin provides:
void dumphex(uint8_t* mem, size_t len,uint8_t W);
char* payloadToCstring(uint8_t* data,size_t len);
int payloadToInt(uint8_t* data,size_t len);
std::string payloadToStdstring(uint8_t* data,size_t len);
String payloadToString(uint8_t* data,size_t len);Pangolin is "alpha" so of course it is likely to contain bugs. What can be said though is that whatever yet-to-be-found bugs it may still contain, even at this first release it does not contain the bug that causes AsyncMqttClient to regularly (but at random) enter a disconnect/reconnect cycle - which is so common it will be referred to from now as "DCX/CNX". Neither does it contain the bug which prevents AsyncMqttClient from totally failing to implement QoS1 and 2, nor any of the other 16 fatal bugs in AsyncMqttClient listed here
At the time of writing, a "soak test" of the Session Recovery 2 example sketch shows that PangolinMQTT has been running for 426023 seconds (118.34 hours, 4.93 days) without a single reconnection or message lost, while sending and receiving a QoS2 message every second.
Pangolin depends upon the excellent libraries ESPAsyncTCP (for ESP8266 targets) or AsyncTCP for ESP32 targets.
You will need to install one or both of those before using Pangolin
See also Using TLS for additional dependencies
See also STM32 targets for additional dependencies and information on using Pangolin on STM32-NUCLEO boards
Pangolin is an Arduino library, and is 100% compatible with the ArduinoIDE and build system. PlatformIO, sadly, is not. If PlatformIO has problems with code that compiles and runs correctly under ArduinoIDE, then it is a PlatformIO problem, not an issue with this - or any other - valid Arduino library.
For over 3 years I have been notifying the PlatformIO team of errors in their build setting related to the use of non-standard and non ArduinoIDE-compatible architecture #defines which break many vaild Arduino libraries. For over 3 years they have failed to fix it,so I refuse to provide any support on any of my 100% Arduino compatible libraries for users of PlatformIO until they do.
For this reason, I will not accept any issues relating to build problems with PlatformIO, nor any pull requests or other suggestions which involve any changes that render the library less than 100% ArduinoIDE compatible. If you don't like this, you have two options:
- Petition, moan, complain to PlatformIO dev team to make it 100% Arduino-compatible
- Use another library: my position will not change until PlatformIO
get off their lazy assesfix their problems
Obviously I will check the issues here from time to time, but if you want a rapid resonse, I can usually be found moderating on of these FB groups, including a new one especially set up for Pangolin users:
(C) 2020 Phil Bowles