setBufferSize() does not work.

[sebashb]

I'm trying to send packets that are bigger than the default 256 bytes. If I change the #define MQTT_MAX_PACKET_SIZE 256 to #define MQTT_MAX_PACKET_SIZE 1024 everything works, but if I call setBufferSize(1024) after the connect()method the function appears to have no effect, even though getBufferSize() returns 1024, the publish()method returns false due to the packet being too long.

[d-a-v]

There are bits of an answer in #685 .

[sebashb]

Thanks for you quick reply, but version 2.8 according to CHANGES.TXT "Add setBufferSize() to override MQTT_MAX_PACKET_SIZE" which in my understanding should be used to do what I want to do.

[d-a-v]

Thanks @sebashb for the heads up, I wasn't aware of this change. Sorry to have mislead you.
Does it work if you call the function before connect() ?

Thanks @knolleary too.

[sebashb]

No it does not work also.

[viktak]

For me PSclient.setBufferSize(1024*5); works fine. When issued just after PSclient.connect(). I can verify it by looking at PSclient.getBufferSize() just before whenever I am ready to transmit a message.
And this works for messages of bigger than than the default 256 bytes.

However, when I try to submit a string that is about 3kB it simply doesn't transmit it at all.

Is there a hard limit? If yes, what it it?

I am using an ESP32.

thank you

[viktak]

Some more details:

I am trying to send a list of files as a JSON string over MQTT. A sample JSON string is something like this:

{
  "type" : "FileList",
  "0" : {
    "name" : "20200622.TXT",
    "size" : 225292
  },
  "1" : {
    "name" : "20200623.TXT",
    "size" : 471960
  },
  "2" : {
    "name" : "20200626.TXT",
    "size" : 195513
  },
  "3" : {
    "name" : "20200627.TXT",
    "size" : 1140239
  },
  "4" : {
    "name" : "20200628.TXT",
    "size" : 504215
  },
  "5" : {
    "name" : "20200704.TXT",
    "size" : 204655
  },
  "6" : {
    "name" : "20200705.TXT",
    "size" : 1140131
  },
  "7" : {
    "name" : "20200706.TXT",
    "size" : 1139879
  },
  "8" : {
    "name" : "20200707.TXT",
    "size" : 1140093
  },
  "9" : {
    "name" : "20200708.TXT",
    "size" : 1140524
  },
  "10" : {
    "name" : "20200709.TXT",
    "size" : 1139211
  },
  "11" : {
    "name" : "20200710.TXT",
    "size" : 1138494
  },
  "12" : {
    "name" : "20200711.TXT",
    "size" : 1136883
  },
  "13" : {
    "name" : "20200712.TXT",
    "size" : 1137211
  },
  "14" : {
    "name" : "20200713.TXT",
    "size" : 487807
  },
  "15" : {
    "name" : "20200716.TXT",
    "size" : 264454
  },
  "16" : {
    "name" : "20200717.TXT",
    "size" : 1123603
  },
  "17" : {
    "name" : "20200718.TXT",
    "size" : 1137191
  },
  "18" : {
    "name" : "20200719.TXT",
    "size" : 1137000
  },
  "19" : {
    "name" : "20200720.TXT",
    "size" : 542104
  },
  "20" : {
    "name" : "20200722.TXT",
    "size" : 246385
  },
  "21" : {
    "name" : "20200723.TXT",
    "size" : 475774
  },
  "22" : {
    "name" : "20200724.TXT",
    "size" : 30207
  },
  "23" : {
    "name" : "test.txt",
    "size" : 1048576
  },
  "24" : {
    "name" : "foo.txt",
    "size" : 14
  },
  "25" : {
    "name" : "alma.txt",
    "size" : 67
  },
  "26" : {
    "name" : "20200727.txt",
    "size" : 122650
  },
  "27" : {
    "name" : "20200728.TXT",
    "size" : 534027
  },
  "28" : {
    "name" : "20200729.TXT",
    "size" : 13425
  },
  "29" : {
    "name" : "20200730.TXT",
    "size" : 34596
  },
  "30" : {
    "name" : "19700101.TXT",
    "size" : 62813
  },
  "31" : {
    "name" : "19691231.TXT",
    "size" : 878
  },
  "32" : {
    "name" : "19341201.TXT",
    "size" : 2984
  }
}

This is the biggest list of files I can send without a problem. It has 33 entries. If I add one more similar entry in my code, PSclient.publish fails.

Just before I the PSclient.publish line I added this:

SerialMon.printf("Buffer size:\t%u\r\nJSON size:\t%u\r\nFree heap size:\t%u\r\n", PSclient.getBufferSize(), myJsonString.length(), ESP.getFreeHeap());

The output of it in case the JSON string has 33 item:

Buffer size:    5120
JSON size:      1411
Free heap size: 287100

The output of it in case the JSON string has 34 item:

Buffer size:    5120
JSON size:      1451
Free heap size: 292096

I hope this helps troubleshooting it....

[sebashb]

I have tried both before and after PSclient.connect() and none of them worked. The published messaged got truncated.

[knolleary]

I'm not currently in a position to debug any PubSubClient issues and unlikely to be for a couple weeks at least.

[viktak]

@sebashb Interesting... For me it's either there or not at all. Never truncated...

[sebashb]

Well I have tried multiple things, I'm sure that I have seen both behaviours ( not being published and being truncated), but I can't remember which code produce each one of them.

[ghost]

I add my experience to this issue. I'm using a ESP32-POE board to connect to AWS via MQTT. Some messages I'm going to receive are quite big (over 9 kbytes). So I set the buffer large enough:

#define MQTT_PACKET_SIZE  16384

WiFiClientSecure networkClient;
PubSubClient mqtt(mqttServer, 8883, MQTT_Callback, networkClient);
...

mqtt.setBufferSize(MQTT_PACKET_SIZE);
Serial.println(mqtt.getBufferSize());

It outputs 16384.
When the message is published by the server, all other clients (that don't use this library) receive the payload correctly. Instead my ESP32 board doesn't receive anything (if the message is shorter it works, though).

I also tried to play with MQTT_MAX_TRANSFER_SIZE without any improvement.
Actually, the behavior is the following:

12:43:26.725 -> [MQTT] Connecting...connected
12:43:32.085 -> [MQTT] Subscribed to: $aws/things/mything/shadow/update/#
12:43:32.085 -> [MQTT] Subscribed to: $aws/things/mything/shadow/get/#
12:43:32.085 -> [MQTT] Publish $aws/things/mything/shadow/get:
12:43:32.085 -> [MQTT] Publish result: 1
12:44:17.084 -> [MQTT] Connecting...connected
12:44:31.276 -> [MQTT] Subscribed to: $aws/things/mything/shadow/update/#
12:44:31.276 -> [MQTT] Subscribed to: $aws/things/mything/shadow/get/#
12:44:31.276 -> [MQTT] Publish $aws/things/mything/shadow/get:
12:44:31.276 -> [MQTT] Publish result: 1
12:45:46.290 -> [MQTT] Connecting...failed, rc=-2 try again in 5 seconds
12:45:55.291 -> [MQTT] Connecting...failed, rc=-2 try again in 5 seconds
12:46:00.287 -> [MQTT] Connecting...failed, rc=-2 try again in 5 seconds
...

It's quite repeatable. After publishing the request message it should receive the answer (the "big" payload). Instead, after some dozens of seconds disconnects and reconnects. Doing this a couple of times leads to a failure to connect anymore, until I reset the board.