mqttNet performance on larger messages

[jesseemus]

Receiving larger messages on mqttNet client is slow

When subscribing to a topic with a wildcard (+/topic) on broker (in our case moquitto),
that has around 900 persistent messages with an average size of 180KB,
the performance of mqttNet client on receiving messages is horribly slow.

I started the project that has only the managed client in it subscribing to +/topic, and incrementing a counter on received messages.
After starting the project I also connected separate mqtt client software (mqtt.fx) to same broker and subscribed to same topic.
For mqtt.fx it took approximately 12 seconds to receive all 900 messages from the broker.

meantime mqttNet client that was running at the same time had only received 40 messages.

it looks like the ApplicationMessageReceivedHandler was firing approximately 3-4 times a second in this case.

I also tested with topics that have higher volume of messages but message size was smaller, there the performance of mqttNet and mqtt.fx was comparable.

Which project is your bug related to?

• Client
  OR
• ManagedClient

[ramonsmits]

Do you know how mqtt.fx does its processing? Does it use a single connection? Without knowing that tool you don't know if you are comparing apples to apples so do provide more details.

[jesseemus]

I have no in depth knowledge on the mqtt.fx tool, but what comes to connections both mqtt.fx and mqttNet are creating only single connection. other tools in addition to mqtt.fx are also providing great speeds, including NodeRed mqtt input node that I believe is using MQTT.js.

I was not aware that you could use multiple connections for single topic subscribe to improve performance, could you clarify what do you mean with it?

[jesseemus]

I did some further testing and added additional 200 retained messages to our broker that have average size of 4MB, now when subscribing with MQTTnet client I receive approximately 10 messages in same time where mqtt.fx receives 1000 messages.

[jesseemus]

I'm also getting comparable speeds to mqtt.fx with M2Mqtt library.

[jesseemus]

Running MQTTnet and M2MQTT side by side demonstrates this issue quite well, with large messages:

with smaller messages:

numbers presented are the amount of received messages for each client.

[JanEggers]

can you provide a repro?

[chkr1011]

Please try again with the latest 4.0 preview from the nuget feed. Version 4 has lots of performance and memory improvements. You can also increase the internal buffer sizes in the options now. So if you usually send messages with 4 MB in size you can set the buffer size to 5 MB etc. If the problem still exists please reopen the ticket.

[kingsley91-cpu]

The problem is still there @chkr1011 even in the library MQTT net 4.3.1.873, it has some performance issue while fetching payloads compared to M2M
QTT, how we could handle the scenario?

[kingsley91-cpu]

@chkr1011 it worked as per your suggestion of increasing the internal buffer size

[chkr1011]

@kingsley91-cpu Do you have some measurements to share? I am interested in how much the performance has increased.