Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Websockets slow for large (ish) payloads #1390

Closed
aalibasic opened this issue Aug 23, 2019 · 9 comments

Comments

@aalibasic
Copy link

commented Aug 23, 2019

I am having an issue where sending a 1mb+ payload over mqtt takes 5-10 seconds to complete over localhost.

The same request over tcp takes 32 ms.

I have created a repo that reproduces the issue: https://github.com/Smart-Ag/mqtt_bug_report

To run the repo:

Open the index.html file in your browser
Run mosquitto docker image (docker-compose up -d)
install python dependencies
run python spammer.py
This will send message to the websocket and tcp listener.

I have tested the same clients with the emqx mosquitto broker and it was 10-40 times faster.

@aalibasic aalibasic changed the title Websockets slow for large payloads Websockets slow for large (ish) payloads Aug 23, 2019
@ralight

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2019

The commit I just pushed helps a lot with this. It seems as though I was making libwebsockets unhappy with write sizes and buffer sizes. The commit message marks this as closed, but it doesn't meet the tcp performance.

@aalibasic

This comment has been minimized.

Copy link
Author

commented Aug 29, 2019

@ralight what kind of performance does it get with your changes?

How can I get this version of mosquitto to test?

@ralight

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2019

I was getting ~170ms per message. You can get this version by getting the fixes branch:

git clone https://github.com/eclipse/mosquitto
cd mosquitto
git checkout fixes

Or get https://github.com/eclipse/mosquitto/archive/fixes.zip

Are you ok compiling yourself or would you like help?

@aalibasic

This comment has been minimized.

Copy link
Author

commented Aug 29, 2019

@ralight I would love help getting it working in a docker image.

@ralight

This comment has been minimized.

Copy link
Contributor

commented Aug 29, 2019

In the root of the source from above, run make localdocker. This should give you an (untagged) image of the current copy of the repo.

@aalibasic

This comment has been minimized.

Copy link
Author

commented Sep 3, 2019

@ralight last question, when do you guys plan on making a release cut (tagged docker image) with this code? Or at least an expected wait time?

@ralight

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2019

The aim is for roughly monthly bug fix releases, so soon. I'm trying to just figure a couple of other issues first.

@ralight ralight closed this in 4658dba Sep 12, 2019
ralight added a commit that referenced this issue Sep 18, 2019
@aalibasic

This comment has been minimized.

Copy link
Author

commented Oct 10, 2019

@ralight I just tested mosquitto docker image: eclipse-mosquitto:1.6.6

  1. Over the local network I am seeing reasonable performance (a few 100 ms)
  2. Over a web network (Host the mosquitto broker on ec2) and then host client on local machine, then I get latency of 10,000 ms and it keeps growing as the broker gets backed up.

The emqx broker runs in 200 ms on scenario 1 & 2.

Any ideas on why there is such a difference in performance over local and internet?

@ralight

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2019

That's bonkers. I've tested this out on test.mosquitto.org, which is under load but isn't using the docker image. Using pubsub.py I get:

Unencrypted:
Latency (QoS 0): 30.886173ms
Latency (QoS 1): 31.774521ms
Latency (QoS 2): 162.973881ms
With TLS:
Latency (QoS 0): 30.335188ms
Latency (QoS 1): 30.725718ms
Latency (QoS 2): 163.333654ms

Those results are pretty consistent across runs. I've tried with libwebsockets 2.4.2 and 3.1.0 with the same results. I haven't tried with the docker image yet.

If I understand what you are saying, you get 10,000ms latency right from the first message. Could you please try the python code with your instance to see what you get? You'll need the Paho python client paho-mqtt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.