Basic performance tests in a docker container #846

Open
rokka-n opened this Issue Jan 3, 2017 · 3 comments

Projects

None yet

2 participants

@rokka-n
rokka-n commented Jan 3, 2017

Environment

  • OS: El Capitan
  • EMQ: emqttd 2.0.5 (in docker container)

Description

I used simple script (https://github.com/fabaff/mqtt-randompub) to test basic performance of emqtt compare to other brokers.

Container for emqtt from here: github.com:steffenix/emqttd2-docker.git

mosquitto and mosca, running both in docker containers yield around 10k/s, emqtt - around 1k

config

[MQTT]
broker = localhost
port = 1883
qos = 0

[Topic]
topic = mqtt-test
subtopic1 = test1
subtopic2 = test2

[Payload]
load = temp: 100 humidity: 20 timestamp: 2016-12-20T06:10:16.232332 level-sound: 32

Run test:
python mqtt_randompub/mqtt_randompub.py --config config -n 100000000 -i 0.000000001

Results:
mosquitto_sub -h localhost -p 1883 -t "#" | pv -l -i5 -r >/dev/null

Log:

10:26 $ docker run -it -p 1883:1883 -p 8083:8083 -p 18083:18083 emqttd2
emqttd 2.0.5 is started successfully!
emq_auth_clientid
Start apps: [emq_auth_clientid]
Plugin emq_auth_clientid loaded successfully.
emq_auth_http
Start apps: [emq_auth_http]
Plugin emq_auth_http loaded successfully.
emq_auth_ldap
Start apps: [emq_auth_ldap]
Plugin emq_auth_ldap loaded successfully.
emq_auth_mongo
load plugin error: {emq_auth_mongo,
                    {bad_return,
                     {{emq_auth_mongo_app,start,[normal,[]]},
                      {'EXIT',
                       {{badmatch,
                         {error,
                          {shutdown,
                           {failed_to_start_child,emq_auth_mongo,
                            {shutdown,
                             {failed_to_start_child,worker_sup,
                              {shutdown,
                               {failed_to_start_child,
                                {worker,1},
                                econnrefused}}}}}}}},
                        [{emq_auth_mongo_app,start,2,
                          [{file,"src/emq_auth_mongo_app.erl"},{line,35}]},
                         {application_master,start_it_old,4,
                          [{file,"application_master.erl"},{line,273}]}]}}}}}
emq_auth_mysql
load plugin error: {emq_auth_mysql,
                    {bad_return,
                     {{emq_auth_mysql_app,start,[normal,[]]},
                      {'EXIT',
                       {{badmatch,
                         {error,
                          {shutdown,
                           {failed_to_start_child,emq_auth_mysql,
                            {shutdown,
                             {failed_to_start_child,worker_sup,
                              {shutdown,
                               {failed_to_start_child,
                                {worker,1},
                                {{badmatch,{error,econnrefused}},
                                 [{mysql,init,1,
                                   [{file,"src/mysql.erl"},{line,484}]},
                                  {gen_server,init_it,6,
                                   [{file,"gen_server.erl"},{line,328}]},
                                  {proc_lib,init_p_do_apply,3,
                                   [{file,"proc_lib.erl"},
                                    {line,240}]}]}}}}}}}}},
                        [{emq_auth_mysql_app,start,2,
                          [{file,"src/emq_auth_mysql_app.erl"},{line,33}]},
                         {application_master,start_it_old,4,
                          [{file,"application_master.erl"},{line,273}]}]}}}}}
emq_auth_pgsql
load plugin error: {emq_auth_pgsql,
                    {bad_return,
                     {{emq_auth_pgsql_app,start,[normal,[]]},
                      {'EXIT',
                       {{badmatch,
                         {error,
                          {shutdown,
                           {failed_to_start_child,emq_auth_pgsql,
                            {shutdown,
                             {failed_to_start_child,worker_sup,
                              {shutdown,
                               {failed_to_start_child,
                                {worker,1},
                                econnrefused}}}}}}}},
                        [{emq_auth_pgsql_app,start,2,
                          [{file,"src/emq_auth_pgsql_app.erl"},{line,33}]},
                         {application_master,start_it_old,4,
                          [{file,"application_master.erl"},{line,273}]}]}}}}}
emq_auth_redis
load plugin error: {emq_auth_redis,
                    {bad_return,
                     {{emq_auth_redis_app,start,[normal,[]]},
                      {'EXIT',
                       {{badmatch,
                         {error,
                          {shutdown,
                           {failed_to_start_child,emq_auth_redis,
                            {shutdown,
                             {failed_to_start_child,worker_sup,
                              {shutdown,
                               {failed_to_start_child,
                                {worker,1},
                                {connection_error,
                                 {connection_error,econnrefused}}}}}}}}}},
                        [{emq_auth_redis_app,start,2,
                          [{file,"src/emq_auth_redis_app.erl"},{line,26}]},
                         {application_master,start_it_old,4,
                          [{file,"application_master.erl"},{line,273}]}]}}}}}
emq_auth_username
Start apps: [emq_auth_username]
Plugin emq_auth_username loaded successfully.
emq_coap
Start apps: [emq_coap]
Plugin emq_coap loaded successfully.
emq_dashboard
load plugin error: already_started
emq_mod_presence
load plugin error: already_started
emq_mod_retainer
load plugin error: already_started
emq_mod_rewrite
Start apps: [emq_mod_rewrite]
Plugin emq_mod_rewrite loaded successfully.
emq_mod_subscription
load plugin error: already_started
emq_plugin_template
Start apps: [emq_plugin_template]
Plugin emq_plugin_template loaded successfully.
emq_recon
load plugin error: already_started
emq_reloader
Start apps: [emq_reloader]
Plugin emq_reloader loaded successfully.
emq_sn
Start apps: [emq_sn]
Plugin emq_sn loaded successfully.
emq_stomp
Start apps: [emq_stomp]
Plugin emq_stomp loaded successfully.

Of course it could be that docker itself slows it down, so I'll try to run different brokers on linux server to compare. However, running tests in docker is pretty convenient way to compare baseline performance, so I'm curious what you think about this results.

@rokka-n
rokka-n commented Jan 3, 2017

Follow up: when running it as processes on Mac OS X - the results are comparable (14k/s emqtt with 18k/s mosquitto)

I guess it's just the docker specifics.

@rokka-n
rokka-n commented Jan 3, 2017 edited
@emqplus
Member
emqplus commented Jan 4, 2017

Hi @rokka-n, Have you tried our official docker image: http://www.emqtt.io/downloads/latest/docker ?
I will run the mqtt-randompub benchmark later.

@emqplus emqplus added the Benchmark label Jan 4, 2017
@emqplus emqplus self-assigned this Jan 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment