RabbitMQ management API call failed: HTTP 500: Internal Server Error

[suvari3V]

I am using Celery Flower with RabbitMQ as message broker. However I am getting nasty error

ERROR - RabbitMQ management API call failed: HTTP 500: Internal Server Error

every time I try to access RabbitMQ queues on RabbitMQ management interface (http://localhost:15672/#/queues) or Celery Broker (http://localhost:5555/broker).
Here is a detailed log info:

=ERROR REPORT==== 15-Jan-2017::23:12:19 ===
webmachine error: path="/api/queues/%2F"
{error,{error,function_clause,
[{mochijson2,json_encode_string_unicode,
[{error,[],<<166,65,92,180,0,0,0,0,0>>},
{encoder,null,false},
"""],
[{file,"src/mochijson2.erl"},{line,274}]},
{mochijson2,'-json_encode_proplist/2-fun-0-',3,
[{file,"src/mochijson2.erl"},{line,198}]},
{lists,foldl,3,[{file,"lists.erl"},{line,1248}]},
{mochijson2,json_encode_proplist,2,
[{file,"src/mochijson2.erl"},{line,202}]},
{mochijson2,'-json_encode_proplist/2-fun-0-',3,
[{file,"src/mochijson2.erl"},{line,199}]},
{lists,foldl,3,[{file,"lists.erl"},{line,1248}]},
{mochijson2,json_encode_proplist,2,
[{file,"src/mochijson2.erl"},{line,202}]},
{mochijson2,'-json_encode_array/2-fun-0-',3,
[{file,"src/mochijson2.erl"},{line,189}]}]}}

RabbitMQ web management gives me:

Got response code 500 with body {"error":"Internal Server Error","reason":"{error,{error,function_clause,\n [{mochijson2,json_encode_string_unicode,\n [{error,[],<<166,65,92,180,0,0,0,0,0>>},\n {encoder,null,false},\n "\""],\n [{file,"src/mochijson2.erl"},{line,274}]},\n {mochijson2,'-json_encode_proplist/2-fun-0-',3,\n [{file,"src/mochijson2.erl"},{line,198}]},\n {lists,foldl,3,[{file,"lists.erl"},{line,1248}]},\n {mochijson2,json_encode_proplist,2,\n [{file,"src/mochijson2.erl"},{line,202}]},\n {mochijson2,'-json_encode_proplist/2-fun-0-',3,\n [{file,"src/mochijson2.erl"},{line,199}]},\n {lists,foldl,3,[{file,"lists.erl"},{line,1248}]},\n {mochijson2,json_encode_proplist,2,\n [{file,"src/mochijson2.erl"},{line,202}]},\n {mochijson2,'-json_encode_array/2-fun-0-',3,\n [{file,"src/mochijson2.erl"},{line,189}]}]}}\n"}

I am using:

• Ubuntu 14.04
• Python 2.7.6
• Flower 0.9.1
• Celery 4.0.2
• RabbitMQ 3.6.6

RabbitMQ management plugin is enabled (rabbitmq-plugins enable rabbitmq_management).

Here is the output for rabbitmqctl status:

[{pid,21568},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","3.6.6"},
{rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.6.6"},
{webmachine,"webmachine","1.10.3"},
{mochiweb,"MochiMedia Web Server","2.13.1"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.6.6"},
{rabbit,"RabbitMQ","3.6.6"},
{os_mon,"CPO CXC 138 46","2.2.14"},
{ssl,"Erlang/OTP SSL application","5.3.2"},
{public_key,"Public key infrastructure","0.21"},
{crypto,"CRYPTO version 2","3.2"},
{amqp_client,"RabbitMQ AMQP Client","3.6.6"},
{rabbit_common,[],"3.6.6"},
{inets,"INETS CXC 138 49","5.9.7"},
{mnesia,"MNESIA CXC 138 12","4.11"},
{compiler,"ERTS CXC 138 10","4.9.4"},
{xmerl,"XML parser","1.3.5"},
{syntax_tools,"Syntax tools","1.6.12"},
{asn1,"The Erlang ASN1 compiler version 2.0.4","2.0.4"},
{ranch,"Socket acceptor pool for TCP protocols.","1.2.1"},
{sasl,"SASL CXC 138 11","2.3.4"},
{stdlib,"ERTS CXC 138 10","1.19.4"},
{kernel,"ERTS CXC 138 10","2.16.4"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R16B03 (erts-5.10.4) [source] [64-bit] [smp:4:4] [async-threads:64] [kernel-poll:true]\n"},
{memory,
[{total,55140456},
{connection_readers,0},
{connection_writers,0},
{connection_channels,0},
{connection_other,2704},
{queue_procs,60672},
{queue_slave_procs,0},
{plugins,393144},
{other_proc,18574936},
{mnesia,77704},
{mgmt_db,773600},
{msg_index,51856},
{other_ets,1492544},
{binary,129408},
{code,27102195},
{atom,992409},
{other_system,5489284}]},
{alarms,[]},
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,6691391078},
{disk_free_limit,50000000},
{disk_free,59083329536},
{file_descriptors,
[{total_limit,3996},
{total_used,3},
{sockets_limit,3594},
{sockets_used,0}]},
{processes,[{limit,1048576},{used,239}]},
{run_queue,0},
{uptime,15966},
{kernel,{net_ticktime,60}}]

Here is how I run Celery:

Start 3 celery workers:

• celery worker -A celeryapp --config celeryconfig --workdir /proj -l DEBUG --soft-time-limit=3500 --time-limit=3600 --autoscale=2,1 -S /proj/states.db -s /proj/celerybeat-schedule.db -f /proj/logs/celery_worker_1_%n%I.log --pidfile /tmp/celery_w1.pid -n worker1@%d
• celery worker -A celeryapp --config celeryconfig --workdir /proj -l DEBUG --soft-time-limit=3500 --time-limit=3600 --autoscale=2,1 -S /proj/states.db -s /proj/celerybeat-schedule.db -f /proj/logs/celery_worker_2_%n%I.log --pidfile /tmp/celery_w2.pid -n worker2@%d
• celery worker -A celeryapp --config celeryconfig --workdir /proj -l DEBUG --soft-time-limit=3500 --time-limit=3600 --autoscale=2,1 -S /proj/states.db -s /proj/celerybeat-schedule.db -f /proj/logs/celery_worker_3_%n%I.log --pidfile /tmp/celery_w3.pid -n worker3@%d

Start celery beat:

• celery beat -A celeryapp --config celeryconfig --workdir /proj -l DEBUG -s /proj/celerybeat-schedule.db -f /proj/logs/celery_%n%I.log --pidfile /tmp/celery.pid

Start celery flower:

• celery flower -A celeryapp --workdir=/proj/ --broker=amqp://guest:guest@localhost:5672// --broker-api=http://guest:guest@localhost:15672/api/ --db=/proj/flower.db --logging=debug --basic-auth=guest:guest

[michaelklishin]

Please post questions to rabbitmq-users or Stack Overflow. RabbitMQ uses GitHub issues for specific actionable items engineers can work on, not questions. Thank you.

[michaelklishin]

This repository is not for management plugin and we only accept HTTP API request examples (what Celery does is opaque). Please take this to the Celery mailing list.

[michaelklishin]

Also, we have no information on RabbitMQ version used.

[shyamlalRishabh]

com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.

[michaelklishin]

@shyamlalRishabh the message says what your next step should be. Relevant doc guides: Access Control, Connection Lifecycle Events.