500 response from HTTP API on Erlang 19.0

[hoalequang]

Hi guys,

When I install rabbitmq-server version 3.6.3 (latest), I have an issue with management plugin in queue tab.

Got response code 500 with body {"error":"JSON encode error: {bad_term,#{error_logger => true,kill => true,size => 0}}","reason":"While encoding: \n[{total_count,2},\n {item_count,2},\n {filtered_count,2},\n {page,1},\n {page_size,100},\n {page_count,1},\n {items,\n [[{memory,14304},\n {message_stats,\n [{deliver,0},\n {deliver_details,[{rate,0.0}]},\n {deliver_no_ack,6},\n {deliver_no_ack_details,[{rate,0.0}]},\n {get,0},\n {get_details,[{rate,0.0}]},\n {get_no_ack,0},\n {get_no_ack_details,[{rate,0.0}]},\n {publish,6},\n {publish_details,[{rate,0.0}]},\n {publish_in,0},\n {publish_in_details,[{rate,0.0}]},\n {publish_out,0},\n {publish_out_details,[{rate,0.0}]},\n {ack,0},\n {ack_details,[{rate,0.0}]},\n {deliver_get,6},\n {deliver_get_details,[{rate,0.0}]},\n {confirm,0},\n {confirm_details,[{rate,0.0}]},\n

But if I install rabbitmq-server version 3.6.2, it have not this bug.
Can you please help me resolve this bugs?
Thanks a lot!

[michaelklishin]

This belongs to rabbitmq/rabbitmq-management. Please provide the output of GET /api/queues and messages from the JS console. E.g.

curl -u guest:guest -X GET http://127.0.0.1:15672/api/queues/ | python -m json.tool

[michaelklishin]

Also, full stack trace from the logs, please.

[michaelklishin]

I cannot reproduce this, including when using pagination, filtering, regex filtering, with a few thousand queues. So we need a more detailed report about how this can be reproduced.

Please try clearing your browser's cache, we see issues such as rabbitmq/rabbitmq-management#242 from time to time after upgrades.

[kae]

Same issue here.

=ERROR REPORT==== 7-Jul-2016::16:25:03 ===
JSON encode error: {bad_term,#{error_logger => true,kill => true,size => 0}}
While encoding:
[[{memory,14128},
  {reductions,3628},
  {reductions_details,[{rate,0.0}]},
  {messages,0},
  {messages_details,[{rate,0.0}]},
  {messages_ready,0},
  {messages_ready_details,[{rate,0.0}]},
  {messages_unacknowledged,0},
  {messages_unacknowledged_details,[{rate,0.0}]},
  {idle_since,<<"2016-07-07 13:22:09">>},
  {consumer_utilisation,''},
  {policy,<<"ha-all">>},
  {exclusive_consumer_tag,''},
  {consumers,1},
  {slave_nodes,[]},
  {synchronised_slave_nodes,[]},
  {recoverable_slaves,[]},
  {state,running},
  {reductions,3628},
  {garbage_collection,[{max_heap_size,#{error_logger => true,kill => true,size => 0}},
                       {min_bin_vheap_size,46422},
                       {min_heap_size,233},
                       {fullsweep_after,65535},
                       {minor_gcs,1}]},
  {messages_ram,0},
  {messages_ready_ram,0},
  {messages_unacknowledged_ram,0},
  {messages_persistent,0},
  {message_bytes,0},
  {message_bytes_ready,0},
  {message_bytes_unacknowledged,0},
  {message_bytes_ram,0},
  {message_bytes_persistent,0},
  {head_message_timestamp,''},
  {disk_reads,0},
  {disk_writes,0},
  {backing_queue_status,{struct,[{mode,default},
                                 {q1,0},
                                 {q2,0},
                                 {delta,[delta,undefined,0,undefined]},
                                 {q3,0},
                                 {q4,0},
                                 {len,0},
                                 {target_ram_count,infinity},
                                 {next_seq_id,0},
                                 {avg_ingress_rate,0.0},
                                 {avg_egress_rate,0.0},
                                 {avg_ack_ingress_rate,0.0},
                                 {avg_ack_egress_rate,0.0},
                                 {mirror_seen,0},
                                 {mirror_senders,0}]}},
  {node,'rabbit@it-kae-rmq'},
  {arguments,{struct,[{<<"x-dead-letter-exchange"...

=ERROR REPORT==== 7-Jul-2016::16:25:03 ===
webmachine error: path="/api/queues/"
"Internal Server Error"

Logs after

curl -u guest:guest -X GET http://127.0.0.1:15672/api/queues/ | python -m json.tool

Output of command

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  113k  100  113k    0     0  1658k      0 --:--:-- --:--:-- --:--:-- 1667k
{
    "error": "JSON encode error: {bad_term,#{error_logger => true,kill => true,size => 0}}",
    "reason": "While encoding: \n[[{memory,14128},\n  {reductions,3628},\n  {reductions_details,[{rate,0.0}]},\n  {messages,0},\n  {messages_details,[{rate,0.0}]},\n  {messages_ready,0},\n  {messages_ready_details,[{rate,0.0}]},\n  {messages_unacknowledged,0},\n  {messages_unacknowledged_details,[{rate,0.0}]},\n  {idle_since,<<\"2016-07-07 13:22:09\">>},\n  {consumer_utilisation,''},\n  {policy,<<\"ha-all\">>},\n  {exclusive_consumer_tag,''},\n  {consumers,1},\n  {slave_nodes,[]},\n  {synchronised_slave_nodes,[]},\n  {recoverable_slaves,[]},\n  {state,running},\n  {reductions,3628},\n  {garbage_collection,[{max_heap_size,#{error_logger => true,kill => true,size => 0}},\n                       {min_bin_vheap_size,46422},\n                       {min_heap_size,233},\n                       {fullsweep_after,65535},\n                       {minor_gcs,1}]},\n  {messages_ram,0},\n  {messages_ready_ram,0},\n  {messages_unacknowledged_ram,0},\n  {messages_persistent,0},\n  {message_bytes,0},\n  

....
<cut for brevity>

[michaelklishin]

Moved to rabbitmq/rabbitmq-management#244.

[dannykopping]

From error log: error.txt

Can confirm that I encountered the same issue.

I pinned Erlang on my Ubuntu Precise box:

/etc/apt/preferences.d/30erlang:

Package: erlang*
Pin: version 1:18.3-1
Pin-Priority: 999

Purged erlang*, reinstalled rabbitmq-server and it's now working

[chrome050]

Same issue when calling /api/connections

{
   "error" : "JSON encode error: {bad_term,#{error_logger => true,kill => true,size => 0}}",
   "reason" : "While encoding: \n[[{connected_at,1469090807198},\n  {client_properties,\n      {struct,\n          [{<<\"product\">>,<<\"RabbitMQ\">>},\n           {<<\"version\">>,<<\"3.6.3.0\">>},\n           {<<\"platform\">>,<<\".NET\">>},\n           {<<\"copyright\">>,\n            <<\"Copyright (c) 2007-2016 Pivotal Software, Inc.\">>},\n           {<<\"information\">>,\n            <<\"Licensed under the MPL.  See http://www.rabbitmq.com/\">>},\n           {<<\"MachineName\">>,<<\"CLEANMASCHINE\">>},\n           {<<\"capabilities\">>,\n            {struct,\n                [{<<\"publisher_confirms\">>,true},\n                 {<<\"exchange_exchange_bindings\">>,true},\n                 {<<\"basic.nack\">>,true},\n                 {<<\"consumer_cancel_notify\">>,true},\n                 {<<\"connection.blocked\">>,true},\n                 {<<\"authentication_failure_close\">>,true}]}},\n           {<<\"connection_name\">>,undefined}]}},\n  {channel_max,0},\n  {frame_max,131072},\n  {timeout,60},\n  {vhost,<<\"devel\">>},\n  {user,<<\"ats-test\">>},\n  {protocol,<<\"AMQP 0-9-1\">>},\n  {ssl_hash,sha},\n  {ssl_cipher,aes_256_cbc},\n  {ssl_key_exchange,rsa},\n  {ssl_protocol,tlsv1},\n  {auth_mechanism,<<\"PLAIN\">>},\n  {peer_cert_validity,''},\n  {peer_cert_issuer,''},\n  {peer_cert_subject,''},\n  {ssl,true},\n  {peer_host,<<\"CLEANIP\">>},\n  {host,<<\"CLEANIP\">>},\n  {peer_port,32993},\n  {port,5671},\n  {name,<<\"CLEANIP:32993 -> CLEANIP:5671\">>},\n  {node,testbench@localhost},\n  {type,network},\n  {garbage_collection,\n      [{max_heap_size,#{error_logger => true,kill => true,size => 0}},\n       {min_bin_vheap_size,46422},\n       {min_heap_size,233},\n       {fullsweep_after,65535},\n       {minor_gcs,3}]},\n  {reductions,84694},\n  {channels,0},\n  {state,running},\n  {send_pend,0},\n  {send_cnt,9},\n  {recv_cnt,26},\n  {recv_oct_details,[{rate,7.4}]},\n  {recv_oct,1779},\n  {send_oct_details,[{rate,0.0}]},\n  {send_oct,2732},\n  {reductions_details,[{rate,103.4}]},\n  {reductions,84694}]]"
}

Calling from Management Interface im getting:

Got response code 500 with body {"error":"JSON encode error: {bad_term,#{error_logger => true,kill => true,size => 0}}","reason":"While encoding: \n[{total_count,1},\n {item_count,1},\n {filtered_count,1},\n {page,1},\n {page_size,100},\n {page_count,1},\n {items,\n [[{connected_at,1469090807198},\n {client_properties,\n {struct,\n [{<<\"product\">>,<<\"RabbitMQ\">>},\n {<<\"version\">>,<<\"3.6.3.0\">>},\n {<<\"platform\">>,<<\".NET\">>},\n {<<\"copyright\">>,\n <<\"Copyright (c) 2007-2016 Pivotal Software, Inc.\">>},\n {<<\"information\">>,\n <<\"Licensed under the MPL. See http://www.rabbitmq.com/\">>},\n {<<\"MachineName\">>,<<\"CLEANMASCHINE\">>},\n {<<\"capabilities\">>,\n {struct,\n [{<<\"publisher_confirms\">>,true},\n {<<\"exchange_exchange_bindings\">>,true},\n {<<\"basic.nack\">>,true},\n {<<\"consumer_cancel_notify\">>,true},\n {<<\"connection.blocked\">>,true},\n {<<\"authentication_failure_close\">>,true}]}},\n {<<\"connection_name\">>,undefined}]}},\n {channel_max,0},\n {frame_max,131072},\n {timeout,60},\n {vhost,<<\"devel\">>},\n {user,<<\"CLEANUSER\">>},\n {protocol,<<\"AMQP 0-9-1\">>},\n {ssl_hash,sha},\n {ssl_cipher,aes_256_cbc},\n {ssl_key_exchange,rsa},\n {ssl_protocol,tlsv1},\n {auth_mechanism,<<\"PLAIN\">>},\n {peer_cert_validity,''},\n {peer_cert_issuer,''},\n {peer_cert_subject,''},\n {ssl,true},\n {peer_host,<<\"CLEANIP\">>},\n {host,<<\"CLEANIP\">>},\n {peer_port,32993},\n {port,5671},\n {name,<<\"CLEANIP:32993 -> CLEANIP:5671\">>},\n {node,testbench@localhost},\n {type,network},\n {garbage_collection,\n [{max_heap_size,#{error_logger => true,kill => true,size => 0}},\n {min_bin_vheap_size,46422},\n {min_heap_size,233},\n {fullsweep_after,65535},\n {minor_gcs,1}]},\n {reductions,58807},\n {channels,0},\n {state,running},\n {send_pend,0},\n {send_cnt,5},\n {recv_cnt,8},\n {recv_oct_details,[{rate,0.0}]},\n {recv_oct,1113},\n {send_oct_details,[{rate,0.0}]},\n {send_oct,2436},\n {reductions_details,[{rate,94.0}]},\n {reductions,58807}]]}]"}

[michaelklishin]

This issue was moved to rabbitmq/rabbitmq-management#244.