Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

socketio-stderr Connection refused #342

Closed
richardsonlima opened this issue Jan 8, 2015 · 12 comments
Closed

socketio-stderr Connection refused #342

richardsonlima opened this issue Jan 8, 2015 · 12 comments
Labels

Comments

@richardsonlima
Copy link

Hello,

I did configuring BareMetal backend, but I can't see any machine, i found in error logs entries below.

==> var/log/socketio-stderr---supervisor-uvF0OC.log <==
2015-01-08 17:48:48 INFO Dummy-60 sockio - __init__: Initialized MistNamespace for user . Socket 59201288386. Session cea829fab9ad449ab42ac87188779bee
2015-01-08 17:48:48 INFO Dummy-61 sockio - disconnect: Disconnecting MistNamespace for user . Socket 134700865905. Session 587404bec4ce4cb588bf9c74e0374084
2015-01-08 17:48:49 INFO Dummy-60 sockio - on_ready: Ready to go!

Traceback (most recent call last):
  File "/home/richardson/mist.io/eggs/gevent-1.0.1-py2.7-linux-x86_64.egg/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/richardson/mist.io/src/mist/io/sockio.py", line 270, in update_subscriber
    amqp_subscribe_user(user, queue=queue, callback=namespace.process_update)
  File "/home/richardson/mist.io/src/mist/io/helpers.py", line 209, in amqp_subscribe_user
    amqp_subscribe(_amqp_user_exchange(user), queue, callback)
  File "/home/richardson/mist.io/src/mist/io/helpers.py", line 169, in amqp_subscribe
    connection = Connection()
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused
<Greenlet at 0x7fad532e84b0: update_subscriber(<mist.io.sockio.MistNamespace object at 0x7fad5330)> failed with error

2015-01-08 17:48:51 INFO Dummy-62 connectionpool - _new_conn: Starting new HTTPS connection (1): mist.io
Traceback (most recent call last):
  File "/home/richardson/mist.io/eggs/gevent-1.0.1-py2.7-linux-x86_64.egg/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/richardson/mist.io/src/mist/io/sockio.py", line 298, in list_backends_from_socket
    cached = task.smart_delay(user.email, backend_id)
  File "/home/richardson/mist.io/src/mist/io/tasks.py", line 345, in smart_delay
    self.delay(*args, **kwargs)
  File "/home/richardson/mist.io/eggs/celery-3.1.17-py2.7.egg/celery/app/task.py", line 453, in delay
    return self.apply_async(args, kwargs)
  File "/home/richardson/mist.io/eggs/celery-3.1.17-py2.7.egg/celery/app/task.py", line 555, in apply_async
    **dict(self._get_exec_options(), **options)
  File "/home/richardson/mist.io/eggs/celery-3.1.17-py2.7.egg/celery/app/base.py", line 353, in send_task
    reply_to=reply_to or self.oid, **options
  File "/home/richardson/mist.io/eggs/celery-3.1.17-py2.7.egg/celery/app/amqp.py", line 305, in publish_task
    **kwargs
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/messaging.py", line 168, in publish
    routing_key, mandatory, immediate, exchange, declare)
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/connection.py", line 457, in _ensured
    interval_max)
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/connection.py", line 369, in ensure_connection
    interval_start, interval_step, interval_max, callback)
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/utils/__init__.py", line 243, in retry_over_time
    return fun(*args, **kwargs)
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/connection.py", line 237, in connect
    return self.connection
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/connection.py", line 741, in connection
    self._connection = self._establish_connection()
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/connection.py", line 696, in _establish_connection
    conn = self.transport.establish_connection()
  File "/home/richardson/mist.io/eggs/kombu-3.0.24-py2.7.egg/kombu/transport/pyamqp.py", line 112, in establish_connection
    conn = self.Connection(**opts)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/home/richardson/mist.io/eggs/amqp-1.4.6-py2.7.egg/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused
<Greenlet at 0x7fad53301190: list_backends_from_socket(<mist.io.sockio.MistNamespace object at 0x7fad5330)> failed with error

Thanks and I am looking forward to get a support.

@dimrozakis
Copy link
Member

Hey @richardsonlima

Judging by the error output (socket error in amqp lib) I'd say that rabbitmq is most likely not running and socketio-gevent server fails because it can't connect to it.

Could you please paste the output of ./bin/supervisorctl status, after you cd to your mist.io's installation top level directory? Can you also paste the output of tail -n 30 var/log/rabbitmq*.log?

@richardsonlima
Copy link
Author

Hello,

Please see below

  • Service status:
    [998:997 0:649] 02:16:21 Sex Jan 09 [richardson@0p5:/dev/pts/17 +1] ~/mist.io
    (1:649)$ ./bin/supervisorctl status
    celery FATAL Exited too quickly (process log may have details)
    haproxy RUNNING pid 42074, uptime 1 day, 3:09:40
    memcache FATAL Exited too quickly (process log may have details)
    rabbitmq FATAL Exited too quickly (process log may have details)
    socketio RUNNING pid 42075, uptime 1 day, 3:09:40
    uwsgi RUNNING pid 57455, uptime 21:43:45
  • Log output:
    [998:997 0:651] 02:16:58 Sex Jan 09 [richardson@0p5:/dev/pts/17 +1] ~/mist.io
    (1:651)$ sudo tail -n 30 var/log/rabbitmq*.log
    ==> var/log/rabbitmq-stderr---supervisor-hRHNTw.log <==

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k

==> var/log/rabbitmq-stdout---supervisor-0jbqk5.log <==
{error_logger,{{2015,1,8},{18,6,45}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}
{error_logger,{{2015,1,8},{18,6,52}},"Protocol: ~tp: register/listen error: tpn",["inet_tcp",econnrefused]}
{error_logger,{{2015,1,8},{18,6,52}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.21.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,322}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.93>,<0.18.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,789}],[]]}
{error_logger,{{2015,1,8},{18,6,52}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[rabbitmqprelaunch62383,shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2015,1,8},{18,6,52}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2015,1,8},{18,6,52}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]}
{error_logger,{{2015,1,8},{18,6,52}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}
{error_logger,{{2015,1,8},{18,6,54}},"Protocol: ~tp: register/listen error: tpn",["inet_tcp",econnrefused]}
{error_logger,{{2015,1,8},{18,6,54}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.21.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,322}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.93>,<0.18.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,789}],[]]}
{error_logger,{{2015,1,8},{18,6,54}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[rabbitmqprelaunch62410,shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2015,1,8},{18,6,54}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2015,1,8},{18,6,54}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]}
{error_logger,{{2015,1,8},{18,6,54}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}
{error_logger,{{2015,1,8},{18,6,56}},"Protocol: ~tp: register/listen error: tpn",["inet_tcp",econnrefused]}
{error_logger,{{2015,1,8},{18,6,56}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.21.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,322}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.93>,<0.18.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,789}],[]]}
{error_logger,{{2015,1,8},{18,6,56}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[rabbitmqprelaunch62439,shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2015,1,8},{18,6,56}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2015,1,8},{18,6,56}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]}
{error_logger,{{2015,1,8},{18,6,56}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}
{error_logger,{{2015,1,8},{18,7,1}},"Protocol: ~tp: register/listen error: tpn",["inet_tcp",econnrefused]}
{error_logger,{{2015,1,8},{18,7,1}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.21.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,322}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.93>,<0.18.0>]},{dictionary,[{longnames,false}]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,789}],[]]}
{error_logger,{{2015,1,8},{18,7,1}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{name,net_kernel},{mfargs,{net_kernel,start_link,[[rabbitmqprelaunch62495,shortnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
{error_logger,{{2015,1,8},{18,7,1}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{name,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
{error_logger,{{2015,1,8},{18,7,1}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,133}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]}
{error_logger,{{2015,1,8},{18,7,1}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}

@dimrozakis
Copy link
Member

That doesn't look good :) . Can you please do the following?

  1. Paste the output of ps -fe | grep epmd
  2. Run ./bin/supervisorctl shutdown to kill supervisord and all its children processes.
  3. Wait a few seconds, then run ./bin/supervisord to start supervisord again.
  4. Wait a few seconds, then run ./bin/supervisorctl status and paste the output.
  5. Paste the output of tail -n 30 var/log/*.log. This is gonna be kind of huge, so perhaps it'd be better to put in a gist.

@dimrozakis
Copy link
Member

Hey, are you running this as root? If so, this could be the cause of the problem because some of mist's components won't run as root without modification. Please try to do the following:

  1. Run ./bin/supervisorctl shutdown to kill supervisord.
  2. Supposing your user is named 'richardson', run chown -R richardson . so that richardson user can access the mist files.
  3. Start supervisord with ./bin/supervisord.
  4. Check if everything is running correctly by running ./bin/supervisorctl status. Please paste the output of this.

@richardsonlima
Copy link
Author

Hey,

I followed all steps informed, but i receive Connection refused by amqp.

Please see below ... Thanks \o

  • Step 1

(1:817)$ ps -fe | grep epmd |grep -v grep

  • Step 2
    (1:818)$ ./bin/supervisorctl shutdown
    Shut down
  • Step 3
    $ sudo chown -R richardson:richardson mist.io
  • Step 4
    (1:834)$ ./bin/supervisorctl status
    celery RUNNING pid 36962, uptime 0:01:03
    haproxy RUNNING pid 36959, uptime 0:01:03
    memcache RUNNING pid 36964, uptime 0:01:03
    rabbitmq FATAL Exited too quickly (process log may have details)
    socketio RUNNING pid 36960, uptime 0:01:03
    uwsgi RUNNING pid 36961, uptime 0:01:03
  • Step 5
    ==> var/log/celery-stderr---supervisor-xJXYkS.log <==
    2015-01-09 17:07:47,088 ERROR MainThread consumer - _error_handler: consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
    Trying again in 32.00 seconds...

==> var/log/socketio-stderr---supervisor-xBW5xW.log <==
2015-01-09 17:07:50 INFO Dummy-9 sockio - disconnect: Disconnecting MistNamespace for user . Socket 837509587528. Session 2d1af6c31bf84ac585452d48277c4df4

==> var/log/uwsgi-stdout---supervisor-asCgqA.log <==
2015-01-09 17:07:59,651 INFO MainThread methods - add_backend_v_2: Adding new backend in provider 'bare_metal' with Api-Version: 2
2015-01-09 17:07:59,666 INFO MainThread shell - autoconfigure: autoconfiguring Shell for machine 3NixNnLQcd6bRuWy8rKRfZus856A:200238105150
2015-01-09 17:07:59,668 INFO MainThread shell - autoconfigure: ssh -i key1 richardson@192.168.200.3:22
2015-01-09 17:07:59,845 INFO Thread-1 log - log: Connected (version 2.0, client OpenSSH_5.9p1)
2015-01-09 17:08:00,476 INFO Thread-1 log - log: Authentication (publickey) successful!
2015-01-09 17:08:00,480 INFO MainThread shell - command: running command: 'uptime'
2015-01-09 17:08:01,160 INFO MainThread shell - command: running command: 'which sudo'
2015-01-09 17:08:01,465 INFO MainThread shell - command: running command: 'uptime'
2015-01-09 17:08:01,832 INFO MainThread shell - disconnect: Closing ssh connection to 200.238.105.150
2015-01-09 17:08:01,980 INFO MainThread shell - disconnect: Closing ssh connection to 200.238.105.150
2015-01-09 17:08:01,989 INFO MainThread methods - add_backend_v_2: Backend with id '3NixNnLQcd6bRuWy8rKRfZus856A' added succesfully.
[pid: 37070|app: 0|req: 69/229] 127.0.0.1 () {52 vars in 793 bytes} [Fri Jan 9 17:07:59 2015] POST /backends => generated 220 bytes in 2354 msecs (HTTP/1.1 200) 2 headers in 87 bytes (1 switches on core 0)

==> var/log/celery-stderr---supervisor-xJXYkS.log <==
2015-01-09 17:08:19,225 ERROR MainThread consumer - _error_handler: consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 111] Connection refused.
Trying again in 32.00 seconds...

@dimrozakis
Copy link
Member

Could you please try running epmd -daemon and then either restart rabbitmq with ./bin/supervisorctl restart rabbitmq or shutdown and restart supervisord as mentioned above? After that, please run ./bin/supervisorctl status and paste the output. Should there still be some problem with any of the processes, it'd help to paste the relevant log files.

@stephenjamieson
Copy link

I am having a similar issue, it seems epmd will not start when ipv6 is disabled on the server. This is a limitation of the erlang installation on default ubuntu 14.10...

@mgogoulos
Copy link

Hi Stephen,

in the past we have seen epmd processes being hanging, so rabbitmq is having trouble to starting. When this happens to me, I usually stop supervisorctl processes and shutdown supervisor

user@user:~/Desktop/mist.io$ ./bin/supervisorctl stop all ; ./bin/supervisorctl shutdown

I make sure epmd is not running, and if it runs I stop it. I then start it

user@user:~/Desktop/mist.io$ ps aux|grep epmd
user      6615  0.0  0.0   7492   320 ?        S    22:23   0:00 epmd -daemon
user@user:~/Desktop/mist.io$ kill -9 6615
user@user:~/Desktop/mist.io$ epmd -daemon

then I start supervisord

user@user:~/Desktop/mist.io$ ./bin/supervisord

What you describe seems to be ubuntu related, where you able to find a solution?

Cheers

@stephenjamieson
Copy link

Yes, I enabled ipv6 and it was able to be started.

@mgogoulos
Copy link

Nice to read that!

@KeiroD
Copy link

KeiroD commented May 7, 2015

@mgogoulos Looks like it is indeed Ubuntu related.

Doing as you suggested does the trick for me. Ironically... this issue didn't pop up on my initial Vultr VPS... but did on the second VPS. I wonder what's different between the two...

@mgogoulos
Copy link

Version 2 is out, comes with an easier installation process using Docker Compose and many new features, check updated instructions on https://github.com/mistio/mist.io

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants