New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rancher 2.0 fails to start after restarted Machine #9963

Closed
sjkeerthi opened this Issue Sep 27, 2017 · 5 comments

Comments

Projects
None yet
5 participants
@sjkeerthi

sjkeerthi commented Sep 27, 2017

Rancher versions:
rancher/server: Rancher 2.o

I installed rancher 2.0 preview and it works perfectly until I restarted the machine
After I restarted the rancher didn't come up when I check with docker log

On Docker 17.07.0-ce

ERROR

time="2017-09-27T17:17:57Z" level=info msg="Downloading all drivers" service=gms
time="2017-09-27T17:17:57Z" level=info msg="Copying /var/lib/cattle/machine-drivers/cf0a66c94e54aac77fc6883a4677990bf34f47a559c0054afc330becb1868478-docker-machine-driver-packet => /usr/local/bin/docker-machine-driver-packet-tmp" service=gms
time="2017-09-27T17:17:57Z" level=info msg="Done downloading all drivers" service=gms
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening
s6-svscanctl: fatal: unable to control /service: supervisor not listening

@sabiodelhielo

This comment has been minimized.

Show comment
Hide comment
@sabiodelhielo

sabiodelhielo Sep 27, 2017

My rancher/server does not start up after a machine restart. I do not see those errors in my log though just these:

ime="2017-09-27T19:07:40Z" level=info msg="Done downloading all drivers" service=gms
2017-09-27 19:08:00,537 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [23] count [3]
2017-09-27 19:08:00,539 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [24] count [3]
2017-09-27 19:08:00,539 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [34] count [3]
2017-09-27 19:08:00,541 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [35] count [3]
2017-09-27 19:08:05,544 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [23] count [4]
2017-09-27 19:08:05,545 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [24] count [4]
2017-09-27 19:08:05,546 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [34] count [4]
2017-09-27 19:08:05,547 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [35] count [4]
2017-09-27 19:08:10,549 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [23] count [5]

sabiodelhielo commented Sep 27, 2017

My rancher/server does not start up after a machine restart. I do not see those errors in my log though just these:

ime="2017-09-27T19:07:40Z" level=info msg="Done downloading all drivers" service=gms
2017-09-27 19:08:00,537 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [23] count [3]
2017-09-27 19:08:00,539 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [24] count [3]
2017-09-27 19:08:00,539 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [34] count [3]
2017-09-27 19:08:00,541 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [35] count [3]
2017-09-27 19:08:05,544 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [23] count [4]
2017-09-27 19:08:05,545 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [24] count [4]
2017-09-27 19:08:05,546 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [34] count [4]
2017-09-27 19:08:05,547 ERROR [:] [] [] [] [sched-core-14 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [35] count [4]
2017-09-27 19:08:10,549 ERROR [:] [] [] [] [sched-core-15 ] [i.c.p.agent.server.ping.PingMonitor ] Failed to get ping from agent [23] count [5]

@superseb

This comment has been minimized.

Show comment
Hide comment
@superseb

superseb Sep 28, 2017

Member

Can you please share more details, what exact command did you use to start rancher/server? Where are you running the machine, what OS? I can't reproduce this, rebooted multiple times.

Member

superseb commented Sep 28, 2017

Can you please share more details, what exact command did you use to start rancher/server? Where are you running the machine, what OS? I can't reproduce this, rebooted multiple times.

@stefanvangastel

This comment has been minimized.

Show comment
Hide comment
@stefanvangastel

stefanvangastel Oct 29, 2017

It has something to do with mounting /var/lib/mysql. Had the same issue removing -v /var/lib/mysql:/var/lib/mysql made the rancher-server start again.

stefanvangastel commented Oct 29, 2017

It has something to do with mounting /var/lib/mysql. Had the same issue removing -v /var/lib/mysql:/var/lib/mysql made the rancher-server start again.

@ersutton

This comment has been minimized.

Show comment
Hide comment
@ersutton

ersutton Nov 20, 2017

Not sure if this is applicable to the preview. But I recently got bitten by the latest stable version because of database ownership permissions that only reared their head on server restart.
see this http://rancher.com/docs/rancher/v1.6/en/upgrading/#single-container-bind-mount
you have to chown the database directory to be 102:105
hope that helps.

ersutton commented Nov 20, 2017

Not sure if this is applicable to the preview. But I recently got bitten by the latest stable version because of database ownership permissions that only reared their head on server restart.
see this http://rancher.com/docs/rancher/v1.6/en/upgrading/#single-container-bind-mount
you have to chown the database directory to be 102:105
hope that helps.

@superseb

This comment has been minimized.

Show comment
Hide comment
@superseb

superseb Jan 30, 2018

Member

With the release of Tech Preview 2, the architecture behind 2.0 has significantly changed. Any issues with any release prior to v2.0.0-alpha12 is most likely invalid.

Please watch our Tech Preview 2 meetup to learn more about the changes in 2.0 in Tech Preview 2.

Please file new issues after using Tech Preview 2 (any release v2.0.0-alpha12+). As always, thanks for using Rancher!

Member

superseb commented Jan 30, 2018

With the release of Tech Preview 2, the architecture behind 2.0 has significantly changed. Any issues with any release prior to v2.0.0-alpha12 is most likely invalid.

Please watch our Tech Preview 2 meetup to learn more about the changes in 2.0 in Tech Preview 2.

Please file new issues after using Tech Preview 2 (any release v2.0.0-alpha12+). As always, thanks for using Rancher!

@superseb superseb closed this Jan 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment