Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

supervisor crashing on startup with Timeout waiting to bind to 0.0.0.0:9631 #7780

Closed
mwrock opened this issue Jul 1, 2020 · 0 comments · Fixed by #7858
Closed

supervisor crashing on startup with Timeout waiting to bind to 0.0.0.0:9631 #7780

mwrock opened this issue Jul 1, 2020 · 0 comments · Fixed by #7858
Assignees
@mwrock
Labels
Focus:Supervisor Related to the Habitat Supervisor (core/hab-sup) component Type: Bug Issues that describe broken functionality

Comments

@mwrock
Copy link
Contributor

mwrock commented Jul 1, 2020

An excerpt from the supervisor log includes:

2020-06-19 19:58:40,433 - Timeout waiting to bind to 0.0.0.0:9631
2020-06-19 19:58:40,435 - [2020-06-20T02:58:40Z ERROR habitat_sup::ctl_gateway::acceptor] Error polling CtlAcceptor shutdown trigger: oneshot canceled
2020-06-19 19:58:40,435 - [2020-06-20T02:58:40Z ERROR habitat_sup::manager::service_updater] Service updater has gone away, yikes!
2020-06-19 19:58:40,435 - [2020-06-20T02:58:40Z ERROR habitat_sup::manager::service_updater] Service updater has gone away, yikes!
2020-06-19 19:58:40,435 - [2020-06-20T02:58:40Z ERROR habitat_sup::manager::service_updater] Service updater has gone away, yikes!
2020-06-19 19:58:40,435 - [2020-06-20T02:58:40Z ERROR habitat_sup::manager::service_updater] Service updater has gone away, yikes!
2020-06-19 19:58:43,600 - [2020-06-20T02:58:43Z ERROR hab_launch] Launcher exiting with code 86

It looks as though the supervisor waits 10 seconds for the http gateway to start and then throws a BindTimeout it has still not started. While one would assume 10 seconds to be adequate, a low resource instance that is CPU saturated at startup (this occurs upon server reboot) may very well hit this timeout.

We should probably extend this timeout and/or make it configurable.
@christophermaier christophermaier added Focus:Supervisor Related to the Habitat Supervisor (core/hab-sup) component Type: Bug Issues that describe broken functionality and removed A-supervisor labels Jul 24, 2020
@mwrock mwrock self-assigned this Jul 31, 2020
@mwrock mwrock mentioned this issue Aug 5, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mwrock mwrock

Labels
Focus:Supervisor Related to the Habitat Supervisor (core/hab-sup) component Type: Bug Issues that describe broken functionality
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

make bind timeout configurable and add debug output around startup habitat-sh/habitat
2 participants
@christophermaier @mwrock