Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hanging start-stop-daemon in footloose Alpine #3436

Merged
merged 1 commit into from Aug 29, 2023

Conversation

twz123
Copy link
Member

@twz123 twz123 commented Aug 29, 2023

Description

Some distributions package Docker in a way that allows for a vast number of open file descriptors, often achieved through systemd's LimitNOFILE=infinity configuration. This situation leads to start-stop-daemon appearing to hang, consequently blocking the OpenRC init sequence. However, the issue is not an actual hang; rather, it attempts to close all open file descriptors, resulting in a loop with approximately one billion iterations of the close() syscall. This behavior effectively renders the footloose machine unable to boot.

Mitigate this by setting the open file descriptor limit to 1 Mi in OpenRC's rc.conf.

See:

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

How Has This Been Tested?

  • Manual test
  • Auto test added

Checklist:

  • My code follows the style guidelines of this project
  • My commit messages are signed-off
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

Some distributions package Docker in a way that allows for a vast
number of open file descriptors, often achieved through systemd's
`LimitNOFILE=infinity` configuration. This situation leads to start-
stop-daemon appearing to hang, consequently blocking the OpenRC
init sequence. However, the issue is not an actual hang; rather, it
attempts to close all open file descriptors, resulting in a loop with
approximately one billion iterations of the close() syscall. This
behavior effectively renders the footloose machine unable to boot.

Mitigate this by setting the open file descriptor limit to 1 Mi in
OpenRC's rc.conf.

Signed-off-by: Tom Wieczorek <twieczorek@mirantis.com>
Link: moby/moby@8db6109
Link: https://github.com/OpenRC/openrc/blob/eb8831a1416ab2ee8123b3add78421c2aa316b39/src/start-stop-daemon/start-stop-daemon.c#L1104-L1105
@twz123 twz123 added chore area/smoke-tests backport/release-1.27 PR that needs to be backported/cherrypicked to release-1.27 branch labels Aug 29, 2023
@twz123 twz123 requested a review from a team as a code owner August 29, 2023 12:30
@twz123 twz123 requested review from ncopa and kke August 29, 2023 12:30
@twz123 twz123 merged commit 55dceaf into k0sproject:main Aug 29, 2023
73 checks passed
@twz123 twz123 deleted the footloose-max-fds branch August 29, 2023 13:18
@k0s-bot
Copy link

k0s-bot commented Aug 29, 2023

Successfully created backport PR for release-1.27:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/smoke-tests backport/release-1.27 PR that needs to be backported/cherrypicked to release-1.27 branch chore
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants