OCPBUGS-88742: Fix nested container test mount check for BusyBox#31361
Conversation
BusyBox `mount` truncates output when /proc/self/mounts entries exceed ~1008 bytes, causing 700-play.bats to fail. Replace `mount | grep /tmp` with `grep /tmp /proc/self/mounts`. Assisted-by: Claude Code <https://claude.com/claude-code>
|
Pipeline controller notification For optional jobs, comment This repository is configured in: automatic mode |
openshift-ci-robot
added
jira/valid-reference
|
@bitoku: This pull request references Jira Issue OCPBUGS-88742, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
|
@bitoku: This pull request references Jira Issue OCPBUGS-88742, which is invalid:
Comment DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
WalkthroughBoth the embedded testdata bindata and the nested_container skip_tests.sh script were updated to patch the 700-play.bats test, replacing "mount | grep /tmp" with "grep /tmp /proc/self/mounts" via sed, with added comments explaining BusyBox mount truncation. ChangesMount Truncation Fix
Estimated code review effort: 1 (Trivial) | ~3 minutes Poem: 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/jira refresh |
openshift-ci-robot
added
jira/valid-bug
|
@bitoku: This pull request references Jira Issue OCPBUGS-88742, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@bitoku: This pull request references Jira Issue OCPBUGS-88742, which is valid. 3 validation(s) were run on this bug
DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/payload-job periodic-ci-openshift-release-main-ci-5.0-e2e-gcp-ovn-usernamespace |
|
@bitoku: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/dcf1db20-754f-11f1-97b4-03d2d6384f86-0 |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
openshift-ci
Bot
added
the
ready-for-human-review
|
Scheduling required tests: |
|
/payload-job periodic-ci-openshift-release-main-ci-5.0-e2e-gcp-ovn-usernamespace |
|
@bitoku: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/54f4a1d0-755e-11f1-90b7-909de0e271c7-0 |
|
/retest |
| # Replace `mount` with /proc/self/mounts — BusyBox mount truncates output | ||
| # when /proc/self/mounts entries exceed ~1008 bytes | ||
| # https://redhat.atlassian.net/browse/OCPBUGS-88742?focusedCommentId=17458910 | ||
| sed -i 's#mount | grep /tmp#grep /tmp /proc/self/mounts#' $TEST_DIR/700-play.bats |
There was a problem hiding this comment.
non blocking nit:
| sed -i 's#mount | grep /tmp#grep /tmp /proc/self/mounts#' $TEST_DIR/700-play.bats | |
| sed -i 's#mount | grep /tmp#grep /tmp /proc/self/mounts#' "$TEST_DIR/700-play.bats" |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bitoku, dgoodwin, saschagrunert The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
openshift-ci-robot
added
the
verified
|
@haircommander: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@bitoku: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/skip |
|
@bitoku: Jira Issue Verification Checks: Jira Issue OCPBUGS-88742 Jira Issue OCPBUGS-88742 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
5 participants
BusyBox
mounttruncates output when /proc/self/mounts entries exceed ~1008 bytes, causing 700-play.bats to fail. Replacemount | grep /tmpwithgrep /tmp /proc/self/mounts.OCPBUGS-88742: Summary
Bug:
podman kube generate tmpfs on /tmptest failure in OCP 5.0Component: Node / Kubelet | Version: 5.0 | Label: component-regression
Jira: https://redhat.atlassian.net/browse/OCPBUGS-88742
Triage: https://sippy-auth.dptools.openshift.org/sippy-ng/component_readiness/triages/586
What Happened
The podman system test
podman kube generate tmpfs on /tmp(containers/podman:test/system/700-play.bats) passes on OCP 4.22 but fails on OCP 5.0. The test runs inside a user-namespace pod (hostUsers: false), creates a nested container viapodman kube play, and checks for/tmpby runningmount | grep /tmp. On OCP 5.0, BusyBoxmountreturns only 3 entries instead of 36, so/tmpis missing from the output.The
/tmpmount IS present and functional — visible viacat /proc/self/mounts,findmnt, anddf -h. Only BusyBoxmountfails to display it.Root Cause
BusyBox
mountcalls musl'sgetmntent_r()with a fixed ~1008-byte buffer. When a line in/proc/self/mountsexceeds this, musl returns NULL and aborts all iteration. In nested containers, overlay bind-mount entries (e.g./etc/hosts) are ~1881 bytes due to longlowerdir=paths. Once BusyBox hits the first such entry, all remaining mounts — including/tmp— are invisible.Why this regresses on OCP 5.0
The overlay entries are equally long on both OCP versions. The difference is mount ordering in
/proc/self/mounts:/tmpappears at line 5, before the first long overlay entry at line 12. BusyBox parses 11 entries including/tmp.2eea9ce4310d(Linux 6.8) replaced the linked list with a red-black tree keyed bymnt_id_unique. Long overlay entries now appear at line 4,/tmpat line 12. BusyBox parses only 3 entries./tmpposition in/proc/self/mounts/tmp)/tmp)How to Reproduce
On OCP 5.0 (kernel 6.12+), run a user-namespace pod with nested podman, create a container via
podman kube playwith a tmpfs on/tmp, then compare:Automated reproducer: https://gist.github.com/bitoku/770d63cf49a9a1597090a1988cdcf39c
Recommended Fix
Change the test to use
grep /tmp /proc/self/mountsorfindmnt /tmpinstead ofmount | grep /tmp.Sources
2eea9ce4310d: https://patchwork.kernel.org/project/linux-fsdevel/patch/20231025140205.3586473-3-mszeredi@redhat.com/getmntent_rissue): https://gitlab.alpinelinux.org/alpine/aports/-/issues/7093Assisted-by: Claude Code https://claude.com/claude-code
Summary by CodeRabbit