Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: do not attempt to add 'private' symlinks when RootImage/RootDirectory are used #22272

Merged
merged 4 commits into from Jan 28, 2022

Conversation

bluca
Copy link
Member

@bluca bluca commented Jan 27, 2022

A bind mount is added directly from private on the host to the actual
destination directory, no need for the symlinks (which cannot be created
as the bind mount happens first and creates the target as an actual directory)

Fixes #22264

There's an app0.service in the extension app0.raw, so don't use the same
name for a unit in minimal.raw
Makes the setup idempotent, as mksquashfs by default attempts to
append to an existing image
Copy link
Member

@yuwata yuwata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@yuwata yuwata added the good-to-merge/waiting-for-ci 👍 PR is good to merge, but CI hasn't passed at time of review. Please merge if you see CI has passed label Jan 27, 2022
@gdamjan
Copy link
Contributor

gdamjan commented Jan 27, 2022

I can confirm that the PR fixes #22264

should this be also backported to 250-stable ? (edit: 🤦🏻‍ sorry I didn't see the label)

@yuwata
Copy link
Member

yuwata commented Jan 27, 2022

TEST-29-PORTABLE failed. Is it related?

@bluca
Copy link
Member Author

bluca commented Jan 27, 2022

Possibly but I don't quite understand why it's affecting the sanitizer build - @mrc0mmand any idea?

@yuwata yuwata added the ci-fails/needs-rework 🔥 Please rework this, the CI noticed an issue with the PR label Jan 27, 2022
@yuwata
Copy link
Member

yuwata commented Jan 27, 2022

tentatively set the red label.

@mrc0mmand
Copy link
Member

I'm a bit confused now how this all should work, but from a quick glance it looks like a timing issue?

[   43.751222] testsuite-29.sh[337]: + status=running-runtime
[   43.751222] testsuite-29.sh[337]: + [[ running-runtime == \r\u\n\n\i\n\g\-\r\u\n\t\i\m\e ]]
[   43.751222] testsuite-29.sh[337]: + portablectl detach --now --runtime --extension /usr/share/app1.raw /usr/share/minimal_1.raw app1
[   43.779000] portablectl[905]: Bus n/a: changing state UNSET → OPENING
[   43.781944] portablectl[905]: sd-bus: starting bus by connecting to /run/dbus/system_bus_socket...
[   43.782593] portablectl[905]: Bus n/a: changing state OPENING → AUTHENTICATING
[   43.783161] portablectl[905]: (Matching unit files with prefixes 'app1'.)
...
[   44.123614] portablectl[905]: Queued /org/freedesktop/systemd1/job/1235 to call StopUnit on portable service app1.service.
[   44.127002] systemd[1]: app1.service: Deactivated successfully.
[   44.127687] systemd[1]: app1.service: Service restart not allowed.
[   44.128325] systemd[1]: app1.service: Changed exited -> dead
[   44.147912] audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=app1 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   44.148203] systemd[1]: app1.service: Job 1235 app1.service/stop finished, result=done
[   44.148422] systemd[1]: Stopped app1.service.
...
[   45.364478] testsuite-29.sh[337]: + grep -q -F bar /var/lib/private/app0/foo
[   45.366882] testsuite-29.sh[936]: grep: /var/lib/private/app0/foo: No such file or directory
...
[   45.436828] portablectl[905]: Removed /run/systemd/system.attached/app1.service.
[   45.437031] portablectl[905]: Removed /run/systemd/system.attached/app1.service.d/10-profile.conf.
[   45.437260] portablectl[905]: Removed /run/systemd/system.attached/app1.service.d/20-portable.conf.
[   45.437524] portablectl[905]: Removed /run/systemd/system.attached/app1.service.d.
[   45.437773] portablectl[905]: Removed /run/portables/app1.raw.
[   45.438083] portablectl[905]: Removed /run/portables/minimal_1.raw.
[   45.438626] portablectl[905]: Removed /run/systemd/system.attached.
[   45.438903] portablectl[905]: Bus n/a: changing state RUNNING → CLOSED

The portable-related stuff is mostly dark magic for me, so I'm not sure what's going wrong there.

@bluca
Copy link
Member Author

bluca commented Jan 27, 2022

There's two services running, and creating those files, by the time they are removed they should be all done, but maybe I need to add some looping

@mrc0mmand
Copy link
Member

There's two services running, and creating those files, by the time they are removed they should be all done, but maybe I need to add some looping

I see. I'll do a couple of local tests just to confirm this, so we don't have to for CI confirmation in case it's something else.

@mrc0mmand
Copy link
Member

mrc0mmand commented Jan 27, 2022

Indeed. I can reproduce the issue locally and applying:

diff --git a/test/units/testsuite-29.sh b/test/units/testsuite-29.sh
index 3afdc44ff4..8cffb2e66d 100755
--- a/test/units/testsuite-29.sh
+++ b/test/units/testsuite-29.sh
@@ -112,8 +112,13 @@ portablectl detach --now --runtime --extension /usr/share/app1.raw /usr/share/mi
 # Ensure that the combination of read-only images, state directory and dynamic user works, and that
 # state is retained. Check after detaching, as on slow systems (eg: sanitizers) it might take a while
 # after the service is attached before the file appears.
-grep -q -F bar /var/lib/private/app0/foo
-grep -q -F baz /var/lib/private/app1/foo
+for ((i = 0; i < 20; i++)); do
+    if grep -q -F bar /var/lib/private/app0/foo && grep -q -F baz /var/lib/private/app1/foo; then
+        break
+    fi
+
+    sleep 0.5
+done
 
 # portablectl also works with directory paths rather than images
 

makes the test gods happy:

[   16.261704] testsuite-29.sh[1133]: ++ portablectl is-attached --extension app1 minimal_1
[   16.304906] testsuite-29.sh[602]: + status=running-runtime
[   16.305371] testsuite-29.sh[602]: + [[ running-runtime == \r\u\n\n\i\n\g\-\r\u\n\t\i\m\e ]]
[   16.305669] testsuite-29.sh[602]: + portablectl detach --now --runtime --extension /usr/share/app1.raw /usr/share/minimal_1.raw app1
[  OK  ] Stopped app1.service.
[   17.143933] testsuite-29.sh[602]: + (( i = 0 ))
[   17.144483] testsuite-29.sh[602]: + (( i < 20 ))
[   17.145013] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   17.146137] testsuite-29.sh[1166]: grep: /var/lib/private/app0/foo: No such file or directory
[   17.147076] testsuite-29.sh[602]: + sleep 0.5
[   17.647463] testsuite-29.sh[602]: + (( i++ ))
[   17.648123] testsuite-29.sh[602]: + (( i < 20 ))
[   17.648639] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   17.649432] testsuite-29.sh[1168]: grep: /var/lib/private/app0/foo: No such file or directory
[   17.650190] testsuite-29.sh[602]: + sleep 0.5
[   18.148781] testsuite-29.sh[602]: + (( i++ ))
[   18.149340] testsuite-29.sh[602]: + (( i < 20 ))
[   18.149808] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   18.150475] testsuite-29.sh[1170]: grep: /var/lib/private/app0/foo: No such file or directory
[   18.151093] testsuite-29.sh[602]: + sleep 0.5
[   18.650985] testsuite-29.sh[602]: + (( i++ ))
[   18.651937] testsuite-29.sh[602]: + (( i < 20 ))
[   18.652773] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   18.654074] testsuite-29.sh[1172]: grep: /var/lib/private/app0/foo: No such file or directory
[   18.655345] testsuite-29.sh[602]: + sleep 0.5
[   19.151851] testsuite-29.sh[602]: + (( i++ ))
[   19.152393] testsuite-29.sh[602]: + (( i < 20 ))
[   19.152832] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   19.154942] testsuite-29.sh[1174]: grep: /var/lib/private/app0/foo: No such file or directory
[   19.155234] testsuite-29.sh[602]: + sleep 0.5
[   19.656476] testsuite-29.sh[602]: + (( i++ ))
[   19.657033] testsuite-29.sh[602]: + (( i < 20 ))
[   19.657634] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   19.658645] testsuite-29.sh[1176]: grep: /var/lib/private/app0/foo: No such file or directory
[   19.659978] testsuite-29.sh[602]: + sleep 0.5
[   20.160824] testsuite-29.sh[602]: + (( i++ ))
[   20.162071] testsuite-29.sh[602]: + (( i < 20 ))
[   20.162999] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   20.164559] testsuite-29.sh[1178]: grep: /var/lib/private/app0/foo: No such file or directory
[   20.165900] testsuite-29.sh[602]: + sleep 0.5
[   20.663229] testsuite-29.sh[602]: + (( i++ ))
[   20.663552] testsuite-29.sh[602]: + (( i < 20 ))
[   20.664431] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   20.665303] testsuite-29.sh[1180]: grep: /var/lib/private/app0/foo: No such file or directory
[   20.665951] testsuite-29.sh[602]: + sleep 0.5
[   21.164882] testsuite-29.sh[602]: + (( i++ ))
[   21.165414] testsuite-29.sh[602]: + (( i < 20 ))
[   21.165851] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   21.166848] testsuite-29.sh[1182]: grep: /var/lib/private/app0/foo: No such file or directory
[   21.167231] testsuite-29.sh[602]: + sleep 0.5
[   21.668088] testsuite-29.sh[602]: + (( i++ ))
[   21.668487] testsuite-29.sh[602]: + (( i < 20 ))
[   21.668819] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   21.670385] testsuite-29.sh[1184]: grep: /var/lib/private/app0/foo: No such file or directory
[   21.670662] testsuite-29.sh[602]: + sleep 0.5
[   22.171299] testsuite-29.sh[602]: + (( i++ ))
[   22.171664] testsuite-29.sh[602]: + (( i < 20 ))
[   22.171982] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   22.174210] testsuite-29.sh[1186]: grep: /var/lib/private/app0/foo: No such file or directory
[   22.174372] testsuite-29.sh[602]: + sleep 0.5
[   22.677014] testsuite-29.sh[602]: + (( i++ ))
[   22.677426] testsuite-29.sh[602]: + (( i < 20 ))
[   22.677844] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   22.679299] testsuite-29.sh[1188]: grep: /var/lib/private/app0/foo: No such file or directory
[   22.679678] testsuite-29.sh[602]: + sleep 0.5
[   23.181740] testsuite-29.sh[602]: + (( i++ ))
[   23.182075] testsuite-29.sh[602]: + (( i < 20 ))
[   23.182340] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   23.184659] testsuite-29.sh[1190]: grep: /var/lib/private/app0/foo: No such file or directory
[   23.185058] testsuite-29.sh[602]: + sleep 0.5
[   23.686532] testsuite-29.sh[602]: + (( i++ ))
[   23.686877] testsuite-29.sh[602]: + (( i < 20 ))
[   23.687220] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   23.688124] testsuite-29.sh[1192]: grep: /var/lib/private/app0/foo: No such file or directory
[   23.688335] testsuite-29.sh[602]: + sleep 0.5
[   24.189692] testsuite-29.sh[602]: + (( i++ ))
[   24.190043] testsuite-29.sh[602]: + (( i < 20 ))
[   24.190382] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   24.192931] testsuite-29.sh[1194]: grep: /var/lib/private/app0/foo: No such file or directory
[   24.193105] testsuite-29.sh[602]: + sleep 0.5
[   24.696291] testsuite-29.sh[602]: + (( i++ ))
[   24.696878] testsuite-29.sh[602]: + (( i < 20 ))
[   24.697388] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   24.699178] testsuite-29.sh[1196]: grep: /var/lib/private/app0/foo: No such file or directory
[   24.699614] testsuite-29.sh[602]: + sleep 0.5
[   25.201601] testsuite-29.sh[602]: + (( i++ ))
[   25.202213] testsuite-29.sh[602]: + (( i < 20 ))
[   25.202763] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   25.203735] testsuite-29.sh[1198]: grep: /var/lib/private/app0/foo: No such file or directory
[   25.204247] testsuite-29.sh[602]: + sleep 0.5
[   25.706762] testsuite-29.sh[602]: + (( i++ ))
[   25.707293] testsuite-29.sh[602]: + (( i < 20 ))
[   25.707743] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   25.708886] testsuite-29.sh[1200]: grep: /var/lib/private/app0/foo: No such file or directory
[   25.709256] testsuite-29.sh[602]: + sleep 0.5
[   26.210941] testsuite-29.sh[602]: + (( i++ ))
[   26.211406] testsuite-29.sh[602]: + (( i < 20 ))
[   26.211738] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   26.214363] testsuite-29.sh[1202]: grep: /var/lib/private/app0/foo: No such file or directory
[   26.214754] testsuite-29.sh[602]: + sleep 0.5
[   26.717114] testsuite-29.sh[602]: + (( i++ ))
[   26.717523] testsuite-29.sh[602]: + (( i < 20 ))
[   26.717966] testsuite-29.sh[602]: + grep -q -F bar /var/lib/private/app0/foo
[   26.719451] testsuite-29.sh[1204]: grep: /var/lib/private/app0/foo: No such file or directory
[   26.719876] testsuite-29.sh[602]: + sleep 0.5
[   27.221860] testsuite-29.sh[602]: + (( i++ ))
[   27.222363] testsuite-29.sh[602]: + (( i < 20 ))
[   27.222729] testsuite-29.sh[602]: + mkdir /tmp/rootdir /tmp/app0 /tmp/app1 /tmp/overlay /tmp/os-release-fix /tmp/os-release-fix/etc
[   27.223681] testsuite-29.sh[602]: + mount /usr/share/app0.raw /tmp/app0
[   27.225465] testsuite-29.sh[602]: + mount /usr/share/app1.raw /tmp/app1
[   27.228262] testsuite-29.sh[602]: + mount /usr/share/minimal_0.raw /tmp/rootdir
[   27.231265] testsuite-29.sh[602]: + grep -v '^PORTABLE_PREFIXES=' /tmp/rootdir/etc/os-release
[   27.235750] testsuite-29.sh[602]: + mount -t overlay overlay -o lowerdir=/tmp/os-release-fix:/tmp/app1:/tmp/rootdir /tmp/overlay
[   27.240866] testsuite-29.sh[602]: + grep . /tmp/overlay/usr/lib/extension-release.d/extension-release.app2

@bluca
Copy link
Member Author

bluca commented Jan 27, 2022

wow that's really slow - ok thanks, this really helps, will fix it

@mrc0mmand
Copy link
Member

wow that's really slow - ok thanks, this really helps, will fix it

yeah, I guess bumping the sleep to 1s might make sense, given the loop runs for more than one iteration only in sanitizer runs

@yuwata
Copy link
Member

yuwata commented Jan 27, 2022

After the loop, the grep needs to be called again, as all attempts in the loop may be failed.

@bluca
Copy link
Member Author

bluca commented Jan 27, 2022

It's really weird, all the calls before the check should be blocking

@bluca bluca removed the ci-fails/needs-rework 🔥 Please rework this, the CI noticed an issue with the PR label Jan 27, 2022
…ectory are used

A bind mount is added directly from private on the host to the actual
destination directory, no need for the symlinks (which cannot be created
as the bind mount happens first and creates the target as an actual directory)

Fixes systemd#22264
@bluca
Copy link
Member Author

bluca commented Jan 28, 2022

Ah, I understand the issue - with sanitizers, the trusted profile is used, which means DynamicUser is disabled. But in that case the state directory storage is /var/lib/foo, not /var/lib/private/foo, but the test was hard-coded to check the latter only. Should be fixed now.

@yuwata yuwata merged commit 673a181 into systemd:main Jan 28, 2022
@bluca bluca deleted the state_dir_private_rootfs branch January 28, 2022 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good-to-merge/waiting-for-ci 👍 PR is good to merge, but CI hasn't passed at time of review. Please merge if you see CI has passed pid1 portable Anything to do with systemd-portable and portablectl and portables
4 participants