Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

snap/squashfs: handle squashfs-tools 4.5+ #10567

Merged
merged 1 commit into from Jul 29, 2021

Conversation

bboozzoo
Copy link
Collaborator

Arch recently landed an update of squahfs-tools to 4.5 which broke our code. The
main difference is that the output of unsquahfs stdout does not contain the
header we expected. For instance:

google:ubuntu-20.04-64 .../mini/hello# unsquashfs -n -dest . -ll /root/foo.snap
Parallel unsquashfs: Using 1 processor
5 inodes (1 blocks) to write

drwx------ root/root                55 2021-07-27 11:31 .
-rw-r--r-- root/root                 0 2021-07-27 11:31 ./data.bin
drwxr-xr-x root/root                27 2021-07-27 11:31 ./food
drwxr-xr-x root/root                27 2021-07-27 11:31 ./food/bard
drwxr-xr-x root/root                 3 2021-07-27 11:31 ./food/bard/bazd
drwxr-xr-x root/root                45 2021-07-27 11:31 ./meta
drwxr-xr-x root/root                58 2021-07-27 11:31 ./meta/hooks
-rwxr-xr-x root/root                 0 2021-07-27 11:31 ./meta/hooks/bar-hook
drwxr-xr-x root/root                26 2021-07-27 11:31 ./meta/hooks/dir
-rwxr-xr-x root/root                 0 2021-07-27 11:31 ./meta/hooks/dir/baz
-rwxr-xr-x root/root                 0 2021-07-27 11:31 ./meta/hooks/foo-hook
-rw-r--r-- root/root                 9 2021-07-27 11:31 ./meta/snap.yaml

While on Arch with squashfs 4.5:

$ unsquashfs -n -dest . -ll /tmp/check-1302223697476122084/0/foo.snap
drwx------ root/root                55 2021-07-27 13:31 .
-rw-r--r-- root/root                 0 2021-07-27 13:31 ./data.bin
drwxr-xr-x root/root                27 2021-07-27 13:31 ./food
drwxr-xr-x root/root                27 2021-07-27 13:31 ./food/bard
drwxr-xr-x root/root                 3 2021-07-27 13:31 ./food/bard/bazd
drwxr-xr-x root/root                45 2021-07-27 13:31 ./meta
drwxr-xr-x root/root                58 2021-07-27 13:31 ./meta/hooks
-rwxr-xr-x root/root                 0 2021-07-27 13:31 ./meta/hooks/bar-hook
drwxr-xr-x root/root                26 2021-07-27 13:31 ./meta/hooks/dir
-rwxr-xr-x root/root                 0 2021-07-27 13:31 ./meta/hooks/dir/baz
-rwxr-xr-x root/root                 0 2021-07-27 13:31 ./meta/hooks/foo-hook
-rw-r--r-- root/root                 9 2021-07-27 13:31 ./meta/snap.yaml

@bboozzoo bboozzoo added ⚠ Critical High-priority stuff (e.g. to fix master) Simple 😃 A small PR which can be reviewed quickly labels Jul 27, 2021
Arch recently landed an update of squahfs-tools to 4.5 which broke our code. The
main difference is that the output of unsquahfs stdout does not contain the
header we expected. For instance:

google:ubuntu-20.04-64 .../mini/hello# unsquashfs -n -dest . -ll /root/foo.snap
Parallel unsquashfs: Using 1 processor
5 inodes (1 blocks) to write

drwx------ root/root                55 2021-07-27 11:31 .
-rw-r--r-- root/root                 0 2021-07-27 11:31 ./data.bin
drwxr-xr-x root/root                27 2021-07-27 11:31 ./food
drwxr-xr-x root/root                27 2021-07-27 11:31 ./food/bard
drwxr-xr-x root/root                 3 2021-07-27 11:31 ./food/bard/bazd
drwxr-xr-x root/root                45 2021-07-27 11:31 ./meta
drwxr-xr-x root/root                58 2021-07-27 11:31 ./meta/hooks
-rwxr-xr-x root/root                 0 2021-07-27 11:31 ./meta/hooks/bar-hook
drwxr-xr-x root/root                26 2021-07-27 11:31 ./meta/hooks/dir
-rwxr-xr-x root/root                 0 2021-07-27 11:31 ./meta/hooks/dir/baz
-rwxr-xr-x root/root                 0 2021-07-27 11:31 ./meta/hooks/foo-hook
-rw-r--r-- root/root                 9 2021-07-27 11:31 ./meta/snap.yaml

While on Arch with squashfs 4.5:

$ unsquashfs -n -dest . -ll /tmp/check-1302223697476122084/0/foo.snap
drwx------ root/root                55 2021-07-27 13:31 .
-rw-r--r-- root/root                 0 2021-07-27 13:31 ./data.bin
drwxr-xr-x root/root                27 2021-07-27 13:31 ./food
drwxr-xr-x root/root                27 2021-07-27 13:31 ./food/bard
drwxr-xr-x root/root                 3 2021-07-27 13:31 ./food/bard/bazd
drwxr-xr-x root/root                45 2021-07-27 13:31 ./meta
drwxr-xr-x root/root                58 2021-07-27 13:31 ./meta/hooks
-rwxr-xr-x root/root                 0 2021-07-27 13:31 ./meta/hooks/bar-hook
drwxr-xr-x root/root                26 2021-07-27 13:31 ./meta/hooks/dir
-rwxr-xr-x root/root                 0 2021-07-27 13:31 ./meta/hooks/dir/baz
-rwxr-xr-x root/root                 0 2021-07-27 13:31 ./meta/hooks/foo-hook
-rw-r--r-- root/root                 9 2021-07-27 13:31 ./meta/snap.yaml

Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
@bboozzoo bboozzoo force-pushed the bboozzoo/arch-new-squashfs-tools branch from 7accaa3 to dfba7de Compare July 27, 2021 12:46
Copy link
Contributor

@mvo5 mvo5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@bboozzoo
Copy link
Collaborator Author

Arch is green. Other distros are happy as well. There's some failures related to nfs on Ubuntu 20.04 to 21.10.

Copy link
Contributor

@stolowski stolowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, a small nitpick re naming.

I wonder if we could instead check version with 'unsquashfs -v' and base our logic on that, but no strong opinion.

maybeHeaderRegex.Match(raw) {
continue
} else {
seenHeader = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, I found seenHeader name a bit misleading in realation to maybeHeaderRegex; it suggests seenHeader becomes true when we match maybeHeaderRegex while afaiu it is set after we scan first lines (which may or may not contain the optional headers). Maybe rename it to something like pastHeaders or headersScanned?

@mvo5 mvo5 merged commit 36afb87 into snapcore:master Jul 29, 2021
@bboozzoo bboozzoo modified the milestones: 2.51, 2.52 Aug 16, 2021
@gusseppe
Copy link

gusseppe commented Sep 1, 2021

On Fedora 34 I tried the following to fix the problem:

sudo dnf install squashfs-tools-4.4-5.git1.fc34

It seems that because of this (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=993233), snapd is unusable. So the workaround is to downgrading the squashfs package to fix the problem.

@the-loudspeaker
Copy link

@gusseppe downgrading squashfs is a good workaround. But I'd wait for Fedora's snap package to upgrade.

@anonymouse64
Copy link
Member

@bboozzoo just prepared an update to fedora, you can help in testing it out: https://forum.snapcraft.io/t/snapd-updates-in-fedora/4342/34

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-picked ⚠ Critical High-priority stuff (e.g. to fix master) Simple 😃 A small PR which can be reviewed quickly
Projects
None yet
6 participants