-
Notifications
You must be signed in to change notification settings - Fork 549
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: skip tmpfs mounts only for select paths #2918
Conversation
5eac666
to
a18e307
Compare
I've pushed extra tests, a refactor, and updated the PR description to reflect the current state. |
I'm using the new |
This output uses a snapshot build (
|
b00ab1b
to
b4fb927
Compare
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
b4fb927
to
94bdb11
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟢 Thanks for the pairing and checking on this one this AM!
Syft originally attempted to use path based exclusion for known problematic paths to scan (e.g.
/sys
), then eventually moved to a filesystem-based detection to achieve identifying parts of the system to skip. This has caused a few different problems:Syft currently skips everything under a
tmpfs
mount, seesyft/syft/internal/fileresolver/directory_indexer.go
Line 488 in aafa161
An especially bad case of this bug is syft does not find anything in archives if /tmp is a tmpfs #2894 - when pointed at an archive, syft will untar the archive to temp directory, and then ignore the directory it just made if the directory is on a tmpfs mount.
Syft currently ignores files that are on a non-ignored mount type that's mounted at a path under an ignored mount type, e.g. in the nixOS case where
/
istmpfs
, but/home/permanent
isext4
or something,/home/permanent
is incorrectly ignored because it's under ` tmpfs mount. This is resolved in this PR by only considering the longest prefix match between the mount infos and the path under consideration, instead of every prefix match.The main observation out of the above problems is that ignoring
tmpfs
globally is problematic. The goal of this PR is to combine the best of bother pervious approaches -- path based exclusion and filesystem type exclusion -- in order to avoid problematic areas of the filesystem. These should be combined in a way such that we are allowingtmpfs
to be scanned in areas outside of known problematic paths, thereby allowing syft to scan more areas of the system.This PR tries to change tries to make the ignoring of paths more cleanly in terms of
32 business rules:tmpfs
filesystem types (e.g.sysfs
,devfs
,proc
, etc.)tmpfs
type mounts if they're at certain paths.Originally this PR had a third rule:
Special case: never ignore the entire directory being scanned... however, this opens the door for syft to infinity scan a forbidden path, which isn't the intention. Sosyft scan /
should still ignore/proc
and other forbidden paths, just because it's within the scan target doesn't mean we should drop all ignore rules. But this rules doesn't make sense since if a user attemptssyft scan /proc
we would honor it, but it would still appear to be an empty directory (since we ignored all of the contents of/proc
). For these reasons I've elected to drop this rule.Additionally this PR adds a CLI test to show that archive scans function as expected (that at least one cataloger is wired, and a result is found). This is a smoke-like test for this general functionality in the context of the temp dir being the destination for the archive.
Note
This PR also bumps the minimum required go version from
1.21.0
to1.22.0
in order to use new functionalityFixes #2894
Fixes #2847
See for additional context: anchore/grype#1822