Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update zip_read_closer to incorporate zip64 support #1041

Merged
merged 4 commits into from
Jun 16, 2022
Merged

update zip_read_closer to incorporate zip64 support #1041

merged 4 commits into from
Jun 16, 2022

Conversation

spiffcs
Copy link
Contributor

@spiffcs spiffcs commented Jun 13, 2022 •
edited

Syft currently throws a warning when scanning zip64 files.

Screen Shot 2022-06-13 at 3 45 16 PM

This update adds new patches sourced from the golang standard library.

When v1.19 launches in August the syft project should be able to remove most of this code and use the newly added prefix support seen in this CL.
https://go-review.googlesource.com/c/go/+/387976/3/src/archive/zip/reader.go#544

With the new update you can see the warning is no longer thrown:
Screen Shot 2022-06-14 at 12 29 54 PM

Todo:

  • Add unit tests to prevent regressions when moving to v1.19 golang

Signed-off-by: Christopher Phillips christopher.phillips@anchore.com

@spiffcs
update zip_read_closer to incorporate zip64 support
2722466 
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@github-actions
Copy link

github-actions bot commented Jun 13, 2022 •
edited

Benchmark Test Results

Benchmark results from the latest changes vs base branch 
name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/alpmdb-cataloger-2                    14.9ms ±22%    13.2ms ± 4%  -11.55%  (p=0.016 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.57ms ± 4%    1.56ms ± 5%     ~     (p=1.000 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            3.74ms ± 2%    3.63ms ± 6%     ~     (p=0.056 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2    1.27ms ± 1%    1.18ms ± 4%   -6.97%  (p=0.008 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         874µs ± 1%     829µs ± 4%   -5.18%  (p=0.008 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                    1.06ms ± 3%    0.97ms ± 2%   -9.03%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                     1.03ms ± 4%    0.95ms ± 5%   -7.16%  (p=0.016 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      17.3ms ± 2%    15.8ms ± 6%   -8.41%  (p=0.008 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.55ms ± 2%    1.35ms ± 8%  -13.03%  (p=0.008 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          2.83µs ± 3%    2.42µs ± 4%  -14.47%  (p=0.008 n=5+5)
ImagePackageCatalogers/dotnet-deps-cataloger-2               1.71ms ± 5%    1.49ms ± 3%  -12.94%  (p=0.008 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/alpmdb-cataloger-2                    5.25MB ± 0%    5.25MB ± 0%     ~     (p=0.310 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               193kB ± 0%     193kB ± 0%     ~     (p=0.548 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             916kB ± 0%     917kB ± 0%     ~     (p=1.000 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     205kB ± 0%     204kB ± 0%   -0.05%  (p=0.032 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         149kB ± 0%     149kB ± 0%     ~     (p=0.056 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     194kB ± 0%     194kB ± 0%     ~     (p=0.730 n=5+5)
ImagePackageCatalogers/rpmdb-cataloger-2                      224kB ± 0%     224kB ± 0%   +0.07%  (p=0.016 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.36MB ± 0%    3.36MB ± 0%     ~     (p=0.841 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.24MB ± 0%    1.24MB ± 0%     ~     (p=0.151 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            672B ± 0%      672B ± 0%     ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                360kB ± 0%     360kB ± 0%     ~     (p=0.548 n=5+5)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/alpmdb-cataloger-2                     85.4k ± 0%     85.4k ± 0%     ~     (p=1.000 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               3.96k ± 0%     3.96k ± 0%     ~     (p=0.167 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             15.7k ± 0%     15.7k ± 0%     ~     (p=0.754 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     5.24k ± 0%     5.24k ± 0%     ~     (p=0.333 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         3.02k ± 0%     3.02k ± 0%     ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     4.30k ± 0%     4.30k ± 0%     ~     (all equal)
ImagePackageCatalogers/rpmdb-cataloger-2                      5.53k ± 0%     5.53k ± 0%     ~     (all equal)
ImagePackageCatalogers/java-cataloger-2                       54.9k ± 0%     54.9k ± 0%     ~     (p=0.516 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      5.14k ± 0%     5.14k ± 0%     ~     (p=1.000 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2            15.0 ± 0%      15.0 ± 0%     ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                6.98k ± 0%     6.98k ± 0%     ~     (all equal)

@spiffcs
update gocognit
af06250 
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@spiffcs
update script to convert to zip64
82d4f8a 
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@spiffcs
update tests with new bool args
8891542 
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@spiffcs spiffcs marked this pull request as ready for review June 14, 2022 16:18
@spiffcs spiffcs requested a review from a team June 14, 2022 16:30
wagoodman
wagoodman approved these changes Jun 14, 2022
View reviewed changes
Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@spiffcs spiffcs merged commit 9e72771 into main Jun 16, 2022
@spiffcs spiffcs deleted the zip64-support branch June 16, 2022 14:43
spiffcs added a commit to jonasagx/syft that referenced this pull request Jun 27, 2022
@spiffcs
Merge branch 'main' into PR
55c1775 
* main: (70 commits)
  fix: add php catalogers to all catalogers (anchore#1065)
  feat: add use-all-catalogers flag (anchore#1050)
  Updates parsing of `yarn.lock` to use `resolved` URLs that are pulled from yarn and npm registries (anchore#926)
  remove OSS Meetup message (anchore#1057)
  add pom.xml cataloger (anchore#1055)
  Add support for CBL-Mariner distroless images (anchore#1045)
  Add catalogers configuration (anchore#1038)
  add template output (anchore#1051)
  update stereoscope to latest version (anchore#1052)
  update zip_read_closer to incorporate zip64 support (anchore#1041)
  Add pacman (alpm) parser support (anchore#943)
  Update of README.md (anchore#1027)
  bump cosign to v1.9.0 to resolve reporting of GHSA-66x3-6cw3-v5gj (anchore#1025)
  add workflows to test new project automation (anchore#1023)
  improve LanguageByName and add unit tests (anchore#1034)
  Read Description from dpkg status files (anchore#996)
  Add announcement for Anchore OSS Virtual Meetup (anchore#1033)
  add main module field to go bin metadata (anchore#1026)
  Add filters to package cataloger (anchore#1021)
  change draft to false for release process (anchore#1016)
  ...

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
aiwantaozi pushed a commit to aiwantaozi/syft that referenced this pull request Oct 20, 2022
@spiffcs @aiwantaozi
update zip_read_closer to incorporate zip64 support (anchore#1041)
d9340a4
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
@spiffcs
update zip_read_closer to incorporate zip64 support (anchore#1041)
dfa5ba8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wagoodman wagoodman wagoodman approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@spiffcs @wagoodman