Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rust toolchain binary cataloger #1601

Merged
merged 1 commit into from
Mar 1, 2023
Merged

Conversation

westonsteimel
Copy link
Contributor

Adds binary classifier for the rust toolchain for linux and mac

@github-actions
Copy link

github-actions bot commented Feb 21, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux
goarch: amd64
pkg: github.com/anchore/syft/test/integration
cpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
                                                          │ ./.tmp/benchmark-3d006a4.txt │
                                                          │            sec/op            │
ImagePackageCatalogers/alpmdb-cataloger-2                                   11.79m ± 23%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             893.4µ ±  3%
ImagePackageCatalogers/python-package-cataloger-2                           3.032m ±  1%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   691.9µ ±  1%
ImagePackageCatalogers/javascript-package-cataloger-2                       372.8µ ±  1%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   513.5µ ±  1%
ImagePackageCatalogers/rpm-db-cataloger-2                                   484.8µ ±  1%
ImagePackageCatalogers/java-cataloger-2                                     10.76m ±  1%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.223µ ±  3%
ImagePackageCatalogers/apkdb-cataloger-2                                    509.4µ ±  2%
ImagePackageCatalogers/go-module-binary-cataloger-2                         18.94µ ±  1%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              971.4µ ±  3%
ImagePackageCatalogers/portage-cataloger-2                                  322.7µ ±  2%
ImagePackageCatalogers/sbom-cataloger-2                                     105.3µ ±  1%
ImagePackageCatalogers/binary-cataloger-2                                   179.2µ ±  1%
geomean                                                                     465.2µ

                                                          │ ./.tmp/benchmark-3d006a4.txt │
                                                          │             B/op             │
ImagePackageCatalogers/alpmdb-cataloger-2                                   5.062Mi ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             141.9Ki ± 0%
ImagePackageCatalogers/python-package-cataloger-2                           947.3Ki ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   155.9Ki ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                       95.72Ki ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   144.7Ki ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                   170.5Ki ± 0%
ImagePackageCatalogers/java-cataloger-2                                     2.720Mi ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     1.523Ki ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                    123.0Ki ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                         3.102Ki ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              314.7Ki ± 0%
ImagePackageCatalogers/portage-cataloger-2                                  75.47Ki ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                     13.05Ki ± 0%
ImagePackageCatalogers/binary-cataloger-2                                   26.94Ki ± 0%
geomean                                                                     108.6Ki

                                                          │ ./.tmp/benchmark-3d006a4.txt │
                                                          │          allocs/op           │
ImagePackageCatalogers/alpmdb-cataloger-2                                    86.71k ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.159k ± 0%
ImagePackageCatalogers/python-package-cataloger-2                            15.48k ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                    3.458k ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                        1.253k ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                    2.646k ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                    3.759k ± 0%
ImagePackageCatalogers/java-cataloger-2                                      38.26k ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                       40.00 ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                     3.252k ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                           101.0 ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                               5.011k ± 0%
ImagePackageCatalogers/portage-cataloger-2                                   1.487k ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                       392.0 ± 0%
ImagePackageCatalogers/binary-cataloger-2                                     772.0 ± 0%
geomean                                                                      2.205k

@westonsteimel westonsteimel added the enhancement New feature or request label Feb 21, 2023
@westonsteimel westonsteimel requested a review from a team February 21, 2023 20:47
kzantow
kzantow previously approved these changes Feb 22, 2023
Copy link
Contributor

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be nice to cut down the size of some of the test fixtures (some are in the MB range)

syft/pkg/cataloger/binary/default_classifiers.go Outdated Show resolved Hide resolved
Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...some are in the MB range....

it seems like there may be an extra addition to the gitignore needed for the busybox image fixture tar

@kzantow kzantow dismissed their stale review February 22, 2023 19:36

Alex requested changes

@westonsteimel
Copy link
Contributor Author

...some are in the MB range....

it seems like there may be an extra addition to the gitignore needed for the busybox image fixture tar

I think it might already be excluded - I had to force add my test cases since they'd normally be excluded and must have accidentally picked up that as well

@westonsteimel westonsteimel force-pushed the rust-toolchain-cataloger branch 2 times, most recently from 731abd9 to 3090714 Compare February 22, 2023 20:02
Copy link
Contributor

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The /rust-1.67.1/toolchains/stable-x86_64-unknown-linux-musl/lib/libstd-86aefecbddda356d.so is still 1.25 MB, it would be nice to just copy the small (maybe < 100 bytes) binary bits that are required for matching. Other than that, 👍

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
@spiffcs
Copy link
Contributor

spiffcs commented Feb 27, 2023

@wagoodman I think all new fixture changes are now only in the <30kb size now:
Screenshot 2023-02-27 at 2 50 24 PM

Did you have other changes requested for this PR? Otherwise LGTM

@@ -196,4 +196,24 @@ var defaultClassifiers = []classifier{
Package: "postgresql",
PURL: mustPURL("pkg:generic/postgresql@version"),
},
{
Class: "rust-standard-library-linux",
FileGlob: "**/libstd-????????????????.so",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call, that is more specific and not too crazy for the index parsing in stereoscope 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants