Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract go module versions from ldflags for binaries built by go #1832

Merged
merged 5 commits into from
May 23, 2023

Conversation

wagoodman
Copy link
Contributor

This PR brings @westonsteimel s branch across the finish line -- adding pattern matching against ldflags for the main module package for the go binary cataloger. This allows us to extract from:

build	-ldflags="-s -w -X main.version=v2.7.1 -X main.commit=407fa622e9 -X main.date=2023-04-28T13:24:27Z -linkmode=external

a more useful version value:

v2.7.1

allowing us to replace the known useless value of (devel) for a version with v2.7.1. 🎉 !

There is notable overlap in the binary cataloger, so this PR additionally removes classifiers that are against go binaries that set their versions via the ldflags approach and have been demonstrated to work with the versions that were already captured as binary cataloger tests (these have been added as gobinary cataloger tests now).

There is some room for improvement with this approach... in a future PR we could try more aggressive patterns if the first set of patterns find nothing.

Closes #1785

westonsteimel and others added 5 commits March 10, 2023 10:43
Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
…sions-from-known-build-flags

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
@wagoodman wagoodman added the enhancement New feature or request label May 19, 2023
@wagoodman wagoodman requested a review from a team May 19, 2023 21:31
@wagoodman wagoodman self-assigned this May 19, 2023
@wagoodman wagoodman changed the title Extract go binary versions from known build flags Extract go module versions from ldflags for binaries built by go May 19, 2023
@github-actions
Copy link

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz%0A                                                          │ ./.tmp/benchmark-c6adf9b.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   15.22m ±  6%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    964.6µ ±  4%25%0AImagePackageCatalogers/binary-cataloger-2                                   259.7µ ±  3%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   811.1µ ±  4%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.618m ±  6%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         144.7µ ±  3%25%0AImagePackageCatalogers/java-cataloger-2                                     16.79m ±  3%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     138.0µ ± 18%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       580.3µ ±  4%25%0AImagePackageCatalogers/nix-store-cataloger-2                                390.1µ ±  9%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   1.069m ±  5%25%0AImagePackageCatalogers/portage-cataloger-2                                  667.0µ ±  7%25%0AImagePackageCatalogers/python-package-cataloger-2                           4.300m ±  2%25%0AImagePackageCatalogers/r-package-cataloger-2                                316.8µ ±  6%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   766.5µ ±  3%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             1.271m ±  3%25%0AImagePackageCatalogers/sbom-cataloger-2                                     157.3µ ±  7%25%0Ageomean                                                                     844.6µ%0A%0A                                                          │ ./.tmp/benchmark-c6adf9b.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.126Mi ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    205.2Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   30.17Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   168.8Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              404.9Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         9.906Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.825Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.594Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       100.9Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                49.14Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   186.6Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  119.9Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           1.003Mi ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                53.28Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   180.9Ki ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             144.1Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.20Ki ± 0%25%0Ageomean                                                                     132.7Ki%0A%0A                                                          │ ./.tmp/benchmark-c6adf9b.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    87.75k ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     4.180k ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     830.0 ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    3.000k ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.338k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.81k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.404k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  895.0 ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    4.079k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   2.267k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            16.44k ± 0%25%0AImagePackageCatalogers/r-package-cataloger-2                                  928.0 ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.989k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.447k ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0Ageomean                                                                      2.582k

@wagoodman wagoodman removed the enhancement New feature or request label May 22, 2023
@@ -113,36 +94,6 @@ classifiers/dynamic/helm-3.10.3:
/usr/local/bin/helm \
$@/helm

classifiers/dynamic/kubectl-1.24.11:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice - glad to see all of these specific cases removed in favor of the more generic solution.

@wagoodman wagoodman merged commit 26c201f into main May 23, 2023
9 checks passed
@wagoodman wagoodman deleted the extract-go-binary-versions-from-known-build-flags branch May 23, 2023 14:27
spiffcs added a commit that referenced this pull request Jun 5, 2023
* main: (21 commits)
  chore(deps): bump github.com/sirupsen/logrus from 1.9.2 to 1.9.3 (#1862)
  chore(deps): bump modernc.org/sqlite from 1.22.1 to 1.23.0 (#1863)
  feat: source-version flag (#1859)
  chore(deps): bump github.com/spf13/viper from 1.15.0 to 1.16.0 (#1851)
  accept main.version ldflags even without vcs (#1855)
  feat: add scope to pom properties (#1779)
  chore(deps): bump github.com/stretchr/testify from 1.8.3 to 1.8.4 (#1852)
  chore(deps): bump github.com/docker/docker (#1849)
  Add test to ensure package metadata is represented in the JSON schema (#1841)
  Fix directory resolver to consider CWD and root path input correctly (#1840)
  Migrate location-related structs to the file package (#1751)
  chore(deps): bump github.com/go-git/go-git/v5 from 5.6.1 to 5.7.0 (#1843)
  fix: add panic recovery for license parse (#1839)
  chore: return both failures when failed to retrieve an image with a scheme (#1801)
  Extract go module versions from ldflags for binaries built by go (#1832)
  fix: duplicate packages, support pnpm lockfile v6 (#1778)
  chore(deps): update stereoscope to e14bc4437b2eac481c5b6f101890b22df4f33596 (#1834)
  chore(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#1829)
  chore(deps): bump github.com/docker/docker (#1833)
  Keep original FileInfo persisted on file.Metadata structs (#1794)
  ...

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
…hore#1832)

* wip

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>

* with golang bin ldflags refactor

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* add test for golang binary cataloger for ldflag extraction

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* remove binary classfiers that overlap with new go ldflags detection

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

---------

Signed-off-by: Weston Steimel <weston.steimel@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Weston Steimel <weston.steimel@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Attempt to extract go main module versions from available ldflags
3 participants