Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update cache.fingerprint file to java-builds dir #1748

Merged
merged 1 commit into from
Apr 19, 2023

Conversation

spiffcs
Copy link
Contributor

@spiffcs spiffcs commented Apr 18, 2023

Summary

The validations workflow was looking under the packages directory for a fingerprint file that was not being created correctly. This caused tests to fail on PR that used the older cache after this update was made:

This PR syncs the Makefile for java-build fixtures with patterns that exist already in the codebase. It also links these changes into the main Makefile so that CI can correctly validate when the cache needs to be busted.

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@spiffcs spiffcs marked this pull request as ready for review April 18, 2023 17:08
@github-actions
Copy link

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz%0A                                                          │ ./.tmp/benchmark-3c13e48.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   11.60m ±  1%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             894.8µ ±  1%25%0AImagePackageCatalogers/python-package-cataloger-2                           3.125m ± 21%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   775.8µ ±  2%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       411.6µ ±  3%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   571.3µ ±  3%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   503.3µ ±  2%25%0AImagePackageCatalogers/java-cataloger-2                                     12.81m ±  2%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     94.83µ ±  3%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    622.1µ ±  2%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         95.88µ ±  3%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.244m ±  4%25%0AImagePackageCatalogers/portage-cataloger-2                                  372.7µ ±  1%25%0AImagePackageCatalogers/nix-store-cataloger-2                                271.2µ ±  4%25%0AImagePackageCatalogers/sbom-cataloger-2                                     118.2µ ±  1%25%0AImagePackageCatalogers/binary-cataloger-2                                   211.9µ ±  1%25%0AImagePackageCatalogers/linux-kernel-cataloger-2                             42.59m ±  9%25%0Ageomean                                                                     811.9µ%0A%0A                                                          │ ./.tmp/benchmark-3c13e48.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.069Mi ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             140.4Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           984.3Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   180.0Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       98.61Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   169.9Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   178.2Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.785Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.750Ki ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    145.1Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         10.06Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              409.8Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  85.88Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                48.91Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.20Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   31.95Ki ± 0%25%0AImagePackageCatalogers/linux-kernel-cataloger-2                             62.65Mi ± 0%25%0Ageomean                                                                     192.8Ki%0A%0A                                                          │ ./.tmp/benchmark-3c13e48.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    86.83k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.280k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            15.95k ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    3.797k ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.321k ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    2.989k ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.876k ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.46k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     3.666k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.326k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   1.660k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  884.0 ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     896.0 ± 0%25%0AImagePackageCatalogers/linux-kernel-cataloger-2                              2.798k ± 0%25%0Ageomean                                                                      2.649k

@spiffcs spiffcs requested a review from wagoodman April 18, 2023 17:34
@spiffcs spiffcs merged commit 55a90a2 into main Apr 19, 2023
9 checks passed
@spiffcs spiffcs deleted the bust-java-test-fixture-cache branch April 19, 2023 16:17
@@ -36,8 +36,8 @@ jobs:
- name: Restore Java test-fixture cache
uses: actions/cache@v3
with:
path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages
key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }}
path: syft/pkg/cataloger/java/test-fixtures/java-builds
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this cache would overwrite what's checked into git (all other locations under java-builds that is outside of packages)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ My fault - I thought this is what we wanted where we were checking for any changes under java-builds and rebuilding on changes to those files - thinking more segmenting out the packages folder is the right choice since it keeps the cache smaller, we probably just want to update the makefile to also key off of rebuilding if there are changes to any of the shell scripts *.sh

find example-* build-* Makefile -type f -exec sha256sum {} \; | sort | tee /dev/stderr | tee $(PKGSDIR).fingerprint
sha256sum $(PKGSDIR).fingerprint
.PHONY: cache.fingerprint
cache.fingerprint:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this fundamentally changed anything -- that is, I think this PR now passes since the fingerprint has changed, but ultimately the cache won't be automatically busted when there is a source change.

spiffcs added a commit that referenced this pull request Apr 24, 2023
* main:
  Add sections of interest for Gemfile.lock cataloger (#1749)
  fix: update cache.fingerprint file to java-builds dir (#1748)
  Add ALPM Metadata to CYCLONEDX and SPDX output formats (#1747)
  chore: bump stereoscope to latest version (#1741)
  chore(deps): update bootstrap tools to latest versions (#1744)
  chore(deps): bump github.com/docker/docker (#1746)
  Create consul binary classifier (#1738)
  chore(deps): update bootstrap tools to latest versions (#1740)

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants