Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce log spam on unknown relationship type #1797

Merged
merged 5 commits into from
May 10, 2023
Merged

Conversation

willmurphyscode
Copy link
Contributor

An unknown relationship type doesn't mean things are incorrect; it probably means that the version of syft that generated the SBOM is ahead of the version of syft that's parsing it.

Addresses #1812, but won't fix it until the version of syft that's imported by grype is updated, at https://github.com/anchore/grype/blob/75e7ef43cd03baba4c76af3751274c7aa7452f36/go.mod#L56.

An unknown relationship type doesn't mean things are incorrect; it
probably means that the version of syft that generated the SBOM is ahead
of the version of syft that's parsing it.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Knowing that the event-type was unexpected seems like useful information
while debugging.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
@github-actions
Copy link

github-actions bot commented May 8, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz%0A                                                          │ ./.tmp/benchmark-85096ca.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   11.90m ±  1%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    572.1µ ±  1%25%0AImagePackageCatalogers/binary-cataloger-2                                   210.2µ ±  1%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   559.3µ ±  2%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.233m ±  2%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         94.83µ ±  2%25%0AImagePackageCatalogers/java-cataloger-2                                     13.29m ± 21%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     95.41µ ±  8%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       371.8µ ±  2%25%0AImagePackageCatalogers/nix-store-cataloger-2                                252.4µ ±  3%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   724.3µ ±  0%25%0AImagePackageCatalogers/portage-cataloger-2                                  340.7µ ±  1%25%0AImagePackageCatalogers/python-package-cataloger-2                           3.077m ±  1%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   469.6µ ±  1%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             840.5µ ±  0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     115.9µ ±  1%25%0Ageomean                                                                     612.0µ%0A%0A                                                          │ ./.tmp/benchmark-85096ca.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.117Mi ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    146.3Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   32.04Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   170.5Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              409.4Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         10.06Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.787Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.750Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       98.70Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                49.28Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   179.4Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  86.39Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           978.4Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   178.4Ki ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             139.8Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.21Ki ± 0%25%0Ageomean                                                                     134.3Ki%0A%0A                                                          │ ./.tmp/benchmark-85096ca.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    87.63k ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     3.683k ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     896.0 ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    2.998k ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.326k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.46k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.322k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  890.0 ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    3.796k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   1.668k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            15.96k ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.879k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.279k ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0Ageomean                                                                      2.645k

@willmurphyscode
Copy link
Contributor Author

Discussed this with @kzantow and @wagoodman, and it might be better to still warn about this, but warn only once, rather than once per lost relationship. I will send a revision.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Signed-off-by: Will Murphy <will.murphy@anchore.com>
@willmurphyscode willmurphyscode changed the title Lower log level on unknown relationship type Reduce log spam on unknown relationship type May 9, 2023
Signed-off-by: Will Murphy <will.murphy@anchore.com>
Copy link
Contributor

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 nicely done! -- just a small formatting suggestion

errorCounts[e.Error()] = errorCounts[e.Error()] + 1
}
for msg, count := range errorCounts {
errorMessages = append(errorMessages, fmt.Sprintf("%q occurred %d time(s)", msg, count))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion:

if count > 1 {
  errorMessages = append(errorMessages, fmt.Sprintf("%s (occurred %d times)", msg, count))
} else {
  errorMessages = append(errorMessages, msg)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion - I think I'll leave it as is for now, since I like that the whole block of de-duplicated errors in the logs all look the same, but I'd be open to changing this in a follow-up if it seems important.

Copy link
Contributor

@wagoodman wagoodman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great 👍

@willmurphyscode willmurphyscode merged commit 291da8c into main May 10, 2023
9 checks passed
@willmurphyscode willmurphyscode deleted the lower-log-level branch May 10, 2023 13:51
spiffcs pushed a commit that referenced this pull request May 10, 2023
Rather than log a warning for every instance of an unknown relationship type,
or similar error, log a count of how many times each of these errors is
raised.

---------

Signed-off-by: Will Murphy <will.murphy@anchore.com>
spiffcs added a commit that referenced this pull request May 10, 2023
* main:
  fix: Reduce log spam on unknown relationship type (#1797)
  chore(deps): update bootstrap tools to latest versions (#1807)
  chore(deps): bump golang.org/x/net from 0.9.0 to 0.10.0 (#1802)
  chore(deps): bump github.com/docker/docker (#1795)
  chore(deps): bump github.com/google/go-containerregistry (#1796)
  chore(deps): update bootstrap tools to latest versions (#1792)
  Print package list when extra packages found (#1791)
  chore(deps): update bootstrap tools to latest versions (#1786)
  chore(deps): bump golang.org/x/term from 0.7.0 to 0.8.0 (#1787)
  Update the CPE generation for spring-security-core (#1789)
  chore: do not HTML escape PackageURLs (#1782)
  chore: do not include kernel module cataloger by default (#1784)

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
spiffcs added a commit that referenced this pull request May 18, 2023
* main: (32 commits)
  chore(deps): bump github.com/google/go-containerregistry (#1823)
  chore(deps): bump github.com/sirupsen/logrus from 1.9.0 to 1.9.1 (#1822)
  chore(deps): bump github.com/docker/docker (#1824)
  fix: update field plurality of 8.0.0 schema before release (#1820)
  fix: update cataloger to check for expressions before split (#1819)
  feat: update syft license concept to complex struct (#1743)
  fix: cyclonedx depends-on relationship inverted (#1816)
  fix: retain sbom cataloger relationships (#1509)
  feat: warn if parsing newer SBOM (#1810)
  feat: Add R cataloger (#1790)
  update cosign to v2 release (different go module) (#1805)
  fix: Reduce log spam on unknown relationship type (#1797)
  chore(deps): update bootstrap tools to latest versions (#1807)
  chore(deps): bump golang.org/x/net from 0.9.0 to 0.10.0 (#1802)
  chore(deps): bump github.com/docker/docker (#1795)
  chore(deps): bump github.com/google/go-containerregistry (#1796)
  chore(deps): update bootstrap tools to latest versions (#1792)
  Print package list when extra packages found (#1791)
  chore(deps): update bootstrap tools to latest versions (#1786)
  chore(deps): bump golang.org/x/term from 0.7.0 to 0.8.0 (#1787)
  ...

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Rather than log a warning for every instance of an unknown relationship type,
or similar error, log a count of how many times each of these errors is
raised.

---------

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spurious warnings - unknown relationship type: evident-by form-lib=syft
3 participants