Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add inline-comparison as acceptance test #106

Merged
merged 5 commits into from
Aug 10, 2020

Conversation

wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Aug 9, 2020

  • Updated the test/inline-compare python script to function with the latest JSON output
  • Added inline-compare step to the acceptance test pipeline, which runs multiple images (currently "debian:10.5", "centos:8.2.2004", "alpine:3.12.0", a vulnerable java image, vulnerable python image, and vulnerable alpine image ) against both Anchore engine inline-scan against syft and ensure a lower threshold in discovered vulnerabilities is met.
  • Updates the RPM matcher to search for vulnerabilities against the sourcerpm (if there is one)

Note: there are two discrepancies found that lower the test result values and I've added comments that explain these discrepancies. I've also added a upper-threshold for these exceptions such that when future enhancements are put in place the test will fail as an indication that the lower threshold should be bumped.

Example run: https://github.com/anchore/grype/runs/963498059?check_suite_focus=true

Note: go.mod needs to be updated once anchore/syft#130 is merged.

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
grype/matcher/rpmdb/matcher.go Show resolved Hide resolved
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
@wagoodman wagoodman merged commit 56b9576 into main Aug 10, 2020
@wagoodman wagoodman deleted the add-comparison-acceptance-tests branch August 10, 2020 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants