Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acceptances tests not indicating failures #156

Closed
wagoodman opened this issue Aug 24, 2020 · 0 comments · Fixed by #158
Closed

Acceptances tests not indicating failures #156

wagoodman opened this issue Aug 24, 2020 · 0 comments · Fixed by #158
Assignees
Labels
bug Something isn't working

Comments

@wagoodman
Copy link
Contributor

Running make acceptance-linux results in:

+ mkdir -p ./.tmp/results
+ cp /tmp/syft-acceptance-test-rpm-HGwqJC/acceptance-rpm-centos-8.2.2004.json ./.tmp/results
+ ./test/acceptance/compare.py ./test/acceptance/test-fixtures/acceptance-centos-8.2.2004.json /tmp/syft-acceptance-test-rpm-HGwqJC/acceptance-rpm-centos-8.2.2004.json
+ tee ./.tmp/results/acceptance-rpm.txt
Traceback (most recent call last):

Comparing two Syft reports...

  File "./test/acceptance/compare.py", line 108, in <module>
    rc = main(sys.argv[1], sys.argv[2])
  File "./test/acceptance/compare.py", line 41, in main
    report2_packages, report2_metadata = report2_obj.packages()
  File "./test/acceptance/compare.py", line 31, in packages
    metadata=repr(entry.get("metadata", "")), sources=repr(entry["sources"])
KeyError: 'sources'
+ cleanup
+ rm -rf /tmp/syft-acceptance-test-rpm-HGwqJC

However, the exit code is 0 (but should be 1). Two fixes are needed:

  1. the compare script should be updated to support the new JSON format
  2. the make target should fail on bad comparisons
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant