Placeholder: currently failing tests after #2421 merge #2430
Description
Merging #2421 was expected to cause some test failures at least until the cache updated. I'm going to re-run the tests shortly now that the cache should be updated, but I'm making myself a place holder with the tests that were failing after merge in case I need that data later for comparison (both because github clears the logs after a few days and because the interface is just not great for comparing this kind of data)
Linux Long tests:
=========================== short test summary info ============================
FAILED test/test_cli.py::TestCLI::test_update_flags - AssertionError: assert 0 != 0
+ where 0 = main(['cve-bin-tool', '-x', '-u', 'now', '-n', 'json', ...])
FAILED test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.toml] - AssertionError: assert 0 != 0
+ where 0 = main(['cve-bin-tool', '-C', '/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.toml', '-l', 'info'])
FAILED test/test_cli.py::TestCLI::test_config_file[/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.yaml] - AssertionError: assert 0 != 0
+ where 0 = main(['cve-bin-tool', '-C', '/home/runner/work/cve-bin-tool/cve-bin-tool/test/config/cve_bin_tool_config.yaml', '-l', 'info'])
FAILED test/test_cli.py::TestCLI::test_severity - FileNotFoundError: [Errno 2] No such file or directory: 'sevtest.csv'
FAILED test/test_cli.py::TestCLI::test_CVSS_score - FileNotFoundError: [Errno 2] No such file or directory: 'sevtest.csv'
FAILED test/test_cli.py::TestCLI::test_SBOM - AssertionError: assert ('cve_bin_tool', 20, 'There are 1 products with known CVEs detected') in [('cve_bin_tool', 20, 'CVE Binary Tool v3.2rc0'), ('cve_bin_tool', 20, 'This product uses the NVD API but is not endorsed or certified by the NVD.'), ('cve_bin_tool.CVEDB', 20, 'Using cached CVE data (<24h old). Use -u now to update immediately.'), ('cve_bin_tool.CVEDB', 20, 'There are 190938 CVE entries in the database'), ('cve_bin_tool.CVEDB', 20, 'There are 189973 CVE entries from NVD in the database'), ('cve_bin_tool.CVEDB', 20, 'There are 743 CVE entries from GAD in the database'), ...]
+ where [('cve_bin_tool', 20, 'CVE Binary Tool v3.2rc0'), ('cve_bin_tool', 20, 'This product uses the NVD API but is not endorsed or certified by the NVD.'), ('cve_bin_tool.CVEDB', 20, 'Using cached CVE data (<24h old). Use -u now to update immediately.'), ('cve_bin_tool.CVEDB', 20, 'There are 190938 CVE entries in the database'), ('cve_bin_tool.CVEDB', 20, 'There are 189973 CVE entries from NVD in the database'), ('cve_bin_tool.CVEDB', 20, 'There are 743 CVE entries from GAD in the database'), ...] = <_pytest.logging.LogCaptureFixture object at 0x7fad6d7f3ca0>.record_tuples
======= 6 failed, 14 passed, 2 skipped, 24 warnings in 487.47s (0:08:07) =======Windows long tests (short tests also failed but they look like the same fails)
=========================== short test summary info ===========================
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\go.mod-products5] - AssertionError: assert 'json-patch' in ['api', 'go']
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\Package.resolved-products6] - AssertionError: assert 'alliance_web_platform' in []
FAILED test/test_language_scanner.py::TestLanguageScanner::test_python_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\PKG-INFO] - UnboundLocalError: local variable 'product_info' referenced before assignment
FAILED test/test_package_list_parser.py::TestPackageListParser::test_valid_requirements[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\txt\\test_requirements.txt-parsed_data0] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 3 more items:
{ProductInfo(vendor='html5lib*', product='html5lib', version='0.999999999'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}},
ProductInfo(vendor='httplib2_project*', product='httplib2', version='0.18.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}},
ProductInfo(vendor='python*', product='requests', version='2.25.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='html5lib*', product='html5lib', version='0.999999999'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- ProductInfo(vendor='httplib2_project*', product='httplib2', version='0.18.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- ProductInfo(vendor='python*', product='requests', version='2.25.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_spdx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\spdx_test.spdx-spdx_parsed_data0] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_spdx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\spdx_test.spdx.rdf-spdx_parsed_data1] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_spdx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\spdx_test.spdx.json-spdx_parsed_data2] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_spdx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\spdx_test.spdx.xml-spdx_parsed_data3] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_spdx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\spdx_test.spdx.yml-spdx_parsed_data4] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_spdx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\spdx_test.spdx.yaml-spdx_parsed_data5] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_cyclonedx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\cyclonedx_test.xml-cyclonedx_parsed_data0] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_cyclonedx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\cyclonedx_test.json-cyclonedx_parsed_data1] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_cyclonedx_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\cyclonedx_test2.json-cyclonedx_parsed_data2] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='ubuntu', product='ubuntu', version='22.04'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='ubuntu', product='ubuntu', version='22.04'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_sbom.py::TestSBOM::test_valid_swid_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\sbom\\swid_test.xml-swid_parsed_data0] - AssertionError: assert defaultdict(<...s 'dict'>, {}) == {ProductInfo(...paths': {''}}}
Right contains 1 more item:
{ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
'remarks': <Remarks.NewFound: 1>,
'severity': ''},
'paths': {''}}}
Full diff:
+ defaultdict(<class 'dict'>, {},
+ )
- {
- ProductInfo(vendor='gnu', product='glibc', version='2.11.1'): {'default': {'comments': '',
- 'remarks': <Remarks.NewFound: 1>,
- 'severity': ''},
- 'paths': {''}},
- }
FAILED test/test_csv2cve.py::TestCSV2CVE::test_csv2cve_valid_file - AssertionError: assert ('cve_bin_tool', 20, 'There are 3 products with known CVEs detected') in [('cve_bin_tool', 20, 'CVE Binary Tool v3.2rc0'), ('cve_bin_tool', 20, 'This product uses the NVD API but is not endorsed or certified by the NVD.'), ('cve_bin_tool', 30, '\n **********************************************\n Warning: this utility was developed for Linux.\n You may need to install additional utilities\n to use it on other operating systems.\n **********************************************\n '), ('cve_bin_tool.CVEDB', 20, 'Using cached CVE data (<24h old). Use -u now to update immediately.'), ('cve_bin_tool.CVEDB', 20, 'There are 191014 CVE entries in the database'), ('cve_bin_tool.CVEDB', 20, 'There are 188725 CVE entries from NVD in the database'), ...]
+ where [('cve_bin_tool', 20, 'CVE Binary Tool v3.2rc0'), ('cve_bin_tool', 20, 'This product uses the NVD API but is not endorsed or certified by the NVD.'), ('cve_bin_tool', 30, '\n **********************************************\n Warning: this utility was developed for Linux.\n You may need to install additional utilities\n to use it on other operating systems.\n **********************************************\n '), ('cve_bin_tool.CVEDB', 20, 'Using cached CVE data (<24h old). Use -u now to update immediately.'), ('cve_bin_tool.CVEDB', 20, 'There are 191014 CVE entries in the database'), ('cve_bin_tool.CVEDB', 20, 'There are 188725 CVE entries from NVD in the database'), ...] = <_pytest.logging.LogCaptureFixture object at 0x000001D8121F05B0>.record_tuples
FAILED test/test_exploits.py::TestExploitScanner::test_exploit_checker[True-exploits_list0-product_info0-triage_info0-CRITICAL-EXPLOIT] - IndexError: list index out of range
FAILED test/test_exploits.py::TestExploitScanner::test_exploit_checker[False-exploits_list1-product_info1-triage_info1-CRITICAL] - IndexError: list index out of range
FAILED test/test_exploits.py::TestExploitScanner::test_exploit_checker[True-exploits_list2-product_info2-triage_info2-CRITICAL] - IndexError: list index out of range
FAILED test/test_exploits.py::TestExploitScanner::test_exploit_checker[False-exploits_list3-product_info3-triage_info3-CRITICAL] - IndexError: list index out of range
FAILED test/test_input_engine.py::TestInputEngine::test_vex_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\vex\\test_triage.vex-parsed_data0] - AssertionError: assert {ProductInfo(...paths': {''}}} == {ProductInfo(...paths': {''}}}
Left contains 2 more items:
{ProductInfo(vendor='UNKNOWN', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
'remarks': <Remarks.Unexplored: 2>,
'severity': 'HIGH'},
'paths': {''}},
ProductInfo(vendor='UNKNOWN', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
'priority '
'need '
'to '
'resolve '
'fast',
'remarks': <Remarks.Confirmed: 3>,
'severity': 'CRITICAL'},
'paths': {''}}}
Right contains 2 more items:
{ProductInfo(vendor='gentoo', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
'remarks': <Remarks.Unexplored: 2>,
'severity': 'HIGH'},
'paths': {''}},
ProductInfo(vendor='libjpeg-turbo', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
'priority '
'need '
'to '
'resolve '
'fast',
'remarks': <Remarks.Confirmed: 3>,
'severity': 'CRITICAL'},
'paths': {''}}}
Full diff:
{
- ProductInfo(vendor='gentoo', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
? ^^^^^^
+ ProductInfo(vendor='UNKNOWN', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
? ^^^^^^^
- 'remarks': <Remarks.Unexplored: 2>,
+ 'remarks': <Remarks.Unexplored: 2>,
? +
- 'severity': 'HIGH'},
+ 'severity': 'HIGH'},
? +
- 'paths': {''}},
+ 'paths': {''}},
? +
- ProductInfo(vendor='libjpeg-turbo', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
? ^^^^^^^^^^^^^
+ ProductInfo(vendor='UNKNOWN', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
? ^^^^^^^
- 'priority '
? ------
+ 'priority '
- 'need '
? ------
+ 'need '
- 'to '
? ------
+ 'to '
- 'resolve '
? ------
+ 'resolve '
- 'fast',
? ------
+ 'fast',
- 'remarks': <Remarks.Confirmed: 3>,
? ------
+ 'remarks': <Remarks.Confirmed: 3>,
- 'severity': 'CRITICAL'},
? ------
+ 'severity': 'CRITICAL'},
- 'paths': {''}},
? ------
+ 'paths': {''}},
}
FAILED test/test_input_engine.py::TestInputEngine::test_vex_file[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\vex\\test_triage_cyclonedx.vex-parsed_data1] - AssertionError: assert {ProductInfo(...paths': {''}}} == {ProductInfo(...paths': {''}}}
Left contains 2 more items:
{ProductInfo(vendor='UNKNOWN', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
'remarks': <Remarks.Unexplored: 2>,
'severity': 'HIGH'},
'paths': {''}},
ProductInfo(vendor='UNKNOWN', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
'priority '
'need '
'to '
'resolve '
'fast',
'remarks': <Remarks.Confirmed: 3>,
'severity': 'CRITICAL'},
'paths': {''}}}
Right contains 2 more items:
{ProductInfo(vendor='gentoo', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
'remarks': <Remarks.Unexplored: 2>,
'severity': 'HIGH'},
'paths': {''}},
ProductInfo(vendor='libjpeg-turbo', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
'priority '
'need '
'to '
'resolve '
'fast',
'remarks': <Remarks.Confirmed: 3>,
'severity': 'CRITICAL'},
'paths': {''}}}
Full diff:
{
- ProductInfo(vendor='gentoo', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
? ^^^^^^
+ ProductInfo(vendor='UNKNOWN', product='glibc', version='2.33'): {'CVE-2021-1234': {'comments': '',
? ^^^^^^^
- 'remarks': <Remarks.Unexplored: 2>,
+ 'remarks': <Remarks.Unexplored: 2>,
? +
- 'severity': 'HIGH'},
+ 'severity': 'HIGH'},
? +
- 'paths': {''}},
+ 'paths': {''}},
? +
- ProductInfo(vendor='libjpeg-turbo', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
? ^^^^^^^^^^^^^
+ ProductInfo(vendor='UNKNOWN', product='libjpeg-turbo', version='2.0.1'): {'CVE-2018-19664': {'comments': 'High '
? ^^^^^^^
- 'priority '
? ------
+ 'priority '
- 'need '
? ------
+ 'need '
- 'to '
? ------
+ 'to '
- 'resolve '
? ------
+ 'resolve '
- 'fast',
? ------
+ 'fast',
- 'remarks': <Remarks.Confirmed: 3>,
? ------
+ 'remarks': <Remarks.Confirmed: 3>,
- 'severity': 'CRITICAL'},
? ------
+ 'severity': 'CRITICAL'},
- 'paths': {''}},
? ------
+ 'paths': {''}},
}
FAILED test/test_language_scanner.py::TestLanguageScanner::test_java_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\pom.xml-commons_io] - UnboundLocalError: local variable 'product_info' referenced before assignment
FAILED test/test_language_scanner.py::TestLanguageScanner::test_javascript_package_none_found[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\fail-package-lock.json] - assert None is not None
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\renv.lock-products0] - AssertionError: assert 'cli' in ['mime']
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\Cargo.lock-products1] - AssertionError: assert 'bumpalo' in []
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\Gemfile.lock-products2] - AssertionError: assert 'addressable' in ['builder', 'curses', 'debug', 'nokogiri', 'pg', 'redcarpet', ...]
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\requirements.txt-products3] - AssertionError: assert 'plotly' in []
FAILED test/test_language_scanner.py::TestLanguageScanner::test_language_package[D:\\a\\cve-bin-tool\\cve-bin-tool\\test\\language_data\\package-lock.json-products4] - AssertionError: assert 'cache' in []
FAILED test/test_triage.py::test_triage - AssertionError: assert 1 >= 2
+ where 1 = len([{'comments': '', 'cve_number': 'GMS-2016-69', 'cvss_vector': 'unknown', 'cvss_version': 'unknown', ...}])
= 29 failed, 1333 passed, 33 skipped, 21 xpassed, 26 warnings in 1269.85s (0:21:09) =