bug: application should not use nvd statistics when nvd is disabled or nvd is using api2

[mulder999]

Description

Application attempts to download nvd statistics despite nvd being disabled or nvd configured to use api v2.

To reproduce

Run this command:
Steps to reproduce the behaviour:

1. On your FW, block access to nvd.nist.gov (or briefly saturate them with requests so that your IP gets a temporary ban)
2. Run cve-bin-tool --disable-data-source "NVD,OSV,GAD,CURL,REDHAT" --nvd api2 --nvd-api-key <key> --sbom-type cyclonedx --sbom-output application.sbom.json application.apk

Expected behaviour:
SBOM is generated without any report of vulnerabilities

Actual behaviour:
Application crashes with a ServerTimeoutError:

ServerTimeoutError: Connection timeout to host https://nvd.nist.gov/rest/public/dashboard/statistics
Exception ignored in: <function ClientSession.__del__ at 0x7f1ea369ee80>
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 367, in __del__
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1838, in call_exception_handler
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1568, in error
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1684, in _log
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1700, in handle
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1028, in handle
  File "/usr/local/lib/python3.12/site-packages/rich/logging.py", line 160, in emit
  File "/usr/local/lib/python3.12/site-packages/rich/logging.py", line 221, in render
  File "/usr/local/lib/python3.12/site-packages/rich/_log_render.py", line 43, in __call__
ImportError: sys.meta_path is None, Python is likely shutting down
Exception ignored in: <function ClientSession.__del__ at 0x7f1ea369ee80>
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 367, in __del__
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1838, in call_exception_handler
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1568, in error
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1684, in _log
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1700, in handle
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1762, in callHandlers
  File "/usr/local/lib/python3.12/logging/__init__.py", line 1028, in handle
  File "/usr/local/lib/python3.12/site-packages/rich/logging.py", line 160, in emit
  File "/usr/local/lib/python3.12/site-packages/rich/logging.py", line 221, in render
  File "/usr/local/lib/python3.12/site-packages/rich/_log_render.py", line 43, in __call__
ImportError: sys.meta_path is None, Python is likely shutting down

Version/platform info

Version of CVE-bin-tool: 3.2.1.

Platform is completly irrelevant, here is some docker config you might want to use as a startup point:

# File cvebintool/Dockerfile
FROM python

RUN pip install cve-bin-tool

# File docker-compose.yml
version: '3.9'

services:
  cvebintool:
    build: cvebintool
    image: local/cvebintool
    environment:
      - NVD_API_KEY=<key>
    command: cve-bin-tool --disable-data-source "NVD,OSV,GAD,CURL,REDHAT" --nvd api2 --sbom-type cyclonedx --sbom-output application.sbom.json application.apk
    working_dir: /data
    volumes:
      - ./data:/data

Run with mkdir -p ./data; touch ./data/application.apk; docker compose run --rm cvebintool.

REM: The application is crashing before reaching the apk file. Feel free to use some real binary file for a more realistic case.

[Mayankrai449]

I'm looking into this

[Mayankrai449]

@mulder999 Seems like I am not able to produce the same ServerTimeoutError as I am getting the expected behavior with SBOM generation without any report of vulnerabilities. At first, the problem seemed to be in error handling where If the NVD data source is disabled, the application should not attempt to download NVD statistics. But, despite blocking the access to nvd.nist.gov, the exception seems to be handled already. Do update me if I am missing something.

[terriko]

I don't think we ever built the code to disable NVD the way the other sources can be disabled. As in, I was surprised to discover that it's even being listed as an option because I remember having a conversation about how disabling it probably wouldn't work because of the database setup so we shouldn't make it an option. I'd have to do some digging to see when it was made an option and whether anyone actually plumbed it through or whether it was a mistake in the argparse setup.

That said, you're right that this is a bug. We either need to fix the disabling or accept that we shouldn't be providing it as an option. I think there have been enough changes that it should be possible to disable NVD correctly now but it may be harder than it is for the other data sources.

Short term workaround: try --offline mode for when you need NVD disabled. offline mode definitely will disable NVD.

[terriko]

@Mayankrai449 are you using an NVD_API_KEY in your testing and using API2? If you don't set a key, you're actually using the mirror and not talking to NVD directly which might be why you're not seeing the issue.

[terriko]

@mulder999 okay, I took a quick peek and there is no code for disabling NVD. In fact, it's explicitly added as a default source, so the opposite is happening.

If anyone's interested in working on this, take a look around line 695 in cli.py:

cve-bin-tool/cve_bin_tool/cli.py

Lines 695 to 700 in b4feb03

	if "CURL" not in disabled_sources:
	source_curl = curl_source.Curl_Source()
	enabled_sources.append(source_curl)
	default_sources = [source_nvd]
	default_sources.extend(enabled_sources)

You'll want to try checking if NVD is disabled and doing the right thing there similar to the other sources. It MAY break things in unexpected ways but it might just work smoothly; won't know until you try or you read through every single place the default_sources is used. @Mayankrai449 if you won't to take it from here, that's where I'd start.

[Mayankrai449]

@terriko I actually used my own requested NVD_API key. I will look into this bug and help with appropriate way of handling the disabling of NVD.

[Mayankrai449]

@terriko After modification, NVD will no longer be added as default_source if it is in disabled_sources list. Do update if more changes are needed, I'd like to work on it.

[mulder999]

@terriko in fact the offline mode turned out as a chicken-egg problem for me because it mandates a database and exports also presumes access to nvd. I believe an interesting feature request would be to be able to generate only the SBOM, ie have to possibility to optionnally disable completely the vulnerability analysis as this could be performed later through other central vulnerability management tools (eg: dependency track).

[terriko]

@mulder999 Do you mind if I ask: why use cve-bin-tool for generating sboms if you're not intending to use it for scanning? I'd like to understand the use case better. I had long assumed that people would use more established tools for SBOM generation -- are we doing something that isn't being provided by other tools that we should be making sure we keep doing?

[terriko]

This auto-closed because #3814 should provide the requested functionality, but I'm going to re-open it while we work on adding tests.

[mulder999]

@terriko Extracting a SBOM from a binary is an exceptionally valuable feature, in my opinion. I utilize a variety of tools and also endeavor to modestly gather and share knowledge around the OSS SBOM extraction tools that can be leveraged for various tasks. As of now, I am not aware of any other concurrent tool to work with binaries in the OSS community. Please keep it !

[mulder999]

I tested again with version 3.3, and it is still not possible to successfully disable NVD, making the whole tool virtually unusable unfortunately.

Also the tool calls countsbystatus which does not seems to respect NVD api best practices

Here is the stack trace:

│ /usr/local/bin/cve-bin-tool:8 in <module>                                                        │
│                                                                                                  │
│   5 from cve_bin_tool.cli import main                                                            │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(main())                                                                         │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/cli.py:808 in main                          │
│                                                                                                  │
│    805 │                                                                                         │
│    806 │   # update db if needed                                                                 │
│    807 │   if db_update != "never":                                                              │
│ ❱  808 │   │   cvedb_orig.get_cvelist_if_stale()                                                 │
│    809 │   else:                                                                                 │
│    810 │   │   LOGGER.warning("Not verifying CVE DB cache")                                      │
│    811 │   │   if not cvedb_orig.check_cve_entries():                                            │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/cvedb.py:281 in get_cvelist_if_stale        │
│                                                                                                  │
│    278 │   │   │   datetime.datetime.today()                                                     │
│    279 │   │   │   - datetime.datetime.fromtimestamp(self.dbpath.stat().st_mtime)                │
│    280 │   │   ) > datetime.timedelta(hours=24):                                                 │
│ ❱  281 │   │   │   self.refresh_cache_and_update_db()                                            │
│    282 │   │   │   self.time_of_last_update = datetime.datetime.today()                          │
│    283 │   │   else:                                                                             │
│    284 │   │   │   _ = self.get_db_update_date()                                                 │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/cvedb.py:264 in refresh_cache_and_update_db │
│                                                                                                  │
│    261 │   │   """Refresh cached NVD and update CVE database with latest data."""                │
│    262 │   │   self.LOGGER.debug("Updating CVE data. This will take a few minutes.")             │
│    263 │   │   # refresh the nvd cache                                                           │
│ ❱  264 │   │   run_coroutine(self.refresh())                                                     │
│    265 │   │                                                                                     │
│    266 │   │   # if the database isn't open, open it                                             │
│    267 │   │   self.init_database()                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/async_utils.py:90 in run_coroutine          │
│                                                                                                  │
│    87 │   """                                                                                    │
│    88 │   loop = get_event_loop()                                                                │
│    89 │   aws = asyncio.ensure_future(coro, loop=loop)                                           │
│ ❱  90 │   result = loop.run_until_complete(aws)                                                  │
│    91 │   return result                                                                          │
│    92                                                                                            │
│    93                                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.12/asyncio/base_events.py:687 in run_until_complete                       │
│                                                                                                  │
│    684 │   │   if not future.done():                                                             │
│    685 │   │   │   raise RuntimeError('Event loop stopped before Future completed.')             │
│    686 │   │                                                                                     │
│ ❱  687 │   │   return future.result()                                                            │
│    688 │                                                                                         │
│    689 │   def stop(self):                                                                       │
│    690 │   │   """Stop running the event loop.                                                   │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/cvedb.py:258 in refresh                     │
│                                                                                                  │
│    255 │   │   if self.version_check:                                                            │
│    256 │   │   │   check_latest_version()                                                        │
│    257 │   │                                                                                     │
│ ❱  258 │   │   await self.get_data()                                                             │
│    259 │                                                                                         │
│    260 │   def refresh_cache_and_update_db(self) -> None:                                        │
│    261 │   │   """Refresh cached NVD and update CVE database with latest data."""                │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/cvedb.py:379 in get_data                    │
│                                                                                                  │
│    376 │   │   │   if source is not None:                                                        │
│    377 │   │   │   │   tasks.append(source.get_cve_data())                                       │
│    378 │   │                                                                                     │
│ ❱  379 │   │   for r in await asyncio.gather(*tasks):                                            │
│    380 │   │   │   self.data.append(r)                                                           │
│    381 │                                                                                         │
│    382 │   def init_database(self) -> None:                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/data_sources/nvd_source.py:93 in            │
│ get_cve_data                                                                                     │
│                                                                                                  │
│    90 │                                                                                          │
│    91 │   async def get_cve_data(self):                                                          │
│    92 │   │   """Retrieves the CVE data from the data source."""                                 │
│ ❱  93 │   │   await self.fetch_cves()                                                            │
│    94 │   │                                                                                      │
│    95 │   │   if self.nvd_type == "api2":                                                        │
│    96 │   │   │   return self.format_data_api2(self.all_cve_entries), self.source_name           │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/data_sources/nvd_source.py:329 in           │
│ fetch_cves                                                                                       │
│                                                                                                  │
│   326 │   │   tasks = []                                                                         │
│   327 │   │   LOGGER.info("Getting NVD CVE data...")                                             │
│   328 │   │   if self.nvd_type == "api2":                                                        │
│ ❱ 329 │   │   │   self.all_cve_entries = await asyncio.create_task(                              │
│   330 │   │   │   │   self.nist_fetch_using_api(),                                               │
│   331 │   │   │   )                                                                              │
│   332 │   │   else:                                                                              │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/data_sources/nvd_source.py:391 in           │
│ nist_fetch_using_api                                                                             │
│                                                                                                  │
│   388 │   │   │   │   )                                                                          │
│   389 │   │   │   )                                                                              │
│   390 │   │   else:                                                                              │
│ ❱ 391 │   │   │   await nvd_api.get_nvd_params()                                                 │
│   392 │   │   await nvd_api.get()                                                                │
│   393 │   │   await nvd_api.session.close()                                                      │
│   394 │   │   nvd_api.session = None                                                             │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/nvd_api.py:143 in get_nvd_params            │
│                                                                                                  │
│   140 │   │   │   )                                                                              │
│   141 │   │                                                                                      │
│   142 │   │   self.logger.info("Fetching metadata from NVD...")                                  │
│ ❱ 143 │   │   cve_count = await self.nvd_count_metadata(self.session)                            │
│   144 │   │   self.logger.debug(f"NVD metadata {cve_count}")                                     │
│   145 │   │                                                                                      │
│   146 │   │   await self.validate_nvd_api()                                                      │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/cve_bin_tool/nvd_api.py:89 in nvd_count_metadata         │
│                                                                                                  │
│    86 │   │   │   "Rejected": 0,                                                                 │
│    87 │   │   │   "Received": 0,                                                                 │
│    88 │   │   }                                                                                  │
│ ❱  89 │   │   async with await session.get(                                                      │
│    90 │   │   │   NVD_CVE_STATUS,                                                                │
│    91 │   │   │   params={"reporttype": "countsbystatus"},                                       │
│    92 │   │   │   raise_for_status=True,                                                         │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/aiohttp/client.py:1197 in __aenter__                     │
│                                                                                                  │
│   1194 │   │   return self.__await__()                                                           │
│   1195 │                                                                                         │
│   1196 │   async def __aenter__(self) -> _RetType:                                               │
│ ❱ 1197 │   │   self._resp = await self._coro                                                     │
│   1198 │   │   return self._resp                                                                 │
│   1199                                                                                           │
│   1200                                                                                           │
│                                                                                                  │
│ /usr/local/lib/python3.12/site-packages/aiohttp/client.py:585 in _request                        │
│                                                                                                  │
│    582 │   │   │   │   │   │   │   │   req, traces=traces, timeout=real_timeout                  │
│    583 │   │   │   │   │   │   │   )                                                             │
│    584 │   │   │   │   │   except asyncio.TimeoutError as exc:                                   │
│ ❱  585 │   │   │   │   │   │   raise ServerTimeoutError(                                         │
│    586 │   │   │   │   │   │   │   "Connection timeout " "to host {}".format(url)                │
│    587 │   │   │   │   │   │   ) from exc                                                        │
│    588                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ServerTimeoutError: Connection timeout to host https://nvd.nist.gov/rest/public/dashboard/statistics

[terriko]

Thanks for the updated report. Sounds like we've got more work to do here.

[terriko]

Oh, and while we're definitely intending to fix things so NVD can be properly disabled, I've also started a brainstorming thread on what a more complete no-scan mode would need (e.g. making sure we don't print a bunch of "this has no cves" reports if no data sources were enabled, that sort of thing). If you've got any wishlist items you'd like to stick in there, we'd love to know about them!

• link to no-scan mode issue: feat: no-scan mode #4110