Investigate vulnerability scanning on windows and macos

[michalnicp]

Goal

Investigate vulnerability scanning on windows and MacOS hosts, including false positive/negative rate.

How?

• Install software listed in Measure false negatives  #5691. Use older versions that have known vulnerabilities
• Evaluate whether Improve detection of CPEs for macOS apps #5471 would be a viable option, how it would affect vulnerability detection on MacOS and the rate of false positives/negatives. False positives contribute to "noise".
• Evaluate potential alternatives for vulnerability scanning. Should we continue to use the NVD CVE feed? How can we improve?

Results

To analyze the cpe matching strategy for MacOS apps:

• install MacOS 12.4 inside a VirtualBox VM

• enroll the host with fleet

• install some software on it.

• dump the software from fleet to a csv (using mycli). Let's say the MacOS host has a host ID 8.

  \T csv; \o software.csv; select s.name, s.version, s.source from software s join host_software hs on hs.software_id = s.id where hs.host_id = 8; \T ascii
  

• load the software into /tmp/vulndb/cpe.sqlite

sqlite3 /tmp/vulndbs/cpe.sqlite
sqlite> create table software (name, version, source)
sqlite> .mode csv
sqlite> .import software.csv software

• Run a query to determine matches. This is similar to the logic used to match cpes in server/vulnerabilities/cpe.go

select
    cpe.title,
    cpe.cpe23,
    cpe.version,
    cpe.target_sw,
    software.name,
    software.version
from
    cpe_search
    join cpe on cpe.rowid = cpe_search.rowid
    join software on cpe_search.title match software.name
where
    cpe.version = software.version;

Summary:

• target_sw is frequently empty. Matching on it leads to many false negatives. However, omitting it would lead to too many false positives because many of the built-in apps eg Notes, Computer are very generic and match too easily on the title.

title	cpe23	version	target_sw	name	version
AirDrop Project AirDrop 1.0 for Android	cpe:2.3:a:airdrop_project:airdrop:1.0:*:*:*:*:android:*:*	1.0	android	AirDrop	1.0
Uncanny Owl Uncanny Automator 2.10 for WordPress	cpe:2.3:a:uncannyowl:uncanny_automator:2.10:*:*:*:*:wordpress:*:*	2.10	wordpress	Automator	2.10
Xorbin Analog Flash Clock 1.0 for Joomla!	cpe:2.3:a:xorbin:analog_flash_clock:1.0:*:*:*:*:joomla\!:*:*	1.0	joomla!	Clock	1.0
Xorbin Digital Flash Clock 1.0 for WordPress	cpe:2.3:a:xorbin:digital_flash_clock:1.0:*:*:*:*:wordpress:*:*	1.0	wordpress	Clock	1.0
Computer Associates Common Services 1.0	cpe:2.3:a:ca:common_services:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Computer Associates eTrust EZ Armor 1.0	cpe:2.3:a:ca:etrust_ez_armor:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Computer Associates eTrust Secure Content Manager 1.0	cpe:2.3:a:ca:etrust_secure_content_manager:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Computer Associates eTrust Security Command Center 1.0	cpe:2.3:a:ca:etrust_security_command_center:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Computer Associates Internet Security Suite 1.0	cpe:2.3:a:ca:internet_security_suite:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Computer Associates Resource Initialization Manager 1.0	cpe:2.3:a:ca:resource_initialization_manager:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
CIRCL (Computer Incident Response Center Luxembourg) cve-search 1.0	cpe:2.3:a:circl:cve-search:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Computer And Mobile Repair Shop Management System Project Computer And Mobile Repair Shop Management System 1.0	cpe:2.3:a:computer_and_mobile_repair_shop_management_system_project:computer_and_mobile_repair_shop_management_system:1.0:*:*:*:*:*:*:*	1.0		Computer	1.0
Jenkins Computer Queue 1.0 for Jenkins	cpe:2.3:a:jenkins:computer_queue:1.0:*:*:*:*:jenkins:*:*	1.0	jenkins	Computer	1.0
Beanstalk Console Project Beanstalk Console 1.1	cpe:2.3:a:beanstalk_console_project:beanstalk_console:1.1:*:*:*:*:*:*:*	1.1		Console	1.1
Haudenschilt Family Connections CMS (FCMS) 1.0	cpe:2.3:a:haudenschilt:family_connections_cms:1.0:*:*:*:*:*:*:*	1.0		Family	1.0
Mozilla Firefox 93.0	cpe:2.3:a:mozilla:firefox:93.0:*:*:*:*:*:*:*	93.0		Firefox	93.0
Mobatek MobaXterm 6.0 Home Edition	cpe:2.3:a:mobatek:mobaxterm:6.0:*:*:*:home:*:*:*	6.0		Home	6.0
CodeCabin WP Google Maps 3.0 Pro Edition WordPress	cpe:2.3:a:codecabin:wp_google_maps:3.0:*:*:*:pro:wordpress:*:*	3.0	wordpress	Maps	3.0
mapsplugin Google Maps plugin 3.0 for Joomla!	cpe:2.3:a:mapsplugin:googlemaps:3.0:*:*:*:*:joomla\!:*:*	3.0	joomla!	Maps	3.0
Think Up Themes Responsive Vector Maps 3.0 for WordPress	cpe:2.3:a:thinkupthemes:responsive_vector_maps:3.0:*:*:*:*:wordpress:*:*	3.0	wordpress	Maps	3.0
10-Strike Network Monitor 1.0	cpe:2.3:a:10-strike:network_monitor:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Axis Communications AXIS 700 Network Document Server 1.0	cpe:2.3:h:axis:700_network_document_server:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Cisco Cloud Network Automation Provisioner 1.0	cpe:2.3:a:cisco:cloud_network_automation_provisioner:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Cisco Network Assistant 1.0	cpe:2.3:a:cisco:network_assistant:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Design Chemical Social Network Tabs 1.0 for WordPress	cpe:2.3:a:designchemical:social_network_tabs:1.0:*:*:*:*:wordpress:*:*	1.0	wordpress	Network	1.0
Fuel Rewards Network (aka com.excentus.frn) for Android 1.0	cpe:2.3:a:fuelrewards:fuel_rewards_network:1.0:*:*:*:*:android:*:*	1.0	android	Network	1.0
GNOME Network Manager VPNC 1.0	cpe:2.3:a:gnome:network_manager_vpnc:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Indoreators Web Creations Network Stark CRM 1.0	cpe:2.3:a:iwcn:stark_crm:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Microsoft Neural Network Intelligence (NNI) 1.0	cpe:2.3:a:microsoft:neural_network_intelligence:1.0:*:*:*:*:*:*:*	1.0		Network	1.0
Siemens SINEC Network Management System (NMS) 1.0	cpe:2.3:a:siemens:sinec_network_management_system:1.0:-:*:*:*:*:*:*	1.0		Network	1.0
Siemens SINEC Network Management System (NMS) 1.0 Service Pack 1	cpe:2.3:a:siemens:sinec_network_management_system:1.0:sp1:*:*:*:*:*:*	1.0		Network	1.0
News System Project News System 7.3.1 for TYPO3	cpe:2.3:a:news_system_project:news_system:7.3.1:*:*:*:*:typo3:*:*	7.3.1	typo3	News	7.3.1
Vovsoft Vov Sticky Notes 4.9	cpe:2.3:a:vovsoft:vov_sticky_notes:4.9:*:*:*:*:*:*:*	4.9		Notes	4.9
Python 3.8.9	cpe:2.3:a:python:python:3.8.9:*:*:*:*:*:*:*	3.8.9		Python	3.8.9
Cold Storage Management System Project Cold Storage Management System 1.0	cpe:2.3:a:cold_storage_management_system_project:cold_storage_management_system:1.0:*:*:*:*:*:*:*	1.0		Storage Management	1.0
Simple Cold Storage Management System Project Simple Cold Storage Management System 1.0	cpe:2.3:a:simple_cold_storage_management_system_project:simple_cold_storage_management_system:1.0:*:*:*:*:*:*:*	1.0		Storage Management	1.0
Storage Unit Rental Management System Project Storage Unit Rental Management System 1.0	cpe:2.3:a:storage_unit_rental_management_system_project:storage_unit_rental_management_system:1.0:*:*:*:*:*:*:*	1.0		Storage Management	1.0

• the version reported by software does not fit the standard format. For example, Zoom reports the version as 5.8.3 (2240).
• the app name includes extra terms that don't appear in the title. For example, zoom.us is treated as zoom us (2 terms) and does not match the title commonly used for zoom eg "Zoom 4.6.9 for macOS" or "Zoom Meetings 5.8.0 for macOS".
• no vulnerabilities are currently found for Chromium or Chromium based browsers eg Edge

General issues:

• NVD updates very slowly. CVEs are reserved, but can take a long time to update other information. For example, here is a chrome release that mentions CVE-2022-1364, but NVD does not contain any information on it. Another example is that mentions CVE-2022-29914. We should detect vulnerabilities as soon as possible, so that attackers have less time to exploit them. Most CVEs are published to NVD within a week, but for browsers it is much longer for some reason.
• Matching on title, cpe, and version leads to many false negatives/positives, especially for MacOS apps
• Sometimes the CPE dictionary is incomplete. For example, CVE-2021-24043 should have a matching CPE cpe:2.3:a:whatsapp:whatsapp:2.2145.0:*:*:*:desktop:*:*:*, but it is absent. Also not that it would not match on windows because target_sw is empty, but we try to match on windows*. Removing the target_sw would lead to many false positives.

Alternatives/Improvements:

• Optionally match on target_sw, but prefer non empty ie change the query for CPEs to ... AND (cpe.target_sw like 'windows%' OR cpe.target_sw = '') ... ORDER BY ??
• Maintain a list of rules to assign CPEs to software on MacOS and Windows to improve cpe matching accuracy.
• Aggregate vulnerabilities from security bulletins/feeds/blogs for software eg Google Chrome Releases, Zoom Security Bulletin