Support for OSX using command line tools and curl

[ams2121]

I've been looking through vuls to determine the effort to support OSX support and Windows. (For windows, I expect someone who knows powershell and wmi/wmic can probably put this together.) My goal is to talk to the server within the existing scope of the processing model, so that there is not additional development / O&M work required.

• Is there an interface doc (that I missed) for the CURL/ remote server option so I know exactly what it can accept and in what formats? (ie. Can I just send it the json or xml?)
• If OSX and/or Windows information is sent to the server in the proper format, would the rest of the pipeline work correctly?
• What would need to updated (if needed) to understand the OS and version keywords for Mac and windows?

Below I've captured some of the command lines and example output I would use to generate this. I wanted to open this ticket / reach out before going too far down the path.

Using the Ubuntu server mode command line as an example (at bottom with link), curl appears to send the following...

<headers>
X-Vuls-OS-Family: ubuntu
X-Vuls-OS-Release: 20.04
X-Vuls-Kernel-Release: 5.4.0-96-generic
X-Vuls-Server-Name: host name
<data segment>
binary:Package, db:Status-Abbrev, Version, Source, source:Version

It would appear OSX can gather similar data using the system profiling tool

os_family = system_profiler -json SPSoftwareDataType | grep "os_version" | cut -d \" -f4 | cut -d " " -f1
Outputs ["macOS"]

os_release = system_profiler -json SPSoftwareDataType | grep "os_version" | cut -d \" -f4 | cut -d " " -f2
Outputs ["11.6.2"]

The above is sub parsing this string which has an additional os version string.
os_version = system_profiler -json SPSoftwareDataType | grep "os_version" | cut -d \" -f4
Outputs ["macOS 11.6.2 (20G314)"]

kernel_version = "system_profiler -json SPSoftwareDataType | grep "kernel_version" | cut -d \" -f4"
Outputs ["Darwin 20.6.0"]

The individual software packages can be captured from the command
system_profiler SPApplicationsDataType SPSoftwareDataType SPDisabledSoftwareDataType SPExtensionsDataType SPLegacySoftwareDataType SPPrintersSoftwareDataType
which can also be output in JSON or XML

If this is data that the database has, and or can fuzzy match, I expect the below type data can be massaged into the above format too meet the desired data input. ie, I'm not sure what goes in the key "db:Status-Abbrev" or the difference between "Version" and "source:Version".

Example output of the above for Firefox

Plain text:

    Firefox:

      Version: 96.0.3
      Obtained from: Identified Developer
      Last Modified: 1/29/22, 9:15 AM
      Kind: Universal
      Signed by: Developer ID Application: Mozilla Corporation (43AQ936H96), Developer ID Certification Authority, Apple Root CA
      Location: /Applications/Firefox.app
      Get Info String: Firefox 96.0.3

JSON:

    {
      "_name" : "Firefox",
      "arch_kind" : "arch_arm_i64",
      "info" : "Firefox 96.0.3",
      "lastModified" : "2022-01-29T14:15:10Z",
      "obtained_from" : "identified_developer",
      "path" : "/Applications/Firefox.app",
      "signed_by" : [
        "Developer ID Application: Mozilla Corporation (43AQ936H96)",
        "Developer ID Certification Authority",
        "Apple Root CA"
      ],
      "version" : "96.0.3"
    },

XML:

			<dict>
				<key>_name</key>
				<string>Firefox</string>
				<key>arch_kind</key>
				<string>arch_arm_i64</string>
				<key>info</key>
				<string>Firefox 96.0.3</string>
				<key>lastModified</key>
				<date>2022-01-29T14:15:10Z</date>
				<key>obtained_from</key>
				<string>identified_developer</string>
				<key>path</key>
				<string>/Applications/Firefox.app</string>
				<key>signed_by</key>
				<array>
					<string>Developer ID Application: Mozilla Corporation (43AQ936H96)</string>
					<string>Developer ID Certification Authority</string>
					<string>Apple Root CA</string>
				</array>
				<key>version</key>
				<string>96.0.3</string>
			</dict>

Feature request is based on the ubuntu curl post to server from:
https://vuls.io/docs/en/usage-server.html

$ export VULS_SERVER=[Your Vuls Server]
$ curl -X POST -H "Content-Type: text/plain" -H "X-Vuls-OS-Family: `lsb_release -si | awk '{print tolower($1)}'`" -H "X-Vuls-OS-Release: `lsb_release -sr | awk '{print $1}'`" -H "X-Vuls-Kernel-Release: `uname -r`" -H "X-Vuls-Server-Name: `hostname`" --data-binary "$(dpkg-query -W -f="\${binary:Package},\${db:Status-Abbrev},\${Version},\${Source},\${source:Version}\n")" http://${VULS_SERVER}:5515/vuls > $LOCAL_REPORT

[ams2121]

Update:
I found the below example to use JSON to upload the system data. I think I can map up the Name, Arch and Version. I'm not sure I understand what is expected for release. I'll follow up as I start bouncing data off a server.

https://vuls.io/docs/en/usage-server.html

$ cat centos6.json
{
  "family": "centos",
  "release": "6.9",
  "runningKernel": {
    "release": "2.6.32-696.6.3.el6.x86_64",
    "version": "",
    "rebootRequired": false
  },
  "packages": {
    "ntp": {
      "name": "ntp",
      "version": "4.2.6p5",
      "release": "10.el6.centos.2",
      "arch": "x86_64"
    },
    "openssh": {
      "name": "openssh",
      "version": "5.3p1",
      "release": "122.el6",
      "arch": "x86_64"
    }
  }
}

[kotakanbe]

Hi, @ams2121

Release: https://github.com/future-architect/vuls/blob/master/scanner/redhatbase.go#L787

[kotakanbe]

Hi, @ams2121

BTW, is there a vulnerability database for Mac OSX?
If it is not publicly available, it is difficult to detect vulnerabilities.

Or, if we can use OSX commands to get the CVE-ID included in the unpatched part, we can do it.
An example from freeBSD may be helpful.
https://github.com/future-architect/vuls/blob/master/scanner/freebsd.go#L198

[ams2121]

Greetings Kotakanbe,
OSX looks like it will be more tricky than Linux.

> is there a vulnerability database for Mac OSX?
NVD appears to be the best option at present as Apple does not appear to have their own database. It also looks like OS vs software are in different places. I believe these include:

• OS version with any security updates
• Apple Software
• Open Source Installers: HomeBrew, MacPorts, ??

I'm going to try and address each of these in their own comment

[ams2121]

The Apple OS:
The OS appears to track by version, With CPEs being tied to words versus build number. The NVD record appears to be tied to a large combination of year, version, etc. I will need to look in the database for this CVE and see if I can consistently generate a match.

The NVD CPE dictionary (JSON & XML) does appear to call these out. I will need to look into what conversion Vuls does to OS + Version into CPE. I will try first to generate this CPE as much as possible.

https://nvd.nist.gov/products/cpe

example:

  <cpe-item name="cpe:/o:apple:mac_os_x:10.15.7:security_update_2020-007">
    <title xml:lang="en-US">Apple Mac OS X 10.15.7 Security Update 2020-007</title>
    <references>
      <reference href="https://support.apple.com/en-us/HT201222">Change Log</reference>
    </references>
    <cpe-23:cpe23-item name="cpe:2.3:o:apple:mac_os_x:10.15.7:security_update_2020-007:*:*:*:*:*:*"/>
  </cpe-item>
  <cpe-item name="cpe:/o:apple:mac_os_x:10.15.7:security_update_2021-001">
    <title xml:lang="en-US">Apple Mac OS X 10.15.7 Security Update 2021-001</title>
    <references>
      <reference href="https://support.apple.com/en-us/HT201222">Advisory</reference>
    </references>
    <cpe-23:cpe23-item name="cpe:2.3:o:apple:mac_os_x:10.15.7:security_update_2021-001:*:*:*:*:*:*"/>
  </cpe-item>

Example NVD entry for an Apple OS CVE - https://nvd.nist.gov/vuln/detail/CVE-2021-1813
CVE.org JSON record - https://www.cve.org/CVERecord?id=CVE-2021-1813
from https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2021.json.gz

}, {
          "vulnerable" : true,
          "cpe23Uri" : "cpe:2.3:o:apple:mac_os_x:10.14.6:supplemental_update_2:*:*:*:*:*:*",
          "cpe_name" : [ ]
 }, {
          "vulnerable" : true,
          "cpe23Uri" : "cpe:2.3:o:apple:mac_os_x:*:*:*:*:*:*:*:*",
          "versionStartIncluding" : "10.15",
          "versionEndIncluding" : "10.15.5",
          "cpe_name" : [ ]
 }, {
          "vulnerable" : true,
          "cpe23Uri" : "cpe:2.3:o:apple:mac_os_x:10.15.6:-:*:*:*:*:*:*",
          "cpe_name" : [ ]
 }, {

Below is someone who has done a good amount of work figuring out apple build numbers
https://tidbits.com/2020/07/08/how-to-decode-apple-version-and-build-numbers/

[matbgn]

@ams2121 did you get any further? Really interested by the topic and think of porting your "path" in Windows at the end.

[ams2121]

I haven't had the free time to look at this for a while. With the limited time I've had to look at it, I really have not been able to a handle on the osx CPE model, which appears much more dynamic (and confusing) than that used for linux. If I find some time, I'm tempted to pivot to windows as well, given there are so many more windows systems.

As notes for future me, when I get the time, I'll probably use either OSQuery or WMIC on windows to generate a software inventory list.

wmic or powershell - https://www.maketecheasier.com/get-installed-software-list-windows/

OSQUERY -
https://www.uptycs.com/blog/osquery-tutorial-gathering-software-inventory
https://fleetdm.com/queries/get-installed-windows-software
https://www.uptycs.com/blog/using-osquery-for-complete-software-inventory
osquery/osquery#3945

Maybe this could be used as a preprocessor
https://github.com/facebookincubator/nvdtools/tree/main#wfn