fleetdm/vulnerabilities feed releases can be tagged when parsing the feed would cause a panic on the currently released Fleet server

[iansltx]

Fleet version: all

---

💥  Actual behavior

The vulnerabilities feed build workflow is capable of pushing a release that panics Fleet server attempts to parse it (see #21239).

🧑‍💻  Steps to reproduce

1. Run the vulnerabilities build workflow from a commit prior to Vuln cron fail fix #21240, e.g. in a fork. This will succeed and push artifacts.
2. Attempt to import the feed (e.g. by revising the hard-coded GitHub remote to the fork used above) into e.g. a version of Fleet without NVD import fails (panic) when description_data array is empty #21242.

At first glance the above sounds like "switch back to a broken version and wow it's broken!" but the mitigation we're looking for here ensures that if we have issues in some other part of the JSON we're delivering we catch those issues as well and opt to fail out (and raise a P1 alert) rather than publishing what amounts to a poisoned vulns feed.

🛠️ To fix

We need to add a step in the workflow that basically does at least the inverse of what the vulnerability processing cron (or, more simply, the vuln_processing command) does when the disable_data_sync config value is set to true: sync vulnerabilities without worrying about scanning against installed packages on hosts. Add that step after this one:

https://github.com/fleetdm/vulnerabilities/blob/b17bd30c22abe6e23f01ae89b5162e8acfa8a990/.github/workflows/generate-cve.yml#L62-L66

and fail the build if the command returns nonzero.

In order to do this, we'll need to decide what's in-scope to integration test, and how to effectively mock the rest; my thought here is that we can place the files where the server would expect them rather than downloading over GitHub, and that will still catch the sorts of issues we're looking for. That's about as good as we can do without publishing releases.

/cc @sharon-fdm @getvictor

QA Steps

1. fork https://github.com/fleetdm/vulnerabilities
2. modify .github/workflows/generate-cve.yml
   Add a step to install jq sqlite3 xz-utils

      - name: Set up environment
        run: |
          apt-get update
          apt-get install -y jq sqlite3 xz-utils

Add a step after name: Build goval-dictionary and ensure destination dir exists step to break the feed files

      - name: Break files
        run: |
          gzip -dk ./fleet/cvefeed/nvdcve-1.1-2002.json.gz
          jq '(.CVE_Items[].cve.description) = {"description_data":""}' ./fleet/cvefeed/nvdcve-1.1-2002.json > ./fleet/cvefeed/nvdcve-1.1-2002_modified.json && mv ./fleet/cvefeed/nvdcve-1.1-2002_modified.json ./fleet/cvefeed/nvdcve-1.1-2002.json
          gzip -f ./fleet/cvefeed/nvdcve-1.1-2002.json
          sqlite3 ./fleet/cvefeed/amzn_01.sqlite3 <<EOF
          PRAGMA foreign_keys=off;
          BEGIN TRANSACTION;
          CREATE TABLE packages_new AS SELECT id, name, arch, definition_id FROM packages;
          INSERT INTO packages_new SELECT id, name, arch, definition_id FROM packages;
          DROP TABLE packages;
          ALTER TABLE packages_new RENAME TO packages;
          COMMIT;
          PRAGMA foreign_keys=on;
          EOF

4. Add a step to run the validation after the name: Build goval-dictionary and ensure destination dir exists to run the validation

      - name: Validate NVD Feeds
        run: |
          cd fleet
          go run cmd/cve/validate/main.go --db_dir ./cvefeed --debug

Testing with patch.

1. fork https://github.com/fleetdm/vulnerabilities
2. download the patch into the vulnerabilities repository
3. modify .github/workflows/generate-cve.yml
   Right after the step that checks out the fleet repository name: Checkout Repo add a step that applies the patch to the checked out fleet repository

      - name: Apply patch to Fleet repository
        run: |
          cd fleet
          git apply ../../description_failure.patch.txt

4. Run the workflow. You should see it fail, this is intended, because it means the new code is properly detecting an error in the import process.

[iansltx]

@sharon-fdm can we bring this into the sprint along with the NVD one, as it's going to be basically the same work on both sides? I actually thought the NVD issue was this one. Estimate would be 2, given that we'd do both at the same time.