[chef-compliance] Scan Report Calculations

[username-is-already-taken2]

Description

I've noticed within the scan report that skipped controls (due to using an only_if statement) are being counted as critical's

control 'nagios-1.1' do
  impact 1.0
  only_if { os.family == 'redhat' }
  ....

InSpec and Platform Version

Chef Compliance 1.7.7

Possible Solutions

At the moment we are having to patch the results (ie subtract skipped from critial) to display the true number of critial issues found becuase at first glance people get scared ;)

[chris-rock]

Just tried to identify where the issue is located

control 'a' do
  impact 1.0
  describe 1 do
    it { should cmp 2 }
  end
  only_if { 1 == 2 }
end

The following result is generated by inspec exec trials/count.rb --format=json | jq .

{
  "version": "1.11.0",
  "controls": [
    {
      "status": "skipped",
      "code_desc": "Operating System Detection",
      "skip_message": "Skipped control due to only_if condition.",
      "resource": "Operating System Detection",
      "run_time": 1.2e-05,
      "start_time": "2017-02-15 13:23:13 +0100"
    }
  ],
  "other_checks": [],
  "profiles": [
    {
      "supports": [],
      "controls": [
        {
          "title": null,
          "desc": null,
          "impact": 1,
          "refs": [],
          "tags": {},
          "code": "control 'a' do\n  impact 1.0\n  describe 1 do\n    it { should cmp 2 }\n  end\n  only_if { 1 == 2 }\nend\n",
          "source_location": {
            "ref": "trials/count.rb",
            "line": 1
          },
          "id": "a",
          "results": [
            {
              "status": "skipped",
              "code_desc": "Operating System Detection",
              "skip_message": "Skipped control due to only_if condition.",
              "resource": "Operating System Detection",
              "run_time": 1.2e-05,
              "start_time": "2017-02-15 13:23:13 +0100"
            }
          ]
        }
      ],
      "groups": [
        {
          "title": null,
          "controls": [
            "a"
          ],
          "id": "count.rb"
        }
      ],
      "attributes": []
    }
  ],
  "statistics": {
    "duration": 0.000469
  }
}

Therefore the counting is done incorrectly in Chef Compliance instead of InSpec.

[chris-rock]

This is an official ticket for our support now. Next steps will be tracked in https://getchef.zendesk.com/agent/tickets/13766