Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot report meta-profiles to Chef Compliance #155

Closed
chris-rock opened this issue Nov 14, 2016 · 5 comments

Comments

Projects
None yet
4 participants
@chris-rock
Copy link
Contributor

commented Nov 14, 2016

The current version of the audit cookbook does not support to report meta-profiles as defined here: inspec/inspec#1283 to Chef Compliance. This is caused by the json-min reporter.

@chris-rock chris-rock self-assigned this Nov 14, 2016

@chris-rock chris-rock added the ready label Nov 14, 2016

@chris-rock

This comment has been minimized.

Copy link
Contributor Author

commented Nov 23, 2016

Hi @alexpop this is my hack to run meta-profiles: b281654 The issue is that the json-min is including information about the different profiles, but we just report as one profile. We need to find a way that enables us to not reverse engineer the profile information (as we do right now in audit cookbook). Therefore we have two options:

  • extend json-min information in InSpec
  • Change the reporting API endpoint in Chef Compliance (to consume the same JSON as we get from a plain inspec exec call)
  • generate json-min from json reporter and reuse the additional information to fulfill the need for the API

Lets first gather some information and add this to our ticket:

  • Reporting format of audit cookbook to Chef Compliance API
  • Reporting format of inspec exec in Chef Compliance (plain json-min)

The main issue is the additional information about profiles:

blob[:profiles] = {}

@chris-rock chris-rock added in progress and removed ready labels Dec 2, 2016

@username-is-already-taken2

This comment has been minimized.

Copy link

commented Dec 2, 2016

I hit this problem today with what we are trying to achieve with profile chaining so would like to subcribe to any updates regarding this

@chris-rock

This comment has been minimized.

Copy link
Contributor Author

commented Dec 2, 2016

The current problem is located in our use with json-min. This format is optimize to minimize the amout of data transfered between client and server. Here is one example:

$ inspec exec examples/meta-profile examples/profile --format json-min | jq .
{
  "version": "1.7.0",
  "controls": [
    {
      "id": "ssh-01",
      "status": "passed",
      "code_desc": "File /etc/ssh/ssh_config should exist",
      "profile_id": "ssh-hardening"
    },
    ...
    {
      "id": "gordon-1.0",
      "status": "passed",
      "code_desc": "File  content should match nil",
      "profile_id": "profile"
    },
  ],
  "statistics": {
    "duration": 0.196187
  }
}

Problem:
If we execute more then one profile, we cannot distinguish between the source profiles.

This problem does not occur with the json format:

$ inspec exec examples/meta-profile examples/profile --format json | jq .
{
  "version": "1.7.0",
  "profiles": [
    {
      "name": "meta-profile",
      "title": "Meta Compliance Profile",
      "maintainer": "InSpec Authors",
      "copyright": "InSpec Authors",
      "copyright_email": "support@chef.io",
      "license": "Apache 2",
      "summary": "InSpec Profile that is only consuming dependencies",
      "version": "0.2.0",
      "depends": [
        {
          "name": "hardening/ssh-hardening"
        },
        {
          "git": "https://github.com/dev-sec/ssl-benchmark.git"
        },
        {
          "name": "windows-patch-benchmark",
          "git": "https://github.com/chris-rock/windows-patch-benchmark.git"
        }
      ],
      "supports": [],
      "controls": [
        {
          "title": "client: Check ssh_config owner, group and permissions.",
          "desc": "The ssh_config should owned by root, only be writable by owner and readable to all.",
          "impact": 1,
          "refs": [],
          "tags": {},
          "code": "",
          "source_location": {
            "ref": "tests-ssh-hardening-master/controls/ssh_spec.rb",
            "line": 25
          },
          "id": "ssh-01",
          "results": [
            {
              "status": "passed",
              "code_desc": "File /etc/ssh/ssh_config should exist",
              "run_time": 0.001083,
              "start_time": "2016-12-02 18:54:23 +0100"
            },
            ...
          ]
        },
      ],
      "groups": [
         ...
      ],
      "attributes": []
    },
    {
      "name": "profile",
      "title": "InSpec Example Profile",
      "maintainer": "Chef Software, Inc.",
      "copyright": "Chef Software, Inc.",
      "copyright_email": "support@chef.io",
      "license": "Apache 2 license",
      "summary": "Demonstrates the use of InSpec Compliance Profile",
      "version": "1.0.0",
      "supports": [
        {
          "os-family": "unix"
        }
      ],
      "controls": [
        {
          "title": "Create /tmp directory",
          "desc": "An optional description...",
          "impact": 0.7,
          "refs": [
            {
              "url": "http://...",
              "ref": "Document A-12"
            }
          ],
          "tags": {
            "data": "temp data",
            "security": null
          },
          "code": "control \"tmp-1.0\" do                        # A unique ID for this control\n  impact 0.7                                # The criticality, if this control fails.\n  title \"Create /tmp directory\"             # A human-readable title\n  desc \"An optional description...\"         # Describe why this is needed\n  tag data: \"temp data\"                     # A tag allows you to associate key information\n  tag \"security\"                            # to the test\n  ref \"Document A-12\", url: 'http://...'    # Additional references\n\n  describe file('/tmp') do                  # The actual test\n    it { should be_directory }\n  end\nend\n",
          "source_location": {
            "ref": "examples/profile/controls/example.rb",
            "line": 8
          },
          "id": "tmp-1.0",
          "results": [
            {
              "status": "passed",
              "code_desc": "File /tmp should be directory",
              "run_time": 0.0001,
              "start_time": "2016-12-02 18:54:24 +0100"
            }
          ]
        },
      ],
      "groups": [
         ...
      ],
      "attributes": []
    }
  ],
  "other_checks": [],
  "statistics": {
    "duration": 0.17154
  }
}

Possible solutions:

  • we use UUIDs instead of "profile_id": "ssh-hardening". This requires a change in InSpec report generation and inspec json, keeps the output format identical, but may still break existing applications.
  • generate json min from full json, to know all the details, we'll derive everything from full json. This has the advantage that it does not introduce a change and works with all reporter. It does not solve the root problem.
  • use new json-min structure to support multiple profiles in output report as shown following. It has the disadvantage of introducing a breaking change.
$ inspec exec examples/meta-profile examples/profile --format json-min | jq .
{
  "version": "1.7.0",
  "profiles": [
    {
      "id": "lockfile_id of profile",
      "name": "meta-profile",
      "controls": [
        {
          "id": "ssh-01",
          "status": "passed",
          "code_desc": "File /etc/ssh/ssh_config should exist",
          "profile_id": "ssh-hardening"
        },
      ],
    },
    {
      "id": "lockfile_id of profile"
      "name": "profile",
      "controls": [
        {
          "id": "gordon-1.0",
          "status": "passed",
          "code_desc": "File  content should match nil",
          "profile_id": "profile"
        },
      ],
    }
  ],
  "other_checks": [],
  "statistics": {
    "duration": 0.17154
  }
}

@chris-rock chris-rock added ready and removed in progress labels Dec 5, 2016

@chris-rock

This comment has been minimized.

Copy link
Contributor Author

commented Dec 5, 2016

As discussed with @arlimus we go forward with the intermediate solution:

generate json min from full json, to know all the details, we'll derive everything from full json.

@chris-rock chris-rock added in progress and removed ready labels Dec 12, 2016

@tas50 tas50 removed the in progress label Jan 5, 2017

@chris-rock chris-rock referenced this issue Apr 26, 2017

Merged

Always use json format for inspec report #212

4 of 4 tasks complete
@chris-rock

This comment has been minimized.

Copy link
Contributor Author

commented May 4, 2017

This should be fixed with 3.1. Please re-open if this continues to be an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.