New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot run profiles from Supermarket #139

Closed
jeremymv2 opened this Issue Nov 1, 2016 · 3 comments

Comments

Projects
None yet
4 participants
@jeremymv2
Contributor

jeremymv2 commented Nov 1, 2016

Cookbook version

2.0.0

Chef-client version

12.15.19

Platform Details

ubuntu 14.04

Scenario:

Cannot run Supermarket profiles

Steps to Reproduce:

Use a wrapper cookbook and set your attributes to this:

default['audit']['inspec_version'] = '1.2.0'

# collector possible values: chef-server, chef-compliance, chef-visibility, json-file
# chef-visibility requires inspec version 0.27.1 or above
default['audit']['collector'] = 'chef-server'

# Attributes server, insecure and token/refresh_token are only needed for the 'chef-compliance' collector
# server format example: 'https://comp-server.example.com/api'
default['audit']['server'] = nil

# choose between the permanent refresh_token or ephemeral token(access_token). Needed only for the 'chef-compliance' collector
default['audit']['refresh_token'] = nil

# the token(access_token) expires in 12h after creation
default['audit']['token'] = nil

# set this insecure attribute to true if the compliance server / chef server uses self-signed ssl certificates
default['audit']['insecure'] = nil

# Chef Compliance organization to post the report to. Defaults to Chef Server org if not defined
# needed for the 'chef-compliance' collector, optional for 'chef-server' collector
default['audit']['owner'] = nil

# raise exception if Compliance API endpoint is unreachable
# while fetching profiles or posting report
default['audit']['raise_if_unreachable'] = true

# fail converge if downloaded profile is not present
default['audit']['fail_if_not_present'] = false

# by default run audit every time
default['audit']['interval']['enabled'] = false

# by default run compliance once a day
default['audit']['interval']['time'] = 1440

# quiet mode, on by default because this is testing, resources aren't converged in the normal chef sense
default['audit']['quiet'] = true

# overwrite existing profile in upload mode
default['audit']['overwrite'] = true

# use json format since this is for reporting
default['audit']['format'] = 'json'

# set profiles to empty array as default
default['audit']['profiles'] = [
    {
      "name" => "ssh-hardening",
      "supermarket" => "hardening/ssh-hardening"
    }
]

Expected Result:

Execute and report on Supermarket profile

Actual Result:

Stacktrace:

root@node:/tmp/vagrant-chef# chef-client -c client.rb
[2016-11-01T20:31:26+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.15.19
[2016-11-01T20:31:26+00:00] INFO: *** Chef 12.15.19 ***
[2016-11-01T20:31:26+00:00] INFO: Platform: x86_64-linux
[2016-11-01T20:31:26+00:00] INFO: Chef-client pid: 14557
[2016-11-01T20:31:27+00:00] INFO: Run List is [recipe[audit_wrapper]]
[2016-11-01T20:31:27+00:00] INFO: Run List expands to [audit_wrapper]
[2016-11-01T20:31:27+00:00] INFO: Starting Chef Run for node
[2016-11-01T20:31:27+00:00] INFO: Running start handlers
[2016-11-01T20:31:27+00:00] INFO: Start handlers complete.
[2016-11-01T20:31:27+00:00] INFO: HTTP Request Returned 404 Not Found:
resolving cookbooks for run list: ["audit_wrapper"]
[2016-11-01T20:31:27+00:00] INFO: Loading cookbooks [audit_wrapper@0.1.0, audit@2.0.0, compat_resource@12.16.1, chef_handler@2.0.0]
Synchronizing Cookbooks:
  - audit (2.0.0)
  - chef_handler (2.0.0)
  - compat_resource (12.16.1)
[2016-11-01T20:31:27+00:00] INFO: Storing updated cookbooks/audit_wrapper/attributes/default.rb in the cache.
  - audit_wrapper (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
[2016-11-01T20:31:27+00:00] INFO: Chef Handlers will be located at: /var/chef/handlers
Recipe: chef_handler::default
  * remote_directory[/var/chef/handlers] action create
  Recipe: <Dynamically Defined Resource>
    * cookbook_file[/var/chef/handlers/README] action create (up to date)
     (up to date)
  Converging 5 resources
Recipe: chef_handler::default
  * remote_directory[/var/chef/handlers] action nothing (skipped due to action :nothing)
Recipe: audit::default
  * inspec[inspec] action install
    * chef_gem[inspec] action install (up to date)
    - install/update inspec[2016-11-01T20:31:28+00:00] WARN: Using inspec version: (1.2.0)

    - verifies the inspec version
    * chef_gem[inspec] action install (up to date)

  * directory[/var/chef/cache/handler] action create (up to date)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
  * chef_handler[Chef::Handler::AuditReport] action enable[2016-11-01T20:31:28+00:00] INFO: Disabling Chef::Handler::AuditReport as a report handler.

    - disable Chef::Handler::AuditReport as a report handler[2016-11-01T20:31:28+00:00] INFO: Disabling Chef::Handler::AuditReport as a exception handler.

    - disable Chef::Handler::AuditReport as a exception handler
    - load Chef::Handler::AuditReport from /var/chef/cache/handler/audit_report.rb[2016-11-01T20:31:28+00:00] INFO: Enabling Chef::Handler::AuditReport as a report handler.

    - enable chef_handler[Chef::Handler::AuditReport] as a report handler[2016-11-01T20:31:28+00:00] INFO: Enabling Chef::Handler::AuditReport as a exception handler.

    - enable chef_handler[Chef::Handler::AuditReport] as a exception handler
[2016-11-01T20:31:28+00:00] INFO: Chef Run complete in 0.716410089 seconds

Running handlers:
[2016-11-01T20:31:28+00:00] INFO: Running report handlers
[2016-11-01T20:31:28+00:00] WARN: Format is json-min
[2016-11-01T20:31:28+00:00] INFO: Initialize InSpec
[2016-11-01T20:31:29+00:00] WARN: URL target https://github.com/dev-sec/tests-ssh-hardening transformed to https://github.com/dev-sec/tests-ssh-hardening/archive/master.tar.gz. Consider using the git fetcher
[2016-11-01T20:31:29+00:00] INFO: Running tests from: [{:name=>"ssh-hardening", :supermarket=>"hardening/ssh-hardening"}]
[2016-11-01T20:31:30+00:00] INFO: Reporting to chef-server
[2016-11-01T20:31:30+00:00] INFO: Control Profile: ["ssh-hardening"]
[2016-11-01T20:31:30+00:00] INFO: Control Profil: ssh-hardening
[2016-11-01T20:31:30+00:00] INFO: Compliance Profils: []
[2016-11-01T20:31:30+00:00] ERROR: Report handler Chef::Handler::AuditReport raised #<NoMethodError: undefined method `[]' for nil:NilClass>
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/cookbooks/audit/libraries/collector_classes.rb:243:in `block in enriched_report'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/cookbooks/audit/libraries/collector_classes.rb:238:in `each'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/cookbooks/audit/libraries/collector_classes.rb:238:in `enriched_report'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/cookbooks/audit/libraries/collector_classes.rb:273:in `send_report'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/handler/audit_report.rb:155:in `send_report'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/handler/audit_report.rb:46:in `block in report'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/handler/audit_report.rb:33:in `each'
[2016-11-01T20:31:30+00:00] ERROR: /var/chef/cache/handler/audit_report.rb:33:in `report'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/handler.rb:259:in `run_report_unsafe'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/handler.rb:247:in `run_report_safely'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/handler.rb:125:in `block in run_report_handlers'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/handler.rb:123:in `each'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/handler.rb:123:in `run_report_handlers'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/handler.rb:135:in `block in <class:Handler>'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/client.rb:440:in `block in run_completed_successfully'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/client.rb:439:in `each'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/client.rb:439:in `run_completed_successfully'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/client.rb:298:in `run'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application.rb:302:in `block in fork_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application.rb:290:in `fork'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application.rb:290:in `fork_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application.rb:255:in `block in run_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/local_mode.rb:44:in `with_server_connectivity'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application.rb:243:in `run_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application/client.rb:450:in `loop'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application/client.rb:450:in `interval_run_chef_client'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application/client.rb:434:in `run_application'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/lib/chef/application.rb:60:in `run'
[2016-11-01T20:31:30+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.15.19/bin/chef-client:26:in `<top (required)>'
[2016-11-01T20:31:30+00:00] ERROR: /usr/bin/chef-client:54:in `load'
[2016-11-01T20:31:30+00:00] ERROR: /usr/bin/chef-client:54:in `<main>'
  - Chef::Handler::AuditReport
Running handlers complete
[2016-11-01T20:31:30+00:00] INFO: Report handlers complete
Chef Client finished, 2/9 resources updated in 04 seconds
root@node:/tmp/vagrant-chef#

@vjeffrey vjeffrey modified the milestone: 2.0 feedback fixes Nov 1, 2016

@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

@jeremymv2 You cannot report supermarket profiles to Chef Compliance yet. That profile needs to be uploaded to Chef Compliance first. Once it is uploaded, the response is visible. The issue is caused by the fact that Chef Compliance is retrieving an optimized report. In order to match the report with the control definition, the report needs to be available in Chef Compliance. This needs to be fixed in Chef Compliance.

Nevertheless, reporting to Chef Visibility should work as expected.

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

Ahh, ok that makes sense. We may need to change the example then in README.md under https://github.com/chef-cookbooks/audit#configure-node where it has 'collector'=>'chef-server' with a Supermarket profile.

"audit" => {
  "collector" => "chef-server",
  "inspec_version" => "1.2.1",
  "profiles" => [
    # profile from Chef Compliance
    {
      "name": "linux",
      "compliance": "base/linux"
    },
    # profile from supermarket
    {
      "name": "ssh",
      "supermarket": "hardening/ssh-hardening"
    },
    # local Windows path
    {
      "name": "brewinc/win2012_audit",
      # filesystem path
      "path": "E:/profiles/win2012_audit"
    },
    # github
    {
      "name": "ssl",
      "git": "https://github.com/dev-sec/ssl-benchmark.git"
    },
    # url
    {
      "name": "ssh",
      "url": "https://github.com/dev-sec/tests-ssh-hardening/archive/master.zip"
    }
  ]
}
@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

Great point @jeremymv2

@jeremymv2 jeremymv2 referenced this issue Nov 2, 2016

Merged

Supermarket #143

0 of 4 tasks complete

@binamov binamov added the in progress label Nov 2, 2016

@binamov binamov removed the in progress label Nov 3, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment