New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

version 2.0.0 reporting resources updated #138

Closed
jeremymv2 opened this Issue Nov 1, 2016 · 16 comments

Comments

Projects
None yet
3 participants
@jeremymv2
Contributor

jeremymv2 commented Nov 1, 2016

Cookbook version

2.0.0

Chef-client version

12.15.19

Platform Details

ubuntu 14.04

Scenario:

version 2.0.0 uses chef-handler in attempt to not report resources being updated, however each subsequent chef-client converge is reporting resources updated.

Steps to Reproduce:

use wrapper cookbook with these attributes:

default['audit']['inspec_version'] = '1.2.0'

# collector possible values: chef-server, chef-compliance, chef-visibility, json-file
# chef-visibility requires inspec version 0.27.1 or above
default['audit']['collector'] = 'chef-server'

# Attributes server, insecure and token/refresh_token are only needed for the 'chef-compliance' collector
# server format example: 'https://comp-server.example.com/api'
default['audit']['server'] = nil

# choose between the permanent refresh_token or ephemeral token(access_token). Needed only for the 'chef-compliance' collector
default['audit']['refresh_token'] = nil

# the token(access_token) expires in 12h after creation
default['audit']['token'] = nil

# set this insecure attribute to true if the compliance server / chef server uses self-signed ssl certificates
default['audit']['insecure'] = nil

# Chef Compliance organization to post the report to. Defaults to Chef Server org if not defined
# needed for the 'chef-compliance' collector, optional for 'chef-server' collector
default['audit']['owner'] = nil

# raise exception if Compliance API endpoint is unreachable
# while fetching profiles or posting report
default['audit']['raise_if_unreachable'] = true

# fail converge if downloaded profile is not present
default['audit']['fail_if_not_present'] = false

# by default run audit every time
default['audit']['interval']['enabled'] = false

# by default run compliance once a day
default['audit']['interval']['time'] = 1440

# quiet mode, on by default because this is testing, resources aren't converged in the normal chef sense
default['audit']['quiet'] = true

# overwrite existing profile in upload mode
default['audit']['overwrite'] = true

# use json format since this is for reporting
default['audit']['format'] = 'json'

# set profiles to empty array as default
default['audit']['profiles'] = [{
      'name' => 'linux',
      'compliance' => 'base/linux'
}]

Expected Result:

Chef-client runs should report 0/x resources updated at the end of the report handlers phase.

Actual Result:

I'm seeing 2 resources updated on each chef-client converge.

root@node:/tmp/vagrant-chef# chef-client -c client.rb
[2016-11-01T19:32:37+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.15.19
[2016-11-01T19:32:37+00:00] INFO: *** Chef 12.15.19 ***
[2016-11-01T19:32:37+00:00] INFO: Platform: x86_64-linux
[2016-11-01T19:32:37+00:00] INFO: Chef-client pid: 4712
[2016-11-01T19:32:39+00:00] INFO: Run List is [recipe[audit_wrapper]]
[2016-11-01T19:32:39+00:00] INFO: Run List expands to [audit_wrapper]
[2016-11-01T19:32:39+00:00] INFO: Starting Chef Run for node
[2016-11-01T19:32:39+00:00] INFO: Running start handlers
[2016-11-01T19:32:39+00:00] INFO: Start handlers complete.
[2016-11-01T19:32:39+00:00] INFO: HTTP Request Returned 404 Not Found:
resolving cookbooks for run list: ["audit_wrapper"]
[2016-11-01T19:32:39+00:00] INFO: Loading cookbooks [audit_wrapper@0.1.0, audit@2.0.0, compat_resource@12.16.1, chef_handler@2.0.0]
Synchronizing Cookbooks:
  - audit_wrapper (0.1.0)
  - audit (2.0.0)
  - compat_resource (12.16.1)
  - chef_handler (2.0.0)
Installing Cookbook Gems:
Compiling Cookbooks...
[2016-11-01T19:32:39+00:00] INFO: Chef Handlers will be located at: /var/chef/handlers
Recipe: chef_handler::default
  * remote_directory[/var/chef/handlers] action create
  Recipe: <Dynamically Defined Resource>
    * cookbook_file[/var/chef/handlers/README] action create (up to date)
     (up to date)
  Converging 5 resources
Recipe: chef_handler::default
  * remote_directory[/var/chef/handlers] action nothing (skipped due to action :nothing)
Recipe: audit::default
  * inspec[inspec] action install
    * chef_gem[inspec] action install (up to date)
    - install/update inspec[2016-11-01T19:32:39+00:00] WARN: Using inspec version: (1.2.0)

    - verifies the inspec version
    * chef_gem[inspec] action install (up to date)

  * directory[/var/chef/cache/handler] action create (up to date)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
  * chef_handler[Chef::Handler::AuditReport] action enable[2016-11-01T19:32:39+00:00] INFO: Disabling Chef::Handler::AuditReport as a report handler.

    - disable Chef::Handler::AuditReport as a report handler[2016-11-01T19:32:39+00:00] INFO: Disabling Chef::Handler::AuditReport as a exception handler.

    - disable Chef::Handler::AuditReport as a exception handler
    - load Chef::Handler::AuditReport from /var/chef/cache/handler/audit_report.rb[2016-11-01T19:32:39+00:00] INFO: Enabling Chef::Handler::AuditReport as a report handler.

    - enable chef_handler[Chef::Handler::AuditReport] as a report handler[2016-11-01T19:32:39+00:00] INFO: Enabling Chef::Handler::AuditReport as a exception handler.

    - enable chef_handler[Chef::Handler::AuditReport] as a exception handler
[2016-11-01T19:32:39+00:00] INFO: Chef Run complete in 0.674548823 seconds

Running handlers:
[2016-11-01T19:32:39+00:00] INFO: Running report handlers
[2016-11-01T19:32:39+00:00] WARN: Format is json-min
[2016-11-01T19:32:39+00:00] INFO: Initialize InSpec
[2016-11-01T19:32:39+00:00] INFO: Running tests from: [{:name=>"linux", :compliance=>"base/linux"}]
[2016-11-01T19:32:40+00:00] INFO: Reporting to chef-server
[2016-11-01T19:32:40+00:00] INFO: Control Profile: ["linux"]
[2016-11-01T19:32:40+00:00] INFO: Control Profil: linux
[2016-11-01T19:32:40+00:00] INFO: Compliance Profils: [{:owner=>"base", :profile_id=>"linux"}]
[2016-11-01T19:32:40+00:00] INFO: Report to Chef Server: https://chef-server.test/compliance/organizations/brewinc/inspec
  - Chef::Handler::AuditReport
Running handlers complete
[2016-11-01T19:32:40+00:00] INFO: Report handlers complete
Chef Client finished, 2/9 resources updated in 02 seconds
root@node:/tmp/vagrant-chef#
@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 1, 2016

It seems to be coming from enabling/disabling report and exception handlers.

@vjeffrey vjeffrey modified the milestone: 2.0 feedback fixes Nov 1, 2016

@vjeffrey

This comment has been minimized.

Collaborator

vjeffrey commented Nov 2, 2016

yup, i confirmed it's

&&&&&&&& updated resources
inspec[inspec]
chef_handler[Chef::Handler::AuditReport]

the original issue that brought up the idea of the handler mentioned that resources were being reported as 'converged'. i'm not super familiar with chef and resources...is resources being reported as converged and resources being reported as updated the same thing? i'm not quite sure how to get around these updated resources

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

Yes, in this context they are the same thing.

The default report and exception handlers run at the end of the execution phase (converge) and the report handler sends back the updated_resources object which is a list of resources that were marked as updated as a result of the chef-client run.

@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

@jeremymv2 Any ideas how we can get around that? I was thinking that Chef Handler are covering that properly

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

Taking a deeper look. Looks like it's the inspec resource and the handler that account for the two updates.

[1] pry(#<Chef::Handler::AuditReport>)> run_status.updated_resources.each {|r| Chef::Log.warn "#{r.to_s}"}
[2016-11-02T15:24:13+00:00] WARN: inspec[inspec]
[2016-11-02T15:24:13+00:00] WARN: chef_handler[Chef::Handler::AuditReport]
=> [<inspec[inspec] @name: "inspec" @noop: nil @before: nil @params: {} @provider: nil @allowed_actions: [:nothing, :install] @action: [:install] @updated: true @updated_by_last_action: true @supports: {} @ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "/var/chef/cache/cookbooks/audit/recipes/default.rb:4:in `from_file'" @guard_interpreter: nil @default_guard_interpreter: :default @elapsed_time: 0.496733461 @sensitive: false @resource_name: :inspec @declared_type: :inspec @cookbook_name: "audit" @recipe_name: "default" @version: "1.2.0">,
 <chef_handler[Chef::Handler::AuditReport] @name: "Chef::Handler::AuditReport" @noop: nil @before: nil @params: {} @provider: nil @allowed_actions: [:nothing, :enable, :disable] @action: [:enable] @updated: true @updated_by_last_action: true @supports: {:report=>true, :exception=>true} @ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "/var/chef/cache/cookbooks/audit/recipes/default.rb:19:in `from_file'" @guard_interpreter: nil @default_guard_interpreter: :default @elapsed_time: 0.004970695 @sensitive: false @declared_type: :chef_handler @cookbook_name: "audit" @recipe_name: "default" @source: "/var/chef/cache/handler/audit_report.rb" @class_name: "Chef::Handler::AuditReport" @arguments: []>]
[2] pry(#<Chef::Handler::AuditReport>)>
@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

@vjeffrey figured, those events happen:

* chef_handler[Chef::Handler::AuditReport] action enable
   - disable Chef::Handler::AuditReport as a report handler
   - load Chef::Handler::AuditReport from /opt/kitchen/cache/handler/audit_report.rb
   - enable chef_handler[Chef::Handler::AuditReport] as a report handler

Not sure how we can prevent them

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

@jeremymv2 so we just remove those and it should be fine?

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

I think one option may be to replicate what the chef_handler cookbook is doing when loading the AuditReport handler library, but do so in a library in the audit cookbook, without using the LWRP's resources from chef_handler cookbook.

@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

@jeremymv2 not sure I got that

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

In theory I think it boils down to adding this to a library file:

require "#{::File.join(Chef::Config[:file_cache_path], 'handler')}/audit_report"
Chef::Config.send('report_handlers') << Chef::Handler::AuditReport.new()
Chef::Config.send('exception_handlers') << Chef::Handler::AuditReport.new()

In essence, that's what's going on here: https://github.com/chef-cookbooks/chef_handler/blob/master/libraries/helpers.rb#L26

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

So that works.. @chris-rock
I tested adding a library function like this:

  def load_audit_handler
    Chef::Log.info("loading handler...")
    require "#{::File.join(Chef::Config[:file_cache_path], 'handler')}/audit_report"
    Chef::Config.send('report_handlers') << Chef::Handler::AuditReport.new
    Chef::Config.send('exception_handlers') << Chef::Handler::AuditReport.new
  end

Notice no "chef_handler" cookbook is loaded and 0/6 resources were updated below.

root@node:/tmp/vagrant-chef# chef-client -c client.rb
[2016-11-02T19:15:14+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.15.19
[2016-11-02T19:15:14+00:00] INFO: *** Chef 12.15.19 ***
[2016-11-02T19:15:14+00:00] INFO: Platform: x86_64-linux
[2016-11-02T19:15:14+00:00] INFO: Chef-client pid: 8141
[2016-11-02T19:15:15+00:00] INFO: Run List is [recipe[audit_wrapper]]
[2016-11-02T19:15:15+00:00] INFO: Run List expands to [audit_wrapper]
[2016-11-02T19:15:15+00:00] INFO: Starting Chef Run for node
[2016-11-02T19:15:15+00:00] INFO: Running start handlers
[2016-11-02T19:15:15+00:00] INFO: Start handlers complete.
[2016-11-02T19:15:15+00:00] INFO: HTTP Request Returned 404 Not Found:
resolving cookbooks for run list: ["audit_wrapper"]
[2016-11-02T19:15:15+00:00] INFO: Loading cookbooks [audit_wrapper@0.1.0, audit@2.0.0, compat_resource@12.16.1]
Synchronizing Cookbooks:
  - compat_resource (12.16.1)
  - audit (2.0.0)
[2016-11-02T19:15:15+00:00] INFO: Storing updated cookbooks/audit_wrapper/attributes/default.rb in the cache.
  - audit_wrapper (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
Recipe: audit::default
  * chef_gem[inspec] action install (up to date)
  * directory[/var/chef/cache/handler] action create (up to date)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
[2016-11-02T19:15:16+00:00] INFO: loading handler...
  Converging 3 resources
  * chef_gem[inspec] action install (up to date)
  * directory[/var/chef/cache/handler] action nothing (skipped due to action :nothing)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
[2016-11-02T19:15:16+00:00] INFO: Chef Run complete in 0.308372376 seconds

Running handlers:
[2016-11-02T19:15:16+00:00] INFO: Running report handlers

Frame number: 0/21

From: /var/chef/cache/handler/audit_report.rb @ line 14 Chef::Handler::AuditReport#report:

     9:
    10:        require 'pry'
    11:        binding.pry
    12:
    13:         # ensure reporters is array
 => 14:         reporters = handle_reporters(node['audit']['collector'])
    15:
    16:         # collect attribute values
    17:         server = node['audit']['server']
    18:         user = node['audit']['owner']
    19:         token = node['audit']['token']

[1] pry(#<Chef::Handler::AuditReport>)> run_status.updated_resources.each {|r| Chef::Log.info "#{@line_prefix}#{r.to_s}"}
=> []
[2] pry(#<Chef::Handler::AuditReport>)>
[2016-11-02T19:15:19+00:00] WARN: Format is json-min
[2016-11-02T19:15:19+00:00] INFO: Initialize InSpec
[2016-11-02T19:15:19+00:00] INFO: Running tests from: [{:name=>"linux", :compliance=>"base/linux"}]
[2016-11-02T19:15:20+00:00] INFO: Reporting to chef-server
[2016-11-02T19:15:20+00:00] INFO: Control Profile: ["linux"]
[2016-11-02T19:15:20+00:00] INFO: Control Profil: linux
[2016-11-02T19:15:20+00:00] INFO: Compliance Profils: [{:owner=>"base", :profile_id=>"linux"}]
[2016-11-02T19:15:20+00:00] INFO: Report to Chef Server: https://chef-server.test/compliance/organizations/brewinc/inspec
  - Chef::Handler::AuditReport
Running handlers complete
[2016-11-02T19:15:20+00:00] INFO: Report handlers complete
Chef Client finished, 0/6 resources updated in 06 seconds
root@node:/tmp/vagrant-chef#

I'm going to ping @lamont-granquist though to make sure I'm not overlooking anything.

@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 2, 2016

@jeremymv2 Awesome! Looking forward to have a PR?

@jeremymv2

This comment has been minimized.

Contributor

jeremymv2 commented Nov 2, 2016

Yes - working on changing the inspec resource as well since it was also reporting as updated.

@vjeffrey

This comment has been minimized.

Collaborator

vjeffrey commented Nov 2, 2016

thank youuuuu!!!!

@jeremymv2 jeremymv2 referenced this issue Nov 2, 2016

Merged

fixing resources reporting as updated #142

0 of 4 tasks complete
@chris-rock

This comment has been minimized.

Collaborator

chris-rock commented Nov 3, 2016

Solved via #142

@chris-rock chris-rock closed this Nov 3, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment