Enabled local_mode in solo.rb.erb

[runningman84]

PR for #353

[coveralls]

Coverage remained the same when pulling 4999ca9 on runningman84:master into f95195e on matschaffer:master.

[runningman84]

With this PR chef-solo-search is obsolete!

[matschaffer]

Sweet. A couple questions though:

• Does search still work or does the expected on-disk format change?
• Can we get an integration test to verify the answer to that question?
• Should we also switch to calling chef-client -z

[runningman84]

Search works fine for me. I do a lot of stuff like

# search for all items in the 'components' data bag and loop over them
search(:components, "cookbook:dba_graylog2 AND recipe:server AND hostname:#{node[:fqdn]}") do |component|
  if !component["id"]
    raise "missing id"
  end

  # ...

end

I have now idea how to write an integration test :/

Switching to chef-client -z makes sense, but this would result in a different command for chef >= 11 and chef < 11.

[matschaffer]

We already check the chef version to determine dependencies, so I don't think switching commands would be that bad. Though maybe it's still early to worry about it. Not sure when Chef is planning on deprecating the chef-solo command itself.

[runningman84]

Can you look into this?
chef/chef#1706 (comment)
Chef 11 has a different output...

[runningman84]

I guess the best solution would to use chef-solo for chef < 11 and chef-client -z --force-formatter for Chef 11 and newer.

Ok this is my current solution:

      def cook
        ui.msg "Running Chef..."

        if chef_version_satisfies?('>11.0.0')
           #cmd = "sudo chef-solo -c #{provisioning_path}/solo.rb -j #{provisioning_path}/dna.json --force-formatter"
           cmd = "sudo chef-client -z -c #{provisioning_path}/solo.rb -j #{provisioning_path}/dna.json --force-formatter"
        else
           cmd = "sudo chef-solo -c #{provisioning_path}/solo.rb -j #{provisioning_path}/dna.json"
        end

        cmd << " -l debug" if debug?
        cmd << " -N #{config[:chef_node_name]}" if config[:chef_node_name]
        cmd << " -W" if config[:why_run]
        cmd << " -o #{config[:override_runlist]}" if config[:override_runlist]

        result = stream_command cmd
        raise "chef-solo failed. See output above." unless result.success?
      end

If you use chef-client instead of chef-solo you get warnings like this:

[2014-07-30T11:20:52+02:00] WARN: Child with name 'logsrv.virtual.json' found in multiple directories: /root/chef-solo/nodes/logsrv.virtual.json and /root/chef-solo/nodes/logsrv.virtual.json
[2014-07-30T11:20:52+02:00] WARN: Child with name 'logsrv64.virtual.json' found in multiple directories: /root/chef-solo/nodes/logsrv64.virtual.json and /root/chef-solo/nodes/logsrv64.virtual.json
[2014-07-30T11:20:52+02:00] WARN: Child with name 'mailsrv.virtual.json' found in multiple directories: /root/chef-solo/nodes/mailsrv.virtual.json and /root/chef-solo/nodes/mailsrv.virtual.json
[2014-07-30T11:20:52+02:00] WARN: Child with name 'wafproxy.virtual.json' found in multiple directories: /root/chef-solo/nodes/wafproxy.virtual.json and /root/chef-solo/nodes/wafproxy.virtual.json

[rmoriz]

I'm not sure if chef-zero works as expected. AFAIK it is incompatible wich chef-solo-search and will crash chef-zero (chef-solo local mode) converges, at least this is what I'm run into when using test-kitchen with chef-zero if a cookbook dependency injects chef-solo-search (which knife-solo afaik does by default when uploading the cookbooks).
https://github.com/pspdfkit-ops/chef-solo-search/commit/02ff543f09915a497e0c2c1a5e7dbf63af9e2620

[florian-asche]

If you provide some basic steps to test it (how/what files to change) then i can test tomorrow if search is working as expected.

[matschaffer]

If nothing else, chef zero requires that roles & environment files be json rather than ruby. That's something I'd like to get ironed out (e.g., have a pre-processing step that does the conversion) before supporting local mode from knife-solo.

We also need to add logic to remove the search cookbook when running local mode.

[matschaffer]

Oh, and there's the question of what to do about node writes. When running in local mode do we want to copy the resulting node files back to the client?

If not people might be surprised when their node attributes are not saved for subsequent runs.

[runningman84]

@matschaffer wrting results back sounds like a good idea.

If you publish the feature without that function I want not complain either. I would really like to see local mode included soon because we monkey patch knife-solo right now which is not very user friendly.

We have a lot of cookbooks who look like this:

# search for all items in the 'components' data bag and loop over them
search(:components, "cookbook:myapache AND hostname:#{hostname}") do |component|
  fail 'missing id' unless component['id']

end

They work just fine, we do not see any crashes maybe because we do not include chef-solo-search.

[rmoriz]

knife-zero for example writes back all node attributes which is nice but also some kind of over-verbose (all ohai attributes…lots of changes on each converge )

[rmoriz]

@florian-asche just create a blank cookbook with a dependency to 'chef-solo-search' in its metadata. then try to converge it using chef-zero (e.g. using test-kitchen). Due to the file structure of chef-solo-search the "upload" to the in-memory chef-server will fail (or some parts of the cookbook will be missing).

[matschaffer]

@rmoriz ouch. Perhaps the initial implementation shouldn't worry about getting node data back to the client then.

Wonder if there's something we could do to at least ensure the file on the host doesn't get clobbered so it can be used on the next run.

[rmoriz]

@matschaffer chef-client can white-list/black-list attributes it sends back to chef-server (chef-zero). For example: automatic_attribute_whitelist in https://docs.chef.io/config_rb_client.html

[runningman84]

any news here?

[matschaffer]

Not really. I can't merge as is since I pretty sure it'll break anyone not already using json roles & environments, potentially other issues too.

I'm all for chef zero but it'll have to be a more concerted effort.