Chef-Provisioning support

[rmoriz]

Hi,

I think knife-zero and chef-provisioning could be a great combination:

Chef-Provisioning is cookbook-as-a-gem that provides various resources to spin up nodes in the cloud and converge them using Chef's native cookbook DSL. https://github.com/chef/chef-provisioning

It also can spin up a local chef-zero instance and forward the port to the nodes, which run chef-client -z.

Ideally, one would use chef-provisioning to create a fleet of nodes and use knife-zero to manage them individually in the future.

Afaik, the only issue that prevents this currently, is the different port that knife-zero and chef-provisioning use to forward the local chef-zero service to the nodes (configured in /etc/chef/client.rb while bootstrapping).

Here is a chef-provisioning example (needs chef-provisioning and chef-provisioning-google):
https://gist.github.com/rmoriz/f293ca1402591c5469ae

The downsite of chef-provisioning is, that you can only run the complete recipe, e.g. converge all nodes. This is where knife-zero would be awesome.

[sawanoboly]

This is my conjecture. We already have node objects into nodes/ after the chef-provisioning first run.
knife node list -z shows nodes which was provisioned and machines are searchable.

e.g.

$ knife search node "name:mario" -z

Node Name:   mario
Environment: _default
FQDN:        mario
IP:          10.0.2.15
Run List:    
Roles:       
Recipes:     
Platform:    ubuntu 12.04
Tags:        itsa_me

Maybe, we can use knife-zero to them without zero bootstrap. Please try knife zero chef_client "name:mario"(or "name:testnode01").

[rmoriz]

There is one issue:

knife-zero bootstraps client.rb to:

chef_server_url  "chefzero://localhost:8889"

while chef-provisioning bootstraps ´client.rb`to

chef_server_url "http://localhost:8901"

To match the port, I've added the following to my .chef/knife.rb:

knife[:remote_chef_zero_port] = 8889

Which made it work. Awesome :-)

knife zero chef_client 'name:testnode01' -a cloud_v2.public_ipv4 --sudo -i .chef/keys/google_default -x rmoriz
xxxxxx.82 Starting Chef Client, version 12.3.0
...
xxxxxx.82 Running handlers:
xxxxxx.82 Running handlers complete
xxxxxx.82 Chef Client finished, 0/11 resources updated in 3.149465662 seconds

[sawanoboly]

Thank you for good report 👍 , I'll try it too.

BTW, can I close this issue?

[rmoriz]

yes. thanks (maybe put a link into the readme for others?)

[sawanoboly]

Yes, I'll update README after I confirmed all steps.

[sawanoboly]

@rmoriz

Hi, I tried chef-provisioning with ec2.

My recipe is below.

## simple.rb
require 'chef/provisioning'

with_driver 'fog:AWS:higanworks'
machine 'web'
machine 'db'

And Chef-Client created client.rb after new machine provisioned.

## /etc/chef/client.rb
chef_server_url "http://localhost:8889"
node_name "web"
client_key "/etc/chef/client.pem"
ssl_verify_mode :verify_none

Then, I added just one line to my knife.rb below.

local_mode true

And then, I run knife zero chef_client to two machines, it works.
I think that remote_chef_zero_port is not really necessary.

(recipe sleep::default is created after provisioning. It means that cookbook was not transferred when machiens were provisioned.)

$ knife zero chef_client "name:*b" --identity-file /Users/sawanoboriyu/.chef/keys/chef_default -x ubuntu -o sleep
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com Starting Chef Client, version 12.3.0
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com Starting Chef Client, version 12.3.0
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com [2015-05-31T09:51:26+00:00] WARN: Run List override has been provided.
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com [2015-05-31T09:51:26+00:00] WARN: Original Run List: []
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com [2015-05-31T09:51:26+00:00] WARN: Overridden Run List: [recipe[sleep]]
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com resolving cookbooks for run list: ["sleep"]
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com Synchronizing Cookbooks:
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com   - sleep
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com Compiling Cookbooks...
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com Converging 1 resources
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com Recipe: sleep::default
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com   * ruby_block[sleep] action runhello
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com [2015-05-31T09:51:27+00:00] WARN: Run List override has been provided.
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com [2015-05-31T09:51:27+00:00] WARN: Original Run List: []
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com [2015-05-31T09:51:27+00:00] WARN: Overridden Run List: [recipe[sleep]]
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com resolving cookbooks for run list: ["sleep"]
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com Synchronizing Cookbooks:
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com   - sleep
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com Compiling Cookbooks...
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com Converging 1 resources
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com Recipe: sleep::default
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com   * ruby_block[sleep] action runhello
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com hello
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com hello
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com hello
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com hello
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com hello
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com hello
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com hello
ec2-54-238-69-117.ap-northeast-1.compute.amazonaws.com hello
ec2-54-150-207-19.ap-northeast-1.compute.amazonaws.com hello

.....

knife zero chef_client ignores setting of chef_server_url on remote machine. Because zero chef_client builds and passes chef-server-url to chef-client(remote) per every run.

I wonder what remote_chef zero_port setting is affected your environment ?

[rmoriz]

interesting. I was using the google provisioning plugin.