Concurrency

[damm]

I noticed the other day it's often brittle and breaks when trying to converge 2 nodes at once. It appears to be related to the vendoring of the cookbooks before uploading; turns into a race condition where one knife solo cook will trample the other directories contents.

Usually it leads to both cook's failing; but it would be awesome if I could run knife solo cook on multiple nodes at the same time.

[matschaffer]

Sounds like a noble cause. But first step would be to figure out if librarian & berkshelf can support this. If not the parallelization might have to happen at the ruby-layer.

[damm]

Well the problem I noticed is kitchen solo uses the same temporary path always (typically in the resolver's directory) so maybe having it use different directories to export the cookbooks to?

[arr-dev]

Just got hit with this.

Maybe the solution is to expose the knife[:berkshelf_path] config in CLI options,
so we can set different vendor dir for each node?

[matschaffer]

Random untested thought: does ENV work in the knife.rb?

2015年6月8日月曜日、Nenadnotifications@github.comさんは書きました:

Just got hit with this.

Maybe the solution is to expose the knife[:berkshelf_path] config in CLI
options,
so we can set different vendor dir for each node?

—
Reply to this email directly or view it on GitHub
#433 (comment)
.

-Mat

about.me/matschaffer

[arr-dev]

Hmm.. Probably. I'll try and get back to you.

[arr-dev]

Yeah, it does.

So setting .chef/knife.rb with this

knife[:berkshelf_path] = ENV.fetch('BERKSHELF_VENDOR_PATH', "cookbooks")

and running knife solo with BERKSHELF_VENDOR_PATH=cb1 knife solo cook ...
works just fine.

Maybe add this to the docs?

[matschaffer]

Did a wiki for now https://github.com/matschaffer/knife-solo/wiki/Running-on-multiple-hosts-concurrently

Not quite sure where we should factor this into the main readme but if you have a recommendation let me know.

[arr-dev]

Yeah, wiki should do.
Cheers