Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Berkshelf support for the chef-solo provisioner? #590

Closed
netsensei opened this issue Nov 3, 2013 · 17 comments

Comments

@netsensei
Copy link

commented Nov 3, 2013

Hi,

Is there support for something like Berkshelf on the roadmap? In #126 a chef-solo provisioner was implemented. The cookbook_paths directive points to locations where various vendor/app specific cookbooks are located.

Since Berkshelf manages cookbooks much like Bundler, there is not really a specific cookbooks directory for a given runlist. (chef-librarian, on the other hand, does exactly that)

The vagrant-berkshelf plugin from RiotGames (https://github.com/RiotGames/vagrant-berkshelf) integrates Berkshelf with the Vagrantfile. But since Packer allows you to "pre-package" software into the image rather then provision it on runtime, I don't think this is what I'm looking for.

@mitchellh

This comment has been minimized.

Copy link
Member

commented Nov 4, 2013

It would be easy enough to script the Packer run so that Berkshelf is run prior to Packer being run. I recommend going that route, but would gladly look at a pull request to implement this. Personally it isn't on the roadmap, but a feature I'd be happy to look at.

@mitchellh mitchellh closed this Nov 4, 2013
@masteinhauser

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2013

I have solved this problem by scripting Berkshelf to run before Packer and using the --vendor option that Berkshelf provides.

@hd-deman

This comment has been minimized.

Copy link

commented Nov 9, 2013

@JeanMertz

This comment has been minimized.

Copy link

commented Jan 6, 2014

@mitchellh Just to clarify:

to script the Packer run so that Berkshelf is run prior to Packer being run

I presume with "the Packer" you mean something different than the Packer tool itself? i.e. is it possible to add something like a pre-processor, or even a shell provisioner that runs on dom0 instead of the VM?

@mahmoudimus

This comment has been minimized.

Copy link

commented Apr 9, 2014

@haad

This comment has been minimized.

Copy link

commented Nov 11, 2014

+1

@haad

This comment has been minimized.

Copy link

commented Nov 11, 2014

I will try to make a pull request for this one it's a must have feature these days.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

This seems like an oddly-specific thing for packer to do. IMO, the user should be responsible for vendoring the cookbooks with Berkshelf into a folder.

@masteinhauser

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

I agree with @sethvargo and continue to use the solution I first mentioned. This is simply integrated into the rest of our control scripts around Packer which gives us more control of specifically when and why we need to vendor. Packer simply consuming these cookbooks, especially if they need to be checked at all, made everything more simpler and more deterministic for us.

@haad

This comment has been minimized.

Copy link

commented Nov 11, 2014

There are multiple occasions where they already went this route, vagrant, kitchen. To me berkshelf is just a way of distributing cookbooks. Packer should be able to vendor them and run packing without external tools.

@sethvargo

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

@haad who is "they"? Vagrant does not have support for Berkshelf (there's a third-party plugin). Test Kitchen is not a HashiCorp project. I think it is outside of the scope of Packer to manage Berkshelf. I believe it is the responsibility of the user to have Berkshelf installed and you can run berks vendor as part of your packer build. Maybe @masteinhauser would be nice and share some code snippets to help you out 😄

@masteinhauser

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

This is pretty old code, and not everything that is used now but what I can release.
Usage is essentially ./pipeline.sh example-vm in your local project directory which contains a Berksfile.
We were running into update issues cross platform and thus needed to write the update.sh script, I truly hope that hack is not needed anymore. There is a more complex business deployment case behind these, which I didn't fully strip out.

Yes, this is added on code working around Packer itself but I would argue that Packer is not your build, release and upgrade mechanism but merely an integral component within a larger architecture. Tooling such as this wraps around and operationalizes the individual components into specifically what your project and business case require. As this is often different from project to project, especially across specific users, scripts as these allow for customization and flexibility outside the actual FOSS tool being used. Such as Packer and Berkshelf in this specific case.

Just wanted to share what I could, especially since I got called out on it. :)
https://gist.github.com/masteinhauser/ef3caa1d938f9b944543

@haad or anyone else, feel free to reach out to me not on this ticket to discuss further workflows and ideas for customization.

@Jarodiv

This comment has been minimized.

Copy link

commented Jul 8, 2015

The solution pointed out in this thread multiple times is, to run Berkshelf prior to packer. I am not very happy with that solution, since it requires chef, or at least Berkshelf, to be installed on the machine that runs packer AND on the the virtual machine that is going to be provisioned (even though only one of them needs chef for provisioning).

Since I try to keep my systems as clean as possible that doesn't make much sense to me. Wouldn't it be more logic to run Berkshelf from INSIDE the VM?

The solution I came up with requires only one chef installation:
Make sure, your Berksfile is copied into the VM (for example by placing it beneath your 'cookbook_paths' or by using a file provisioner).
Set the 'execute_command' of the chef-solo provisioner to run berkshelf (vendor) prior to chef-solo with vendor dir = 'staging_directory'.

Even though this works, it is a hack and I am not very happy with it. What I would like to have is an official way to run Berkshelf (vendor) from within the chef-solo provisioner: after chef is installed but before chef-solo is executed

@Joseph-R

This comment has been minimized.

Copy link

commented Dec 15, 2015

Even though this works, it is a hack and I am not very happy with it. What I would like to have is an official way to run Berkshelf (vendor) from within the chef-solo provisioner: after chef is installed but before chef-solo is executed

Same.

For better or worse, Berkshelf is a very commonly used (the default?) tool to package and version chef cookbook dependencies. It's integral to every Chef workflow I've seen.

Trying to use a Chef provisioner without integrating Berkshelf is like trying to build a Ruby app deployment tool without integrating Bundler. It's not that you can't hack around it. It's that those hacks are going to be janky and suck.

@jkburges

This comment has been minimized.

Copy link

commented May 13, 2016

FWIW, I've run in to this same issue.

I want to be able to run packer builds from atlas, so AFAICT, installing berkshelf and what not on the host running packer is not an option (maybe I'm wrong about this though).

I then went down the road of vendoring cookbooks on the target, but the first problem I ran in to was the fact that git is missing there (needed to fetch cookbooks from github) - git being one of the very dependencies that would be installed with chef. Nothing that can't be solved, but it's definitely a bit of an ugly chicken and egg scenario.

@jkburges

This comment has been minimized.

Copy link

commented May 16, 2016

I want to be able to run packer builds from atlas, so AFAICT, installing berkshelf and what not on the host running packer is not an option (maybe I'm wrong about this though).

I was wrong - it is possible to install software on the host from an atlas/packer context.

Ref: https://atlas.hashicorp.com/help/packer/builds/installing-software

@DarwinJS

This comment has been minimized.

Copy link
Contributor

commented Sep 30, 2016

@Joseph-R and @mstrache - do either of you have examples of (or a repo demonstrating) your modified "execute_command" for the chef-solo provisioner?

I am interested in this approach, but having difficulty.

Also does the chef install that results from the chef-solo provisioner contain berks or does it have to be placed on the machine being built by a specific action?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.