Attribute and Template Overrides

[jweiss]

I'm not sure if this is a chef-zero issue or in general how chef-server behaves.

I'm currently using chef-solo 11.10. There I can override templates that are used in recipes from the cookbooks directory by supplying them in site-cookbooks in the same path, e.g.

cookbooks/apache2/templates/default/apache.conf.erb
site-cookbooks/apache2/templates/default/apache.conf.erb

When I run chef-client in local mode against chef-zero, my template override is ignored and only the original in 'cookbooks' is loaded. The same applies to attribute files.

As it works in chef-solo I'm not sure if this is how it is supposed to work or a bug in my client.rb or even chef-zero?

Thanks!

[jkeiser]

This is correct, chef-zero does not merge cookbooks from different directories. It caused a bunch of confusion when it did. That's an interesting use case though. I wonder if there is a way to get the functionality without confusing people ...

[jweiss]

So merging is the right thing to do from your point of view?

How hard would it be to bring it back as an optional option?

What confused people? I'd like to understand how this is different from the expected chef-server behavior?

Thank you

[danielsdeleo]

When using chef-server, you upload cookbooks via knife, which gives you a big warning that the feature is deprecated. It would've been removed in Chef 11.0 if we had enough time. I know some people really like the feature but I think it should be done explicitly via some extra tool that does the overlays and produces a cookbook with the overrides. Doing it implicitly leads to no end of grief.

[jweiss]

Sounds like we should talk at ChefConf :-)

But today the merge is expected? It should be simple then for me to do the merge on the filesystem before starting chef-zero.

[jkeiser]

Expected is probably a strong word :) knife cookbook upload does it, knife upload /cookbooks does not; we are presently in an inconsistent state and we want to move to a world where the merge does not happen.

[jweiss]

Curious to know why you want to abandon it. I always found it most useful when using other people's cookbooks. No easier way to do customisation.

Wrapper cookbooks are too heavyweight for my taste and using them to modify a template location doesn't feel right....at least it is not easy enough with today's solutions.

But I guess I'm just late to the party :-)

[danielsdeleo]

@jweiss we can chat at chefconf for sure. It just occurred to me that we could do something like have an explicit overlays directory that you configure, which might be okay. My problem with the feature is:

1. Too easy to use it accidentally and upload something totally different than what you wanted to the server. IOW, newbie footgun.
2. Very opaque. Only at cookbook upload time (or chef-solo runtime) is the combined cookbook actually synthesized, so even for experienced users you have to figure out what the synthesized cookbook will look like based on your mental model of how the feature works. I believe that more transparency is needed when the user is given opportunities to make mistakes.

[jkeiser]

@danielsdeleo @jweiss did you guys get to talk at ChefConf? What is your current thinking?

[danielsdeleo]

Nope, didn't get to chat. I'd still like to remove or rewrite the overlays feature. The two ways I could see it working are:

1. Have a tool that builds the combined cookbook for you. Then you keep a pristine copy in one place, and your set of overlays in a different place and run something like cookbook-combiner-tool COOKBOOK_NAME to merge them. The good part of this approach is that chef/knife/etc. don't have to handle any of that complexity, it's explicit on the user's part, and it produces a final artifact that you can inspect and understand how it works. The downside is that you have to re-run the combiner command when iterating on a overlay file.
2. Support overlays, but require users to put the overlay files in some sort of overlay-specific location. For example, you might have a cookbooks/ and cookbook-overlays directory, and the second is not part of the cookbook_path setting but is configured as a cookbook_overlays_path or something. The tradeoffs are basically the opposite of approach 1.

[jweiss]

As a workaround we solved it by copying the files before the chef run. So we get back the chef-solo behaviour.

In general, I like suggestion 2).

From my experience overlays are a simple way to customise cookbooks you grabbed from somewhere. Many cookbook authors don't go as far as creating definitions, LWRPs etc.

[thommay]

Given that there's now a supported API to modify resources such that one can point to different template files and so on, and we still don't want to support merging, I'm going to close this issue.
Thanks,
-Thom