New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dialects in Chef #71

Merged
merged 4 commits into from Jan 23, 2015

Conversation

Projects
None yet
9 participants
@coderanger
Copy link
Contributor

coderanger commented Nov 21, 2014

Have at it. There was a lot of discussion about this last time, and the agreement was that while the framework would be added to core, only Ruby and JSON dialects would be included to match existing functionality. All new dialects will be done in external gems/cookbooks until such time as they can prove their worth in battle, as it were.

I think adapting the old patches would be a good step, which covers recipes, attributes, and non-chef_fs loading of roles/data bags/environments. The chef_fs support can build on that, along with template dialects to support things other than Erb.

Ping @sersut @danielsdeleo @sdelano @lamont-granquist

@danielsdeleo

This comment has been minimized.

Copy link
Member

danielsdeleo commented Nov 21, 2014

My opinion on this is the same as before. Overall I dislike the idea because:

  • I am not convinced that it will make the new user experience better. For example, if I'm familiar with python and I decide to get started with Chef, now I have to choose between using a python dialect thing that will be less tested, supported, and documented, or to use the "standard" Ruby DSL. This extra choice alone adds friction to the getting started process.
  • Other languages will necessarily be second class citizens, no matter what. Can you write LWRPs in other languages? Access Chef internals? Understand what causes a bug in a 3rd party library written in the Ruby DSL?
  • It vastly increases the scope of what you may have to know to be successful with Chef. What happens if you're the ops guy and you have to deal with recipes written by developers in 3 or 4 languages? Or if we allow cookbooks in other languages on the community site and now you have a transitive dependency on a cookbook in a different language and it breaks?
  • We're not gonna put every runtime you might want in the Chef omnibus packages, which means you're back to "the bad old days" of bringing your own runtime and hoping you have compatible versions of everything.

I think you could be successful with this if you're in a very structured work environment where you never use any community cookbooks and have access to experts to help you along when you reach the limits of what's possible in your dialect. Then again, you'll probably be successful with the Ruby DSL in that environment, too.

I'll note that this brings a lot of really good code cleanup that I like, so that's a plus.

Despite having some pretty grave reservations about it, I'm not opposed to this as an experiment. In ASF voting terms, I'm -0 on it.

@lamont-granquist

This comment has been minimized.

Copy link
Contributor

lamont-granquist commented Nov 21, 2014

I basically disgree with the premise:

    As a Chef user,
    I want to write in a variety of formats,
    so that cookbook maintenance is easier.

Certainly as someone who gets bug reports about Chef, the first time that we get a Python or YAML bug report it just makes my life that much harder, not easier. For Chef users on the mailing list who don't Python and struggle with Ruby it makes it harder for them to answer questions and follow along. Now new users have more choice -- which in this case I'd actually argue is terrible in this case because the first thing you get when you start with Chef is potentially now a fully blown language war to decide which camp you want to be in (essentially this is strings-vs-symbols in attributes exploded by about 100x).

It'd be more correct to state your argument as:

   As a Chef user who knows Python,
   I'd like to be able to write cookbooks in Python

I can't really argue with that premise.

I also think the vast majority of new users have this statement as a more accurate use case:

  As a Brand New Chef user,
  I'd like to have a set of Best Practices
  So that I go from zero to useful much faster

And the feedback that people constantly give is that they're frustrated by Chef NOT giving out hard opinions on the direction to go and what constitutes Best Practices. We've been overwhelmingly criticized over being too wishy-washy and essentially putting too much choice in front of new users and telling them "whatever fits your needs". When a new user is asking simply "how should I configure a chef cookbook to set the time on my servers?" we should not be answering "well, to start with, pick a programming language, and here is everything you should consider..."

@adamhjk

This comment has been minimized.

Copy link
Contributor

adamhjk commented Nov 25, 2014

feliz-noah-dad

@thommay

This comment has been minimized.

Copy link
Collaborator

thommay commented Nov 26, 2014

Fully agree with @lamont-granquist here.

@adamhjk

This comment has been minimized.

Copy link
Contributor

adamhjk commented Dec 11, 2014

Here is the tl;dr from IRC on the Dec 11 meeting.

  • We like the implementation cleanup that happens here.
  • We like the potential for experimentation.
  • We don't like the idea of having yet more options in core Chef, and think it will bring more confusion, not less.

So, if this RFC becomes essentially in line with above, we're collectively 👍.

@jaymzh

This comment has been minimized.

Copy link
Member

jaymzh commented Dec 11, 2014

Sorry I missed the IRC meeting, I suck. FWIW< I'm with @lamont-granquist here. I like the fact that Chef gives you lots of ways (call it N) to do something, but to then multiply N by M different languages/dialects, only adds confusion, makes it hard to debug and provides less guidance for noobs.

@coderanger

This comment has been minimized.

Copy link
Contributor

coderanger commented Dec 11, 2014

The only places I would advocate for the dialects system being used to add more stuff to core Chef is making chef_fs us the same loading code as everything else. This would add support for .rb formats to a bunch of places in chef_fs, but I think thats a goal everyone agrees on overall. Other than that, the only dialects in core will be:

  • Attributes: .rb
  • Recipes: .rb
  • Templates: .erb (if templates are in the initial release at all, they weren't in my last attempt)
  • Metadata: .rb and .json (this might help resolve some of the issues between the two?)
  • Roles: .rb and .json
  • Data bags: .json
  • Environments: .rb and .json

Node files are rare enough that I'm happy to leave those out for now, but I think knife does support a Ruby DSL for those if you do from file.

@adamhjk

This comment has been minimized.

Copy link
Contributor

adamhjk commented Jan 8, 2015

Ok - @jaymzh @lamont-granquist @mcquin @tyler-ball @danielsdeleo @jonlives - unless one of you thinks we shouldn't have this extensible and puts up a convincing argument, I'm going to 🚢 this RFC.

@jaymzh

This comment has been minimized.

Copy link
Member

jaymzh commented Jan 8, 2015

Wait - the RFC still allows for arbitrary languages of recipes, which, based on your summary of the IRC meeting seems like a no-go with everyone. Did I miss something? I think having recipes being able to be written in arbitrary languages is a really bad idea for all of the reasons @lamont-granquist and @coderanger and @danielsdeleo mentioned.
👎

@coderanger

This comment has been minimized.

Copy link
Contributor

coderanger commented Jan 8, 2015

@jaymzh Only via external support, core chef will only support Ruby.

@jaymzh

This comment has been minimized.

Copy link
Member

jaymzh commented Jan 8, 2015

@coderanger I think this will be a nightmare. People will be posting recipes written in java and ocaml and whatever else. I'm all for adding rb support for the missing ones like databags, but I think allowing people to arbitrarily add their language-of-the-day cookbooks is going to be crazysauce. As @lamont-granquist pointed out, even if it's not "core chef" or fully supported, it'll still make it to the mailing list and make every newbie's life harder when they download 4 community cookbooks in 4 different languages. Someone will copy-paste code from the list archives, but it'll be python which won't work in core ruby, etc.

@coderanger

This comment has been minimized.

Copy link
Contributor

coderanger commented Jan 8, 2015

@jaymzh I think you are way overestimating things. It seems really unlikely anyone beyond me is going to invest the time to write dialects that are anything but a mild change. Writing them is Very Hard. Yes I'll probably release my JS recipe dialect (with some pretty strong warnings that it has a habit of segfaulting if you call stuff incorrectly) but I think its just way over stating things to assume instant and massive confusion. I would be honestly surprised if many people know about the feature in a years time.

@jaymzh

This comment has been minimized.

Copy link
Member

jaymzh commented Jan 8, 2015

Then why have it in the first place? Your assertion in the RFC is that people want to write in other languages, which mean certainly someone will write a python language and then now newbs have to make more difficult and unclear choices. IF you don't think anyone will use it, why add the complexity?

@coderanger

This comment has been minimized.

Copy link
Contributor

coderanger commented Jan 8, 2015

@jaymzh Because it lets me support other people better? I would like to think more people will write them, but its going to be vastly more consuming than contributing. And if the question is "Do you think you (Noah) can write a recipe dialect plugin safely which preserves the semantics and intent of Chef" I would say the answer is yes. That said, I think uptake will be much higher on the other data types. There is already another RFC that is literally wanting to add a Ruby dialect for data bags, that on its own shows interest.

@coderanger

This comment has been minimized.

Copy link
Contributor

coderanger commented Jan 8, 2015

If your argument is "we should add this to every data type except recipes because people might misuse it or get confused", then I'm not really sure that's hugely defensible. Chef recipes already give enough rope to hang yourself (and then light yourself on fire, summon a plague of locusts, etc). I spend a large part of my day answering questions from people that copy-pasta bits of recipe code in ways that won't work, I can't imagine that changing much regardless.

@lamont-granquist

This comment has been minimized.

Copy link
Contributor

lamont-granquist commented Jan 8, 2015

I'm still not so wild about writing roles in YAML and templates in Haml.

@someara

This comment has been minimized.

Copy link

someara commented Jan 8, 2015

This thread makes my head hurt.

On Thu, Jan 8, 2015 at 1:16 PM, Lamont Granquist notifications@github.com
wrote:

I'm still not so wild about writing roles in YAML and templates in Haml.


Reply to this email directly or view it on GitHub
#71 (comment).

@martinb3

This comment has been minimized.

Copy link
Contributor

martinb3 commented Jan 9, 2015

This thread is full of awesome comments I fully intend to quote later :) It seems to me like the proposal from @coderanger will result in cleaned up, more extensible code. And the disagreement is "What if people extend it in a way that makes Chef less approachable and supportable?"

I don't see this as any different than someone coming to the software or the mailing list with any other odd duck that we can't help with. And I think there is tons of prior art for features that aren't exposed to vanilla newcomer users of Chef (chef server ACLs feel like a good example of this).

While my opinion doesn't carry any official weight, 👍 from me. I also see it potentially being a major, major boon for Windows (and other platforms) support down the road, as allowing for other dialects could lead to experimentation and/or adoption on more platforms.

@jonlives

This comment has been minimized.

Copy link
Contributor

jonlives commented Jan 20, 2015

I still have some reservations about this opening a can of worms but with that said I have confidence in @coderanger's ability to implement and support the initial dialect he's proposing, so I'll 👍

@someara

This comment has been minimized.

Copy link

someara commented Jan 20, 2015

Dialects imply a language... and as far as I know, there is no formal
specification of Chef As A Language.

Where does Chef's "public API" begin and end?

Is it just the list of core resources on the docs page?

What about run_contexts, notification / subscriptions?
use_inline_resources? why_run?, the LWRP DSL, etc.

Are we just talking about the pre-compiled bits like metadata.rb, roles,
and data_bags?

What does it even mean to use multiple languages in a chef-client run?

Should resource_collections serialized into a neutral format? Right now its
Ruby objects.

We've already seen things like POSHChef pop up that implement a parts of
Chef, how do they interact?

-s

On Tue, Jan 20, 2015 at 6:05 AM, Jon Cowie notifications@github.com wrote:

I still have some reservations about this opening a can of worms but with
that said I have confidence in @coderanger https://github.com/coderanger's
ability to implement and support the initial dialect he's proposing, so
I'll [image: 👍]


Reply to this email directly or view it on GitHub
#71 (comment).

@adamhjk

This comment has been minimized.

Copy link
Contributor

adamhjk commented Jan 21, 2015

Let's focus first on the obvious: it's healthy to have a cleaner, extensible core chef. We can then experiment outside the mainline with crazy ideas, and see what sticks. I agree that my assumption is that, in this case, more isn't better. But! I could
Be persuaded by compelling working code.

Sent from my iPhone

On Jan 20, 2015, at 9:41 AM, Sean OMeara notifications@github.com wrote:

Dialects imply a language... and as far as I know, there is no formal
specification of Chef As A Language.

Where does Chef's "public API" begin and end?

Is it just the list of core resources on the docs page?

What about run_contexts, notification / subscriptions?
use_inline_resources? why_run?, the LWRP DSL, etc.

Are we just talking about the pre-compiled bits like metadata.rb, roles,
and data_bags?

What does it even mean to use multiple languages in a chef-client run?

Should resource_collections serialized into a neutral format? Right now its
Ruby objects.

We've already seen things like POSHChef pop up that implement a parts of
Chef, how do they interact?

-s

On Tue, Jan 20, 2015 at 6:05 AM, Jon Cowie notifications@github.com wrote:

I still have some reservations about this opening a can of worms but with
that said I have confidence in @coderanger https://github.com/coderanger's
ability to implement and support the initial dialect he's proposing, so
I'll [image: 👍]


Reply to this email directly or view it on GitHub
#71 (comment).


Reply to this email directly or view it on GitHub.

@adamhjk

This comment has been minimized.

Copy link
Contributor

adamhjk commented Jan 22, 2015

Merge it, RFC editors. Make sure its clear we're not accepting new dialects in core with this RFC.

@jonlives jonlives merged commit ced5756 into chef:master Jan 23, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment