Add 'recipe' project type to Backdrop

[ghost]

This is a spin-off of backdrop-ops/backdropcms.org#46 specifically for proposing a new project type for Backdrop: recipes.

With Backdrop supporting modules for functionality, themes for styling, layouts for, well, layout and install profiles for the initial configuration and setup of a site, I see the addition of a 'recipe' as combining all these separate things into a bundled package of sorts.

Use cases include:

• Making it possible to port certain (cough, WordPress) themes to Backdrop - ones that include styling, a custom layout, and features like sliders, etc. (e.g. a theme, a layout and one or more modules).
• Providing a full demo site to go along with a tutorial (users install the recipe and get all the modules, content types, themes, etc. setup automatically).
• Setting up a new site with things installed and configured the way you (as a developer) normally like them (e.g. simplifying the initial setup of multiple, similar sites).

The reason I like the term 'recipe' (which @klonos originally suggested in the issue linked above) is that 'package' (my original suggestion) implies something that you open to find other things inside, whereas 'recipe' is a bunch of ingredients which, when combined, create something new. I think recipes could be as simple as a single info file, such as:

name = My demo
description = Sets up a Backdrop site for demoing.
backdrop = 1.x # Not sure if this line is necessary...
type = recipe

modules[] = devel
modules[] = webform
modules[] = flexible_layout

themes[] = summer_fun

layouts[] = flexible_template

profiles[] = demo_profile

Or maybe even a JSON file and use it in config somehow... Either way, a simple text file should suffice. And as per info files you should be able to optionally specify versions too.

So modules provide the features/functionality, themes the styling, layouts the layout and profiles the setup (install modules, set theme as default, add blocks to a layout, add custom content types, etc.). Of course all of these would be optional (you could have a recipe of just modules, or a theme and matching layout, etc.).

---

Wiki Page Summary
Related META Issue: #5240

[laryn]

Love it!

[herbdool]

Would be good to have a proof of concept. I think it could be mocked up with a custom module, which includes the dependencies and runs some hooks to set up a baseline config. Would need to figure out how to include the theme and layout as dependencies.

[herbdool]

Would need to test out what happens if the recipe is disabled or uninstalled, or the are updates to the recipe from upstream, how to deal with divergent changes...

[Graham-72]

👍

[klonos]

Another thing to decide is where these recipes will live in the file system ...if they are actual files that is.

The reason why I say this, is because the idea behind these "recipes" was that one could include these as plain-text on say a blog post. Then the "consumer" would copy this text, and paste it into our Project Installer (via the "manual installation" dialog - or some similar UI):

^^ that's a mockup 😄

So basically, when I thought of this initially, I have not factored in any new project type, .info files, or versioning. Not implying that that would not be the way to go; just saying.

[ghost]

Hmm, yes, I think that's a good idea. We want to make it as easy as possible for people to create and use recipes, so not having to write info/json files and 'install' them would be handy. That'll also solve @herbdool's problem of disabling/uninstalling recipes and I think would make for a simpler setup all-round.

[klonos]

I am adding the "needs feedback" label. If people agree that this should not be another project type after all, then we can rename this issue accordingly.

[laryn]

It sounds like a recipe could simply be an install script for a bunch of things, without those things being tracked via the recipe going forward. (ie. you use the recipe to make a cake, but the cake doesn't change next week if you later adjust the recipe).

That makes sense to me.

Would it also be possible to load example configuration and content via a recipe, or would that have to be done through a module that the recipe installed? (e.g. getting a shell of a site built out quickly with some sample content, blocks in place on certain layouts, etc.)

[ghost]

@laryn I think install profiles are still the best place for that.

[ghost]

I think I'm starting to get confused... I'm now wondering how recipes will be any different than the 'INSTALL PROJECTS BY NAME' field in @klonos's screenshot above...? If you can enter the names of modules, layouts and themes there and have them all installed for you (with dependencies, etc.) then isn't that what a recipe would have been essentially anyway? And if that field's only for modules, then perhaps a simple feature request to expand that to layouts and themes as well would suffice.

Unless I'm missing something and there's more to the idea of recipes than that...?

[jenlampton]

If you can enter the names of modules, layouts and themes there and have them all installed for you (with dependencies, etc.) then isn't that what a recipe would have been essentially anyway?

Maybe it becomes an info or json file, and the "instructions" just become to copy/paste it into that box?

[docwilmot]

I wasnt thinking that recipes would be entirely different from profiles, but the distinguisher would be that you could install them at any time, using the Installer UI. For example, you want a new church site, with menu and pages and so on; you select a church Recipe in Installer, and it walks you through the "recipe" for building one: you choose a theme, you choose some options, you decide on dummy content, etc, then click install and voila! a new church site. It eventually could probably have an advanced option to backup your dbase first so you can revert.

But something like that.

[findlabnet]

@docwilmot great scenario. Not church only, but restaurant, gallery or real estate agency - here we go to real users.

[jlfranklin]

How is this different from the Features module? What you're describing is close to its original purpose.

In the case of Features, someone would put together a Features module that included all the required modules as dependencies, the content types, and all the config to deploy a particular kind of site. Features went beyond that to allow a site to enable certain features -- enable the calendar or not, enable the blog or not, enable reservations (like for a restaurant), enable catering?

Features + the Installer module would allow someone to upload just the feature module and then download any additional modules they may need, enable all of it, and have a "pre-built" site.

[docwilmot]

How is this different from the Features module? What you're describing is close to its original purpose.

this is a reasonable point. Port Features instead?

[herbdool]

Sure, port Features... if you want to enter a world of pain. 😋 I imagine D7 version can't easily be converted to use config instead of the db as a source. Though maybe I'm wrong.

[jlfranklin]

Sure, port Features... if you want to enter a world of pain. 😋

Same question still applies: How is this different from Features? Specifically, how does it avoid the pain points that Features suffers?

Part of the problem with Features is people tried to use it for configuration management, not feature definitions. Re-applying a feature was never really part of its design, nor was removing something, like a field in a content type. Reapplying sometimes worked, and sometimes it didn't (often with hilarious results.) You could remove something from the feature module, but all Features would do is orphan it on the site.

Features itself had no concept of a migration path. A fantastic dev team could write migrations via hook_update_N(), but the amount of testing it required for an automation that would run exactly once... it was nearly always faster to just write down the migration steps in the ticket.

[jenlampton]

How is this different from Features?

The only real difference is that we'd be saving config, which Backdrop knows how to handle, not some crazy one-of thing that Drupal didn't really understand.

how does it avoid the pain points that Features suffers?

Features was painful because Drupal didn't really understand how to store things that are configuration anywhere other than in the database. This would avoid that because It's Backdrop, and not storing anything that's config in the database.

Part of the problem with Features is people tried to use it for configuration management,

This won't be a use-case for Backdrop. We'd only be using features for it's intended purpose: Packaging config together.

Sure, port Features... if you want to enter a world of pain

I was envisioning we'd build something completely new, and name it "Features". But a port miiiiight be possible?

[stpaultim]

I'm interested in this issue. I've started playing with a simple repo of config recipes: https://github.com/TeamTriplo/backdropcms-features
https://github.com/backdrop-contrib/config_recipes

So far, I have two recipes/features. This is not an ideal system, because it requires that I load each config file separately and in the right order. It's not bad, but there must be another way.

Should I make my config recipes a contrib project? I am not sure if that his the right place for them?

EDIT: After a discussion during a dev meeting, I went ahead and created a github project for this experiment.

[klonos]

That's an interesting subject for the dev meeting @stpaultim. On the one hand, I don't like the idea of "polluting" contrib with recipes, but on the other, d.org did allow for distributions, which is the equivalent of (or similar to) what we are talking about here.

I am interested in this as well.

[stpaultim]

We talked about this on todays DEV call. As things currently stand, a user can ONLY input all config at once or one file at a time. It would be helpful if a user could enter a batch of config files that make up a recipe.

It is possible to export the current config directory, add a batch/recipe to the entire directory, and then import the entire batch.

Would it be possible (relatively easily possible) to add an option that automates this. An UI that allows the user to add a directory of config files and then have Backdrop combine the current files with the new batch?

(OK, now that I write it out, this is probably no easier than simply coming up with a way to add a batch of config on their own. I guess, the combining with active config files is only necessary because of the lack of the alternative UI).

[jenlampton]

Would it be possible (relatively easily possible) to add an option that automates this. A UI that allows the user to add a directory of config files and then have Backdrop combine the current files with the new batch?

Yes. This is exactly what I want in core :)

[stpaultim]

I documented my current process in the forum, which is cumbersome.
https://forum.backdropcms.org/forum/how-add-bundle-or-group-config-files-site

I hope we can make progress on this issue.

Here is a related issue:
#3933 (CLOSED) in favor of #661

AND - I created a github contrib project for a very simple library of config recipes:
https://github.com/backdrop-contrib/config-recipes

[stpaultim]

I think this is related: Allow config files to have dependencies #4188

[stpaultim]

@BWpanda - Thanks for that great summary. This issue is getting some traction in our priority survey, yet it occurs to me that we don't even know what a recipe will look like yet.

I agree with the following points: A recipe should be added to a module post install. It's a one-time thing that is not updated, maintained, or disabled. (The downside is that this makes it difficult to test a recipe to see if it does what you want). It's about adding a "feature" to a website (content types, fields, blocks, and views). A recipe may be dependent upon certain contrib modules or layouts.

It's not clear to me that a recipe should be dependent upon a theme. I think it would be best if recipes were independent of themes (that they should work on all or most themes). However, I can see where theme related dependencies might be necessary.

A few random thoughts:

1. Instead of being dependent upon a theme, maybe a recipe should be allowed to contain some custom CSS. In which case, maybe it should be possible to disable the recipe (to disable the custom css).

2. I could envision a scenario in which config files are included in the project repo, but that the project is uploaded using the config manager rather than the project browser (especially if they are upload only without need to disable).

3. This project https://github.com/backdrop-contrib/config_recipes has some simple sample recipes. The downside is that there is no way current way to require dependencies (modules) in these recipes.

[philsward]

Would a recipe automatically download the dependency modules? My vote is yes.

[klonos]

Would a recipe automatically download the dependency modules

Yep. That's the goal: run a recipe -> all necessary components and their dependencies get downloaded and configured for you.

Instead of being dependent upon a theme, maybe a recipe should be allowed to contain some custom CSS

We could allow recipes to include optional components, which would show checkboxes to allow selecting or skipping them.

[philsward]

Thanks. It was kind of implied; just wanted clarification.

On a side note, it was mentioned to have recipes use a .info file to hold the recipe information. What about a .recipe file? Maybe .reci?

[laryn]

I've had stewing in the back of my mind a module for Paragraphs (I've been calling it "Prefab Paragraphs") that would allow people to click a button to configure a paragraph type -- but perhaps a core "recipe" module would allow some way to do this in a module? e.g. in this case, a recipe for a content type with certain paragraph types pre-configured, or just pre-configuring paragraph types in general (some of which may have dependencies on other modules).

[philsward]

Any discussion on what to do if a machine name used by a recipe is already there?

Instead of a hard stop, I would imagine a page that would show the existing machine names and allow the admin to create a different one before continuing.

Example:
This recipe wants to create a content type called "Page" with a machine name "page" but this already exists. Enter a new machine name to continue:

[stpaultim]

Any discussion on what to do if a machine name used by a recipe is already there?
Instead of a hard stop, I would imagine a page that would show the existing machine names and allow the admin to create a different one before continuing.

This might be getting a little ahead of ourselves. The low tech solution is to give everything a machine name with a unique prefix that would help prevent duplication. But, there might be better solutions to this problem. I'm sure that others have thought about this more than me. It's a common problem in modular development.

NOTE: I created a Wikipage to help keep a summary of where this discussion is going.
https://github.com/backdrop/backdrop-issues/wiki/New-%22Recipe%22-Project-Type

[klonos]

I created a Wikipage to help keep a summary of where this discussion is going.

Thanks for doing that @stpaultim 🙏

[laryn]

This is taking a step further back, but what if this isn't an entirely new project type, but more of a side-car to the existing configuration management? For example:

• A menu item here: Configuration > Development > Configuration Management > Recipes
• Core could provide some recipes by default
• Modules could optionally include a recipes folder (or a config/recipes folder) that automatically provided additional recipes when a module was installed.

I'm not sure if that would help or not, but the thought just occurred to me that it could possibly simplify things for the end user and make it easy for modules to include recipes without making a completely separate project.

[klonos]

• Core could provide some recipes by default

👍 ...I've always wanted core to provide an installer/wizard-like, step-by-step guide to make a site multilingual ...so basically:

• enable required modules
• ask which languages to enable ([UX] Allow bulk-add/remove of languages #4902)
• ask which content types (or other entities) to enable multilingual support for
• magic! 🌟 🧚 🍻 🎉 😌
• present a screen with links to customize further, import language files, documentation etc.

I think that that could be a good use case for that, and it would provide value to Backdrop as a product.

[philsward]

This is taking a step further back, but what if this isn't an entirely new project type, but more of a side-car to the existing configuration management?

I think modules should be able to include recipes but I also feel like they need to stand on their own and be a community driven project.

"If" they are structured well enough that they are easy to create, non-dev people could help contribute their own recipes.

Now THIS idea is pretty far out there, but a "recipe builder" would be a fantastic way to deal with this. A person could create a playground site, set it up almost exactly the way they want it, then create a recipe from it. Kind of like the config management but without any specific data.

[stpaultim]

We had a discussion about "Config Recipes" at last weekends Backdrop LIVE. Here are notes that were taken.
https://docs.google.com/document/d/1ia-ynczOfNsAUPiD1byTDyub7MWbF7CfLyv-Y26djJM/edit

Also - one notable takeaway from the discussion, was that one can actually create a module with nothing but a .info file and config files, which may in fact be the first step towards a MVP recipe. Atten: @hosef

I have yet to test it, but it may provide the same functionality as: https://github.com/backdrop-contrib/config_batch_upload and https://github.com/backdrop-contrib/config_recipes (but the module solutions is easier).

The downside to the module solution is that Backdrop has not way to tell this recipe type module from a regular module.

[laryn]

Also - one notable takeaway from the discussion, was that one can actually create a module with nothing but a .info file and config files, which may in fact be the first step towards a MVP recipe. Atten: @hosef

This old comment was what came vaguely to mind during the discussion. I dug it up again -- maybe @quicksketch can comment again on this in light of the recipe discussion?
#571 (comment)

[hosef]

@laryn hmm, that is interesting. It seems like we should be calling hooks when we import files from modules. Otherwise, you can't really be sure that fields had their database tables set up properly.

[hosef]

Apparently this has been reported as a bug before. #3224

[stpaultim]

We are nearing completion of a bug fix that will make "config recipes" possible in Backdrop.
#3224

The original idea for recipes introduced by @BWpanda is a bit more ambitious than what we have been talking about recently. The current test recipe (https://github.com/backdrop-contrib/testimonial_recipe) that I'm working with to test this other bug fix is a module with nothing but config files. I see no reason that this module could not also include module dependencies. Is it currently possible to make a module dependent upon a theme?

Anyway, we could have the basic building blocks in places for recipes 1.0 for the next release. However, we have to work out the following questions:

1. Do we want recipes that do more than simply install config files?
2. How do we want to manage them on BackdropCMS.org?

[stpaultim]

I've create a second test recipe.
https://github.com/stpaultim/faq_recipe

(EDIT: Now in Backdrop Contrib at https://github.com/backdrop-contrib/faq_recipe)

[stpaultim]

See #5173 for a discussion about a possible problem including config for modules in a recipe.

[stpaultim]

Discussion today during meeting about what a recipe should like: https://youtu.be/CoXHmSUGsXg?t=1632

Should they just be modules or should they be something else. Consensus seems to be leaning strongly towards them being modules, but I am not clear that a decision has been made yet.

The sense is that with the recent bug fix, we're better positioned to experiment with recipes. But, that it might be premature to write the press release announcing it as a new feature.

[ericfoy]

I'll throw my opinion in...
From a position of empathy with the Backdrop Customer (the site builder),

A Recipe should be a separate type of installable from a module, thus it should be categorized and listed separately from modules.

This is because of the following perceptions (some of which might be incorrect, yet useful, nonetheless, for one's understanding):

1. In general, a recipe may depend on one or more modules, but modules do not depend on recipes.
2. A module adds persistent structural functionality to the website,whereas a recipe simply preconfigures said structure with certain features.
3. A module adds stuff that is impossible to add through the administrative UI, whereas a recipe adds stuff that one could add himself through the UI.

I would even go so far as to say that this (or a similar) set of rules should be put in place in order to clearly delineate and define a recipe, and to guide and constrain the development of recipes.

"Install a module to extend Backdrop,
Install a recipe to utilize Backdrop."

[klonos]

Thanks @ericfoy 🙏🏼 (and apologies for the late reply here). In #6595, I outlined the following (among other things):

... What I would like recipes to be is "code-less scripts" or natural-language (sorta) "instructions". ... I would like anyone that can read through a how-to article or watch a tutorial video to be able to convert it to machine-readable, yet plain-language (as much as possible) instructions set with very minimal effort, and no code (almost no code - writing a .json or .info file is still "code"). I want recipes to resemble as much as possible an actual food recipe, and have its simplicity:

• start with a name for the "dish"
• add a section of required "ingredients" (dependencies on modules/themes/templates, which the project installer can pull and install)
• provide the actual recipe instructions (steps outlined in the .info file ideally, which will be followed/executed sequentially)
• sprinkle some "seasoning" if required (things like CSS - possibly via modules like css_injector, which will have already been added as one of the "ingredients" in step 2)