Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'recipe' project type to Backdrop #3763

Open
BWPanda opened this issue May 11, 2019 · 50 comments
Open

Add 'recipe' project type to Backdrop #3763

BWPanda opened this issue May 11, 2019 · 50 comments

Comments

@BWPanda
Copy link
Member

@BWPanda BWPanda commented May 11, 2019

This is a spin-off of backdrop-ops/backdropcms.org#46 specifically for proposing a new project type for Backdrop: recipes.

With Backdrop supporting modules for functionality, themes for styling, layouts for, well, layout and install profiles for the initial configuration and setup of a site, I see the addition of a 'recipe' as combining all these separate things into a bundled package of sorts.

Use cases include:

  • Making it possible to port certain (cough, WordPress) themes to Backdrop - ones that include styling, a custom layout, and features like sliders, etc. (e.g. a theme, a layout and one or more modules).
  • Providing a full demo site to go along with a tutorial (users install the recipe and get all the modules, content types, themes, etc. setup automatically).
  • Setting up a new site with things installed and configured the way you (as a developer) normally like them (e.g. simplifying the initial setup of multiple, similar sites).

The reason I like the term 'recipe' (which @klonos originally suggested in the issue linked above) is that 'package' (my original suggestion) implies something that you open to find other things inside, whereas 'recipe' is a bunch of ingredients which, when combined, create something new. I think recipes could be as simple as a single info file, such as:

name = My demo
description = Sets up a Backdrop site for demoing.
backdrop = 1.x # Not sure if this line is necessary...
type = recipe

modules[] = devel
modules[] = webform
modules[] = flexible_layout

themes[] = summer_fun

layouts[] = flexible_template

profiles[] = demo_profile

Or maybe even a JSON file and use it in config somehow... Either way, a simple text file should suffice. And as per info files you should be able to optionally specify versions too.

So modules provide the features/functionality, themes the styling, layouts the layout and profiles the setup (install modules, set theme as default, add blocks to a layout, add custom content types, etc.). Of course all of these would be optional (you could have a recipe of just modules, or a theme and matching layout, etc.).


Wiki Page Summary
Related META Issue: #5240

@laryn
Copy link
Contributor

@laryn laryn commented May 11, 2019

Love it!

@herbdool
Copy link

@herbdool herbdool commented May 11, 2019

Would be good to have a proof of concept. I think it could be mocked up with a custom module, which includes the dependencies and runs some hooks to set up a baseline config. Would need to figure out how to include the theme and layout as dependencies.

@herbdool
Copy link

@herbdool herbdool commented May 11, 2019

Would need to test out what happens if the recipe is disabled or uninstalled, or the are updates to the recipe from upstream, how to deal with divergent changes...

@Graham-72
Copy link

@Graham-72 Graham-72 commented May 12, 2019

👍

@klonos
Copy link
Member

@klonos klonos commented May 13, 2019

Another thing to decide is where these recipes will live in the file system ...if they are actual files that is.

The reason why I say this, is because the idea behind these "recipes" was that one could include these as plain-text on say a blog post. Then the "consumer" would copy this text, and paste it into our Project Installer (via the "manual installation" dialog - or some similar UI):

Screen Shot 2019-05-13 at 2 05 08 pm

^^ that's a mockup 😄

So basically, when I thought of this initially, I have not factored in any new project type, .info files, or versioning. Not implying that that would not be the way to go; just saying.

@BWPanda
Copy link
Member Author

@BWPanda BWPanda commented May 13, 2019

Hmm, yes, I think that's a good idea. We want to make it as easy as possible for people to create and use recipes, so not having to write info/json files and 'install' them would be handy. That'll also solve @herbdool's problem of disabling/uninstalling recipes and I think would make for a simpler setup all-round.

@klonos
Copy link
Member

@klonos klonos commented May 13, 2019

I am adding the "needs feedback" label. If people agree that this should not be another project type after all, then we can rename this issue accordingly.

@laryn
Copy link
Contributor

@laryn laryn commented May 14, 2019

It sounds like a recipe could simply be an install script for a bunch of things, without those things being tracked via the recipe going forward. (ie. you use the recipe to make a cake, but the cake doesn't change next week if you later adjust the recipe).

That makes sense to me.

Would it also be possible to load example configuration and content via a recipe, or would that have to be done through a module that the recipe installed? (e.g. getting a shell of a site built out quickly with some sample content, blocks in place on certain layouts, etc.)

@BWPanda
Copy link
Member Author

@BWPanda BWPanda commented May 15, 2019

@laryn I think install profiles are still the best place for that.

@BWPanda
Copy link
Member Author

@BWPanda BWPanda commented May 15, 2019

I think I'm starting to get confused... I'm now wondering how recipes will be any different than the 'INSTALL PROJECTS BY NAME' field in @klonos's screenshot above...? If you can enter the names of modules, layouts and themes there and have them all installed for you (with dependencies, etc.) then isn't that what a recipe would have been essentially anyway? And if that field's only for modules, then perhaps a simple feature request to expand that to layouts and themes as well would suffice.

Unless I'm missing something and there's more to the idea of recipes than that...?

@jenlampton
Copy link
Member

@jenlampton jenlampton commented May 17, 2019

If you can enter the names of modules, layouts and themes there and have them all installed for you (with dependencies, etc.) then isn't that what a recipe would have been essentially anyway?

Maybe it becomes an info or json file, and the "instructions" just become to copy/paste it into that box?

@docwilmot
Copy link
Contributor

@docwilmot docwilmot commented May 30, 2019

I wasnt thinking that recipes would be entirely different from profiles, but the distinguisher would be that you could install them at any time, using the Installer UI. For example, you want a new church site, with menu and pages and so on; you select a church Recipe in Installer, and it walks you through the "recipe" for building one: you choose a theme, you choose some options, you decide on dummy content, etc, then click install and voila! a new church site. It eventually could probably have an advanced option to backup your dbase first so you can revert.

But something like that.

@findlabnet
Copy link

@findlabnet findlabnet commented May 30, 2019

@docwilmot great scenario. Not church only, but restaurant, gallery or real estate agency - here we go to real users.

@jlfranklin
Copy link
Member

@jlfranklin jlfranklin commented May 30, 2019

How is this different from the Features module? What you're describing is close to its original purpose.

In the case of Features, someone would put together a Features module that included all the required modules as dependencies, the content types, and all the config to deploy a particular kind of site. Features went beyond that to allow a site to enable certain features -- enable the calendar or not, enable the blog or not, enable reservations (like for a restaurant), enable catering?

Features + the Installer module would allow someone to upload just the feature module and then download any additional modules they may need, enable all of it, and have a "pre-built" site.

@docwilmot
Copy link
Contributor

@docwilmot docwilmot commented May 30, 2019

How is this different from the Features module? What you're describing is close to its original purpose.

this is a reasonable point. Port Features instead?

@herbdool
Copy link

@herbdool herbdool commented May 30, 2019

Sure, port Features... if you want to enter a world of pain. 😋 I imagine D7 version can't easily be converted to use config instead of the db as a source. Though maybe I'm wrong.

@jlfranklin
Copy link
Member

@jlfranklin jlfranklin commented May 30, 2019

Sure, port Features... if you want to enter a world of pain. 😋

Same question still applies: How is this different from Features? Specifically, how does it avoid the pain points that Features suffers?

Part of the problem with Features is people tried to use it for configuration management, not feature definitions. Re-applying a feature was never really part of its design, nor was removing something, like a field in a content type. Reapplying sometimes worked, and sometimes it didn't (often with hilarious results.) You could remove something from the feature module, but all Features would do is orphan it on the site.

Features itself had no concept of a migration path. A fantastic dev team could write migrations via hook_update_N(), but the amount of testing it required for an automation that would run exactly once... it was nearly always faster to just write down the migration steps in the ticket.

@jenlampton
Copy link
Member

@jenlampton jenlampton commented May 30, 2019

How is this different from Features?

The only real difference is that we'd be saving config, which Backdrop knows how to handle, not some crazy one-of thing that Drupal didn't really understand.

how does it avoid the pain points that Features suffers?

Features was painful because Drupal didn't really understand how to store things that are configuration anywhere other than in the database. This would avoid that because It's Backdrop, and not storing anything that's config in the database.

Part of the problem with Features is people tried to use it for configuration management,

This won't be a use-case for Backdrop. We'd only be using features for it's intended purpose: Packaging config together.

Sure, port Features... if you want to enter a world of pain

I was envisioning we'd build something completely new, and name it "Features". But a port miiiiight be possible?

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Oct 10, 2019

I'm interested in this issue. I've started playing with a simple repo of config recipes: https://github.com/TeamTriplo/backdropcms-features
https://github.com/backdrop-contrib/config_recipes

So far, I have two recipes/features. This is not an ideal system, because it requires that I load each config file separately and in the right order. It's not bad, but there must be another way.

Should I make my config recipes a contrib project? I am not sure if that his the right place for them?

EDIT: After a discussion during a dev meeting, I went ahead and created a github project for this experiment.

@klonos
Copy link
Member

@klonos klonos commented Oct 10, 2019

That's an interesting subject for the dev meeting @stpaultim. On the one hand, I don't like the idea of "polluting" contrib with recipes, but on the other, d.org did allow for distributions, which is the equivalent of (or similar to) what we are talking about here.

I am interested in this as well.

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Oct 18, 2019

We talked about this on todays DEV call. As things currently stand, a user can ONLY input all config at once or one file at a time. It would be helpful if a user could enter a batch of config files that make up a recipe.

It is possible to export the current config directory, add a batch/recipe to the entire directory, and then import the entire batch.

Would it be possible (relatively easily possible) to add an option that automates this. An UI that allows the user to add a directory of config files and then have Backdrop combine the current files with the new batch?

(OK, now that I write it out, this is probably no easier than simply coming up with a way to add a batch of config on their own. I guess, the combining with active config files is only necessary because of the lack of the alternative UI).

@jenlampton
Copy link
Member

@jenlampton jenlampton commented Oct 18, 2019

Would it be possible (relatively easily possible) to add an option that automates this. A UI that allows the user to add a directory of config files and then have Backdrop combine the current files with the new batch?

Yes. This is exactly what I want in core :)

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Oct 18, 2019

I documented my current process in the forum, which is cumbersome.
https://forum.backdropcms.org/forum/how-add-bundle-or-group-config-files-site

I hope we can make progress on this issue.

Here is a related issue:
#3933 (CLOSED) in favor of #661

AND - I created a github contrib project for a very simple library of config recipes:
https://github.com/backdrop-contrib/config-recipes

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Nov 1, 2019

@klonos
Copy link
Member

@klonos klonos commented Jan 7, 2020

During the VUG earlier today, @stpaultim mentioned that things like the need to have dependencies declared, as well as being able to specify the order in which config files get imported, often becomes a blocker (some .json files fail to be imported). I had the following thoughts on those:

  • We could solve the dependencies issue if we had .info files for recipes (which implies them being a "proper" project type - which is what this issue here is about).

  • We need to be able to support and handle some sort of "weight" property for .json config files. Can something like this also live in the .info files?

  • How about we keep running the import until all .json files have been imported? I don't like this much, as it has the danger of getting into infinite loop situations, but we could get the config import to place failed config files in a "try again" queue, and continue with the rest. In theory, after the 1st pass, any config that does not depend on other config will have been imported properly, and the ones that have dependencies would be in this "try again" queue. In the 2nd pass, (again, in theory) most of the "try again config" will not fail, because chances are that their dependencies have been imported successfully in the first pass. ...as I said, I don't like this, but it could work as an interim, short-term solution.

Food for thought.

@BWPanda
Copy link
Member Author

@BWPanda BWPanda commented Jan 21, 2021

I've just re-read through (most of) this issue, and it seems we got sidetracked (a few times)... There are lots of different ideas and suggestions, so I'll try to summarise them and clarify what this issue should be focused on...

The are two main ideas around 'recipes':

  • Something you use to setup a new site that configures everything for you (possibly including default content)
  • Something you 'install' on an existing site to add specific functionality

Any ideas that deal with install profiles, demo sites, adding default content, etc. come under the first category. I don't believe recipes should cover this, as we already have install profiles that are very powerful (they're basically fancy modules). Install profiles can setup a new site from scratch, they can install modules, enable themes, add layouts, they can even add default configuration (e.g. Views). They can't yet add default content, but I feel like that should be a totally separate issue.

So I think recipes, as this issue defines them, come under the second category - adding functionality to an existing website. Some examples include:

  • Modules (like an accordion) and configuration (content types and views) to add an FAQs feature
  • A theme, layout and some modules to port a Wordpress 'theme' to Backdrop (which often include functionality like sliders, in addition to layouts and styling)

I think we can ditch the idea of 'recipes' being a new type of project that has a version and can be updated, uninstalled, etc. That's complicated (as discussed re. Drupal's Features module). I see recipes as being something you 'install' on an existing site - content types are created, Views are added, etc. Then the site acts just as if you had manually built those things yourself. You can disable or uninstall the individual parts. The recipe doesn't exist after it's been 'installed'. There's no 'overridden' status, 'recipe update available', etc. Recipes are just a quick way of adding things to your site without you needing to create them all manually from scratch. This also means that you can 'install' a recipe, then change it. For example, install the FAQs recipe, but then add a new field to the content type that lets you upload an icon to be displayed next to each question.

If we want to keep recipes simple and have the ability to just copy & paste them into a UI in Backdrop to 'install' them, then in order to allow for configuration to be included (e.g. views, content types, etc.), I think we need to allow for linking to externally-hosted config files. We could support GitHub, Gists, or people's own websites. For example, a recipe could look like this:

# Modules
- devel
- simplify
- webform

# Themes
- lumi

# Layouts
- bamboo

# Configuration
- https://gist.github.com/BWPanda/123456789
- https://panda.id.au/sites/panda/files/recipes/config-123.json

Modules, themes and layouts would be downloaded from B.org, and the other URLs would be accessed directly (and then checked to make sure they're actually valid Backdrop config files).

Anyway, let's try to focus this issue on that type of idea for recipes and open other issues for the rest of the desired functionality surrounding this topic.

@philsward
Copy link

@philsward philsward commented Jan 21, 2021

Idea:

Block recipes. A collection of things that could create a specific block usage.

Example: a block who's only function is to hold and display images. The recipe would create the content type (field type?) to hold the images and create the view with block to display the images.

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Jan 31, 2021

@BWPanda - Thanks for that great summary. This issue is getting some traction in our priority survey, yet it occurs to me that we don't even know what a recipe will look like yet.

I agree with the following points: A recipe should be added to a module post install. It's a one-time thing that is not updated, maintained, or disabled. (The downside is that this makes it difficult to test a recipe to see if it does what you want). It's about adding a "feature" to a website (content types, fields, blocks, and views). A recipe may be dependent upon certain contrib modules or layouts.

It's not clear to me that a recipe should be dependent upon a theme. I think it would be best if recipes were independent of themes (that they should work on all or most themes). However, I can see where theme related dependencies might be necessary.

A few random thoughts:

  1. Instead of being dependent upon a theme, maybe a recipe should be allowed to contain some custom CSS. In which case, maybe it should be possible to disable the recipe (to disable the custom css).

  2. I could envision a scenario in which config files are included in the project repo, but that the project is uploaded using the config manager rather than the project browser (especially if they are upload only without need to disable).

  3. This project https://github.com/backdrop-contrib/config_recipes has some simple sample recipes. The downside is that there is no way current way to require dependencies (modules) in these recipes.

@philsward
Copy link

@philsward philsward commented Jan 31, 2021

Would a recipe automatically download the dependency modules? My vote is yes.

@klonos
Copy link
Member

@klonos klonos commented Jan 31, 2021

Would a recipe automatically download the dependency modules

Yep. That's the goal: run a recipe -> all necessary components and their dependencies get downloaded and configured for you.

Instead of being dependent upon a theme, maybe a recipe should be allowed to contain some custom CSS

We could allow recipes to include optional components, which would show checkboxes to allow selecting or skipping them.

@philsward
Copy link

@philsward philsward commented Jan 31, 2021

Thanks. It was kind of implied; just wanted clarification.

On a side note, it was mentioned to have recipes use a .info file to hold the recipe information. What about a .recipe file? Maybe .reci?

@laryn
Copy link
Contributor

@laryn laryn commented Feb 1, 2021

I've had stewing in the back of my mind a module for Paragraphs (I've been calling it "Prefab Paragraphs") that would allow people to click a button to configure a paragraph type -- but perhaps a core "recipe" module would allow some way to do this in a module? e.g. in this case, a recipe for a content type with certain paragraph types pre-configured, or just pre-configuring paragraph types in general (some of which may have dependencies on other modules).

@philsward
Copy link

@philsward philsward commented Feb 1, 2021

Any discussion on what to do if a machine name used by a recipe is already there?

Instead of a hard stop, I would imagine a page that would show the existing machine names and allow the admin to create a different one before continuing.

Example:
This recipe wants to create a content type called "Page" with a machine name "page" but this already exists. Enter a new machine name to continue:

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Feb 2, 2021

Any discussion on what to do if a machine name used by a recipe is already there?
Instead of a hard stop, I would imagine a page that would show the existing machine names and allow the admin to create a different one before continuing.

This might be getting a little ahead of ourselves. The low tech solution is to give everything a machine name with a unique prefix that would help prevent duplication. But, there might be better solutions to this problem. I'm sure that others have thought about this more than me. It's a common problem in modular development.

NOTE: I created a Wikipage to help keep a summary of where this discussion is going.
https://github.com/backdrop/backdrop-issues/wiki/New-%22Recipe%22-Project-Type

@klonos
Copy link
Member

@klonos klonos commented Feb 2, 2021

I created a Wikipage to help keep a summary of where this discussion is going.

Thanks for doing that @stpaultim 🙏

@laryn
Copy link
Contributor

@laryn laryn commented Feb 3, 2021

This is taking a step further back, but what if this isn't an entirely new project type, but more of a side-car to the existing configuration management? For example:

  • A menu item here: Configuration > Development > Configuration Management > Recipes
  • Core could provide some recipes by default
  • Modules could optionally include a recipes folder (or a config/recipes folder) that automatically provided additional recipes when a module was installed.

I'm not sure if that would help or not, but the thought just occurred to me that it could possibly simplify things for the end user and make it easy for modules to include recipes without making a completely separate project.

@klonos
Copy link
Member

@klonos klonos commented Feb 3, 2021

  • Core could provide some recipes by default

👍 ...I've always wanted core to provide an installer/wizard-like, step-by-step guide to make a site multilingual ...so basically:

  • enable required modules
  • ask which languages to enable (#4902)
  • ask which content types (or other entities) to enable multilingual support for
  • magic! 🌟 🧚 🍻 🎉 😌
  • present a screen with links to customize further, import language files, documentation etc.

I think that that could be a good use case for that, and it would provide value to Backdrop as a product.

@philsward
Copy link

@philsward philsward commented Feb 4, 2021

This is taking a step further back, but what if this isn't an entirely new project type, but more of a side-car to the existing configuration management?

I think modules should be able to include recipes but I also feel like they need to stand on their own and be a community driven project.

"If" they are structured well enough that they are easy to create, non-dev people could help contribute their own recipes.

Now THIS idea is pretty far out there, but a "recipe builder" would be a fantastic way to deal with this. A person could create a playground site, set it up almost exactly the way they want it, then create a recipe from it. Kind of like the config management but without any specific data.

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Mar 15, 2021

We had a discussion about "Config Recipes" at last weekends Backdrop LIVE. Here are notes that were taken.
https://docs.google.com/document/d/1ia-ynczOfNsAUPiD1byTDyub7MWbF7CfLyv-Y26djJM/edit

Also - one notable takeaway from the discussion, was that one can actually create a module with nothing but a .info file and config files, which may in fact be the first step towards a MVP recipe. Atten: @hosef

I have yet to test it, but it may provide the same functionality as: https://github.com/backdrop-contrib/config_batch_upload and https://github.com/backdrop-contrib/config_recipes (but the module solutions is easier).

The downside to the module solution is that Backdrop has not way to tell this recipe type module from a regular module.

@laryn
Copy link
Contributor

@laryn laryn commented Mar 15, 2021

Also - one notable takeaway from the discussion, was that one can actually create a module with nothing but a .info file and config files, which may in fact be the first step towards a MVP recipe. Atten: @hosef

This old comment was what came vaguely to mind during the discussion. I dug it up again -- maybe @quicksketch can comment again on this in light of the recipe discussion?
#571 (comment)

@hosef
Copy link

@hosef hosef commented Mar 16, 2021

@laryn hmm, that is interesting. It seems like we should be calling hooks when we import files from modules. Otherwise, you can't really be sure that fields had their database tables set up properly.

@hosef
Copy link

@hosef hosef commented Mar 16, 2021

Apparently this has been reported as a bug before. #3224

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Aug 23, 2021

We are nearing completion of a bug fix that will make "config recipes" possible in Backdrop.
#3224

The original idea for recipes introduced by @BWPanda is a bit more ambitious than what we have been talking about recently. The current test recipe (https://github.com/backdrop-contrib/testimonial_recipe) that I'm working with to test this other bug fix is a module with nothing but config files. I see no reason that this module could not also include module dependencies. Is it currently possible to make a module dependent upon a theme?

Anyway, we could have the basic building blocks in places for recipes 1.0 for the next release. However, we have to work out the following questions:

  1. Do we want recipes that do more than simply install config files?
  2. How do we want to manage them on BackdropCMS.org?

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Aug 24, 2021

I've create a second test recipe.
https://github.com/stpaultim/faq_recipe

(EDIT: Now in Backdrop Contrib at https://github.com/backdrop-contrib/faq_recipe)

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Aug 24, 2021

See #5173 for a discussion about a possible problem including config for modules in a recipe.

@stpaultim
Copy link
Member

@stpaultim stpaultim commented Aug 27, 2021

Discussion today during meeting about what a recipe should like: https://youtu.be/CoXHmSUGsXg?t=1632

Should they just be modules or should they be something else. Consensus seems to be leaning strongly towards them being modules, but I am not clear that a decision has been made yet.

The sense is that with the recent bug fix, we're better positioned to experiment with recipes. But, that it might be premature to write the press release announcing it as a new feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet