Allow plugins to provide layouts and includes

[paulrobertlloyd]

Is your feature request related to a problem? Please describe.

I have built a plugin that adds collections, files extensions, asset building etc. and also points to customised Nunjucks and markdown-it libraries. This means authors using the plugin can start creating sites without having to set any of this up themselves.

Here is the plugin:

module.exports = function (eleventyConfig, pluginOptions = {}) {
  const options = require('./lib/data/options.js')(pluginOptions)

  // Libraries
  eleventyConfig.setLibrary('md', require('./lib/markdown-it.js')(options))
  eleventyConfig.setLibrary('njk', require('./lib/nunjucks.js')(eleventyConfig))

  // Collections
  eleventyConfig.addCollection('ordered', require('./lib/collections/ordered.js'))
  eleventyConfig.addCollection('sitemap', require('./lib/collections/sitemap.js'))

  // Extensions and template formats
  eleventyConfig.addExtension('scss', require('./lib/extensions/scss.js'))
  eleventyConfig.addTemplateFormats('scss')

  // Filters
  eleventyConfig.addFilter('date', require('./lib/filters/date.js'))
  eleventyConfig.addFilter('itemsFromCollection', require('./lib/filters/items-from-collection.js'))
  eleventyConfig.addFilter('itemsFromNavigation', require('./lib/filters/items-from-navigation.js'))
  eleventyConfig.addFilter('markdown', require('./lib/filters/markdown.js'))
  eleventyConfig.addFilter('noOrphans', require('./lib/filters/no-orphans.js'))
  eleventyConfig.addFilter('pretty', require('./lib/filters/pretty.js'))
  eleventyConfig.addFilter('tokenize', require('./lib/filters/tokenize.js'))

  // Global data
  eleventyConfig.addGlobalData('options', options)
  eleventyConfig.addGlobalData('eleventyComputed', require('./lib/data/eleventy-computed.js'))

  // Passthrough
  eleventyConfig.addPassthroughCopy({
    'node_modules/govuk-frontend/govuk/assets': 'assets'
  })

  // Plugins
  eleventyConfig.addPlugin(require('@11ty/eleventy-navigation'))
  eleventyConfig.addPlugin(require('@11ty/eleventy-plugin-rss'))

  // Transforms
  eleventyConfig.addTransform('replaceGovukOpenGraphImage', require('./lib/transforms/replace-govuk-open-graph-image.js')(options))

  // Events
  eleventyConfig.on('eleventy.after', async () => {
    require('./lib/events/generate-govuk-assets.js')(eleventyConfig, options)
  })
}

The configuration API works really nicely (mostly), but one issue I’ve found is that it’s very difficult for a plugin to provide a set of layouts or includes. Right now, I ask users to do the following:

const govukEleventyPlugin = require('govuk-eleventy-plugin')

module.exports = function(eleventyConfig) {
  // Register the plugin
  eleventyConfig.addPlugin(govukEleventyPlugin)

  return {
    dataTemplateEngine: 'njk',
    htmlTemplateEngine: 'njk',
    markdownTemplateEngine: 'njk',
    dir: {
      // Use layouts from the plugin
      layouts: 'node_modules/govuk-eleventy-plugin/layouts'
    }
  }
};

This gets a bit more complicated if authors change their input directory, as the value for layouts needs to be relative to that. It also means authors can’t add their own layouts (or at least, not easily – right now they have to create ‘stub’ layouts that use Nunjucks extends feature and point to the layouts in the plugin package).

From what I’ve observed, there are 2 issues preventing a plugin from providing layouts:

1. Eleventy expects a value to be given for dir.includes (or dir.layouts), and for that directory (or the default, _includes) to exist
2. One possible avenue for registering layouts is eleventyConfig.addLayoutAlias(), but this function expects layout paths to be relative to dir.input. Adding the desired value gives an invalid path, something like ./src/_includes/./node_modules/govuk-eleventy-plugin/layouts/post.njk

Describe the solution you'd like

Two possible options:

• Allow greater flexibility around the paths eleventyConfig.addLayoutAlias() will accept (perhaps respecting paths relative to dir.input)
• Add a new method, something like eleventyConfig.addLayout(name, path) explicitly for this purpose. Additionally, should this function be called, then a value for dir.includes would no longer be required?

I’m spit-balling here… but something like this would be magic!

Additional context

Additionally, it would be helpful if for eleventyConfig.setNunjucksEnvironmentOptions() you could add search paths for layouts and includes.

Not only would that remove the need to create a custom Nunjucks environment, it might provide a means to achieving the above – although I don’t think 11ty respects search paths in a custom Nunjucks environment when looking for layouts. Maybe somewhere in this is a potential third option (though possibly too closely tied to Nunjucks, rather than being a more universal option).

[nhoizey]

It would indeed be great to be able to provide layouts in plugins with an easy API.

Would a solution be to accept an array for dir.layouts (and maybe dir.includes), and Eleventy would search layouts in multiple folders in the order of the array, until it finds one?

const govukEleventyPlugin = require('govuk-eleventy-plugin')

module.exports = function(eleventyConfig) {
  // Register the plugin
  eleventyConfig.addPlugin(govukEleventyPlugin)

  return {
    dataTemplateEngine: 'njk',
    htmlTemplateEngine: 'njk',
    markdownTemplateEngine: 'njk',
    dir: {
      // Use personalized layouts, or those from the plugin
      layouts: ['_layouts', 'node_modules/govuk-eleventy-plugin/layouts']
    }
  }
};

It would even be easier if the plugin could add it's own path to the dir.layouts array, like the addLayout() method @paulrobertlloyd mentioned.

Multiple plugins could even add layouts, but the loading order would be important.

[paulrobertlloyd]

Would a solution be to accept an array for dir.layouts (and maybe dir.includes), and Eleventy would search layouts in multiple folders in the order of the array, until it finds one?

Oooh, that might be a simpler solution! (See the note about updating Nunjucks’ search paths, I guess this would achieve that also)

[nhoizey]

See the note about updating Nunjucks’ search paths, I guess this would achieve that also

Indeed, but like you said, something less tied to Nunjucks would be better for Eleventy.

[TigersWay]

@paulrobertlloyd

Additionally, it would be helpful if for eleventyConfig.setNunjucksEnvironmentOptions() you could add search paths for layouts and includes.

@nhoizey

Would a solution be to accept an array for dir.layouts (and maybe dir.includes), and Eleventy would search layouts in multiple folders in the order of the array, until it finds one?

I believe you both said it all 😄
To be able to add an array of (relative?) paths to an array of dir.includes, maybe also dir.layouts (be careful with extends) would solve quite a lot of difficult setups, even outside plugins.

[nhoizey]

@zachleat you added the "plugins" label, but I think it could be useful also outside plugins.

[jonsage]

I would like to concur with the request and also propose my use case for such functionality.

I maintain a theme (in Jekyll) that is used across a suite of sites that ultimately get published to the same S3 bucket. The theme allows me create a consistent experience across independently maintained sites.

Theme functionality would be extremely useful to have in Eleventy.

First step - allow directories in config (layouts, inputs, data) to accept arrays or globs would be a great step forward.

Second step - add function to the configuration API to append (or prepend) to these arrays.

eleventyConfig.addLayoutDirectory(path)
// or
eleventyConfig.addLayouts(glob)
// etc.

Finally, static assets could be added using eleventyConfig.addPassthroughCopy

Then a theme could basically be an eleventy site packaged as a plugin (similar to how Jekyll works) installed via NPM etc. Theme development would be like developing an eleventy site. Although there are some limitations, and I've glossed over relative path resolution which may need some attention.

I feel like it is probably close to being possible with some minor backward compatible changes. Interested to see further thoughts on this functionality.

[lexoyo]

+1
And it would be even better to be able to specify a template language for the added layouts
That way we will not have plugins which only work with a given template language

[julientaq]

we’re trying to use 11ty as a microservice, so we can send plugins.

would love to be able to send layout with it to have a more flexible system. Right now, you need to copy folders of layouts, but if we have the option to actually send those layout through a addLayout function that would be quite amazing.

[zachleat]

Some work has been done for this in #1503 but it is not yet available for use in plugins.

[julientaq]

This is amazing @zachleat, you just cleared a problem we had at work for years :D

[paulrobertlloyd]

Oooooh-eeeeee. This will make my plugin sooooo much easier for users to install. Looking forward to trying it out! 🙏

[theTaikun]

Has anyone managed to get this to work with the new commits yet?

In the past I've created "themes" that consist of common widgets, meta tags, responsive layouts, etc... and then would simlink those into the main project's externals folder. The project would then copy the theme files into a temp folder first, and then overwrite it with any existing local files. This is similar to how Jekyll works, but is a bit messy:

const { promises: fs } = require("node:fs");
const path = require("path")


async function copyDir(src, dest) {
    await fs.mkdir(dest, { recursive: true });
    let entries = await fs.readdir(src, { withFileTypes: true });

    for (let entry of entries) {
        let srcPath = path.join(src, entry.name);
        let destPath = path.join(dest, entry.name);

        entry.isDirectory() ?
            await copyDir(srcPath, destPath) :
            await fs.copyFile(srcPath, destPath);
    }
}

module.exports = function(eleventyConfig) {
    eleventyConfig.ignores.add("src/_layouts");
    eleventyConfig.ignores.add("src/externals");

    eleventyConfig.on('eleventy.before', async ({ dir, runMode, outputMode }) => {
        // Run me before the build starts
        await copyDir("./src/externals/THEME/src/_layouts", "./src/temp/_layouts")
        await copyDir("./src/_layouts", "./src/temp/_layouts")

        await copyDir("./src/externals/THEME/src/_includes", "./src/temp/_includes")
        await copyDir("./src/_includes", "./src/temp/_includes")

        await copyDir("./src/externals/THEME/src/assets", "./src/temp/assets")
        await copyDir("./src/assets", "./src/temp/assets")

    });

    eleventyConfig.addPassthroughCopy({ "./src/temp/assets": "assets" });

  return {
    dir: {
      input: "src",
      // ⚠️ These values are both relative to your input directory.
      includes: "temp/_includes",
      layouts: "temp/_layouts"
    }
  };
};

I think this issue, and the mentioned commit are related to this mess I'm currently using, but would like to know how best to implement the new feature.