Official Asset Pipeline

[zachleat]

Surfaced in #270.

Maybe included as a separate starter project, plugin, and/or with the eleventy-base-blog project

[zachleat]

This repository is now using lodash style issue management for enhancements. This means enhancement issues will now be closed instead of leaving them open.

The enhancement backlog can be found here: https://github.com/11ty/eleventy/issues?utf8=%E2%9C%93&q=label%3Aneeds-votes+sort%3Areactions-%2B1-desc+

Don’t forget to upvote the top comment with 👍!

[kleinfreund]

In my opinion, Eleventy’s job is to expose the relevant pieces of content at appropriate stages of transformation so that available tooling can be used via their Node packages.

There is already some interesting examples of how to realize this is in the docs (e.g. compressing CSS).

It would be interesting to figure out how to use Eleventy with JavaScript bundlers like Parcel.js or Webpack. Can we somehow offer a simple way of using Eleventy and; for example, Parcel.js without resorting to starting both Eleventy and Parcel is separate terminal windows in parallel?

I think Eleventy has great potential in solving this build tool pain points that a lot of static site generators bring with them.

[edwardhorsford]

I wouldn't have considered picking up Eleventy were it not for the Eleventyone project. I'm not a developer, so though I'm sure it's possible to add yourself, it wasn't something I wanted to attempt. I've used grunt before, but again, didn't set it up to start with.

If it's helpful, here's some things I wanted:

My must haves:

• SASS compilation
• Asset concatenation / uglificiation
• Moving assets around
• Cleaning out the build directory / ability to start 'clean'
• Something I understood enough that I could hook in my own code.

For me, gulp has worked well. I can have different scripts for dev and production so different things get built - such as uncompressed / sass-maped sources on dev, and fully minified sources on prod. I don't concatenate all files together - assets that may not be needed, or aren't needed by most pages are kept and served separately. Gulp made this easy.

It's possibly related to my background, but I prefer the idea of css concatenation happening separately from templates.

A possible useful starter might be a tutorial / guidance on adding a build pipeline. I suspect you want to keep the base repo rather agnostic about this - this is where the tutorial section could really help I think.

[jevets]

I'd certainly vote for this to be outside the core, as an explicit opt-in or just a sample/starter project.

I've had luck with laravel-mix in porting one project over (it was already using mix). I just run eleventy --watch in parallel with laravel-mix's watch in the same npm script. Eleventy rebuilds the HTML, and laravel-mix writes directly to the dir.output path. One terminal window.

I've been working on setting up this kind of workflow with webpack v4 (mix is still at v3), but I've also been playing with parcel.js. I'll post a repo once I have something reasonable to share. It would probably bypass browserify completely, but I'd still want live reloading and watchers for all files.

[itlackey]

I'm looking into this as well. 11ty works well with assets in the site content (custom scripts and styles) but how to manage npm dependencies is proving difficult. For example, I added bootstrap to package.json now I need my pipeline to bundle all of bootstraps assets with mine. Currently I'm looking at calling browserify or webpack from gulp but looking for a simple answer if someone has solved it already.

Thanks!

[jevets]

Worth mentioning this here, please upvote if you like this idea of configuring eleventy to watch other globs:
#333

[paulshryock]

A possible useful starter might be a tutorial / guidance on adding a build pipeline. I suspect you want to keep the base repo rather agnostic about this - this is where the tutorial section could really help I think.

I think it would be great to include documentation about how to include some different asset pipelines easily (and maybe go so far as to provide those as plugins), but as others have said, it'd be nice to not include those by default in core. I really like it being "Bring your own ____" (markup/styles/asset pipeline/cms/etc.)

[jevets]

I managed to get something working pretty well with webpack4 and eleventy.

• Watches js, css, 11ty template files
• Auto refresh on change of any of the above files
• Builds minified js/css
• Babel support
• Uses eleventy --serve and webpack --watch

I tried getting it working with webpack-dev-server, but I couldn't manage to get both the 11ty side and the webpack side to recognize each others' changed files — auto refresh wouldn't work.

I'll probably put up a fresh repo soon and link to it here, maybe w/ a brief tutorial about this approach. I could make that repo as a starter project, too/instead.

A plugin of sorts would be nice as a simple drop-in, yet I don't know how I'd approach it as a standalone plugin. (@zachleat any thoughts on how to approach this?) It currently uses package.json scripts, but at least it'd give some people an idea how to approach using it in their own projects.

Looking to gauge interest and get all your opinions/thoughts on this as starter project vs a drop-in plugin type thing.

I think it makes more sense as a starter project or just a demo/tutorial. A drop-in plugin may end up hiding too much, getting us further away from "bring your own x".

I'm not married to webpack (I'm certainly no webpack pro), so I may try another bundler like parcel.js and see if it's any easier.

[robdodson]

One thing that I think is common in the webpack space is to give files cache busting hashes and use the HTMLWebpackPlugin to make sure the proper hashes get included in the html files:

For example:

<html>
  <head>
    <!-- produces main.52l3kj51oij6.css -->
    <link rel="stylesheet" href="<%=htmlWebpackPlugin.files.chunks.main.css %>">
  </head>
  <body>
    <div>Hello, world!</div>
    <!-- produces bundle.lfkl2ekjlk.js -->
    <script src="<%= htmlWebpackPlugin.files.chunks.main.entry %>"></script>
  </body>
</html>

@jevets did you do any of this with your eleventy integration?

[jevets]

@robdodson No, I didn't get that far but am aware of the need. May ask for some input from you if you're down. Would love to have some others help on this.

(Been away for a few weeks, will try to post some code this week.)

[chrisdmacrae]

https://github.com/chrisdmacrae/eleventy-starter-parcel

[dweidner]

One thing that I think is common in the webpack space is to give files cache busting hashes and use the HTMLWebpackPlugin to make sure the proper hashes get included in the html files:

@robdodson You could use something like webpack-manifest-plugin or webpack-asset-manifest to generate a json file that maps your entry points to the generated files.

// Example manifest.json
{
  "main.js": "assets/js/main.12345645423.js"
}

In eleventy a custom shortcode or filter could than be used to lookup the generate file name from the json file.

[Ryuno-Ki]

I'd prefer to be bundler agnostic (yes, I don't like WebPack, but prefer Rollup).

[jevets]

I've been using a stylesheet.11ty.js template file to compile CSS (via postcss in my case).
I have an _includes/actual-styles.css file, and when using eleventy --serve [| --watch], both files update automatically on change (for free).

With this route I get to avoid pulling in any additional packages (webpack, gulp, parcel, etc.).

It's been working very well for me lately, but I haven't yet used this route for sites heavy on client-side JS.

Something like this:

// stylesheet.11ty.js

module.exports = class {
  data() {
    return {
      permalink: '/main.css'
    }
  }

  render() {
    return this.doTheCompileWorkAndGetTheResultingString()
  }

  doTheCompileWorkAndGetTheResultingString() {
    return postcss([...])
  }
}

/* _includes/styles.css */

body {
  font-family: sans-serif;

  /* lives in `_includes` so eleventy automatically watches it for me */
}

[araphiel]

For anyone stumbling across here, this is my general asset setup for static site generators.

.
├── package.json
├── .eleventy.js
├── site
|     └ assets
|     └ (general page files)
└── src
      └ *.scss
      └ *.js

• Webpack - (Basic Config)

  • Pre-processed files are saved + edited in the src dir.
  • Webpack outputs asset files to site/assets dir.

• Eleventy

  • Use addPassthroughCopy to include our post-processed assets site/assets in the build.
  • 11ty ignores anything in .gitignore (aka your webpack assets) so we use setUseGitIgnore(false) to rely on .eleventyignore instead.

/* .eleventy.js */

module.exports = eleventyConfig => {
  eleventyConfig.addPassthroughCopy("site/assets");
  eleventyConfig.setUseGitIgnore(false); 
};

• NPM Scripts
  • For development, use a node package like npm-run-all to run webpack + 11ty in parallel (both in watch mode)

/* package.json */

  "scripts": {
    "start": "run-s webpack-prod eleventy",
    "dev": "run-p webpack-dev eleventy-dev",
    "eleventy": "eleventy --input=site --output=public",
    "eleventy-dev": "eleventy --input=site --output=public --watch --serve",
    "webpack-prod": "webpack --mode production --env.production",
    "webpack-dev": "webpack --mode development --env.production=false --watch"
  },

[gianpaj]

@jevets do you have a working example of stylesheet.11ty.js with node sass and postcss possibly? 🙏 thanks

[brycewray]

@jevets I had the same problem with trying to get webpack and Eleventy to "see" each other's stuff. If I used Eleventy's built-in Browsersync instance, textual changes were auto-refreshed but SCSS changes were not. The reverse was true — text yes, SCSS no — if I used webpack-dev-server with Eleventy set only to watch and not to serve.

So, after thinking about the issue for a few days, today I tried a separate instance of Browsersync via browser-sync-webpack-plugin with, again, Eleventy set only to watch and not to serve — and, lo and behold, that worked. Now I get auto-refresh on any change. I have no doubt there are more elegant solutions out there, but this one, finally, did the trick for me.

I reproduced the relevant config files in my now-updated post here, in case they might help someone.

[pascalw]

What I'm doing which works pretty well is the following:

1. I run both eleventy serve and webpack --watch in development (using npm-run-all).
2. Webpack processes assets, directly writes to _site and writes a JSON manifest to src/includes/.webpack/manifest.json. In development this triggers a reload of eleventy whenever I change an asset managed by webpack.
3. From my templates I can read the webpack manifest (using a template tag) and render the correct stylesheet and JS tags.

This is great because it's fast (webpack runs incremental) and I can utilize webpacks [hash] feature which is great for browser caching.

I have a (slightly outdated) version of this idea up here. I'll update this later with some small improvements I made in my actual (closed source) project.

Only point of friction is that it's inherently a separate system. So adding another entry point is not just a matter of dropping a JS file in a directory, but also needs a change in the webpack config. Not a huge problem since the entrypoints usually don't change much, but still a slight annoyance.

[brycewray]

@pascalw That smart method raises another thought. I often see people wishing for a pagination function that would give each individual post the “next post” and ”previous post” links (e.g., #529); indeed, I wish for it, too; right now, I manually give each post an idx number (saw that technique on another Eleventy site’s repo). While I see from the repo you linked that you’re using webpack-manifest-plugin to generate the manifest, I wonder how difficult it would be to auto-generate a JSON in _data, akin to how some plugins auto-generate the XML for RSS purposes, that would tie itself to whatever collections one has on a site? In my case, that would be posts. Then it would simply (?) be a matter of pulling from that JSON file and using some JS to call them as appropriate for each page. ...I think. :)

[luwes]

can be helpful too

  eleventyConfig.setBrowserSyncConfig({
    files: [
      'public/css',
      'public/js'
    ]
  });

[paulshryock]

Updated 1/11/2021

I wanted to share, this is how I'm using Gulp to:

• Clean my build directory before every build
• Process Markdown and templates with Eleventy to build HTML
• Validate HTML
• Lint CSS and JavaScript
• Parse Sass and post-process CSS files with PostCSS
• Transpile and bundle client-side JavaScript with Webpack and Babel
• Minify everything in production
• Copy assets
• Serve everything and watch for changes

So instead of tasking Eleventy with being a build pipeline that can do everything else, I'm including Eleventy in my external build pipeline, where everything else is handled.

Gulp is invoking Eleventy via the API, instead of Eleventy being invoked via the CLI. This lets me grab the HTML built by Eleventy in a Gulp stream, where I can then post-process the HTML which was built by Eleventy.

From the CLI:

• gulp develop to build for development
• gulp serve to build for development, serve everything, and watch for changes
• gulp build to build for production

[zomars]

@jevets I like your solution, would it be possible to do something like that but for bundling JS?

[darthmall]

I like the idea of keeping asset management independent of 11ty, but I think it would be nice to have some recipes in the docs for stuff like cache busting. It's great to allow people the freedom to configure everything the way they want, but it's also nice to not have to blaze your own trail for every task — especially common tasks like cache busting.

[darthmall]

I just wrote up a technique for processing styles/scripts using Eleventy that allows you to transform assets (i.e. compile SCSS into CSS, or bundle your JS) and use hashed file names for cache-busting.

I think it's a bit abusive of things like Eleventy's global data files, but it has the advantage of not requiring additional dependencies like Webpack or Parcel. I can't necessarily recommend that you do this, but perhaps it can serve as a strawman for coming up with an official technique for managing assets in Eleventy.

[brycewray]

I just wrote up a technique for processing styles/scripts using Eleventy that allows you to transform assets (i.e. compile SCSS into CSS, or bundle your JS) and use hashed file names for cache-busting.

I think it's a bit abusive of things like Eleventy's global data files, but it has the advantage of not requiring additional dependencies like Webpack or Parcel. I can't necessarily recommend that you do this, but perhaps it can serve as a strawman for coming up with an official technique for managing assets in Eleventy.

Very nice! I recently posted this article about doing it with a PostCSS plugin, but your solution is better and definitely more inclusive, I think.

[andyford]

I know not everyone is using Netlify, but I was trying to find a solution for asset bundling in Eleventy and found that it's a built-in option with Netlify either through the admin uI or the netlify.toml file. Here's a blog post about it: https://www.netlify.com/blog/2019/08/05/control-your-asset-optimization-settings-from-netlify.toml/
...even if you have one single /assets/style.css file linked in the HTML, the bundling feature will turn it into something like https://[SOME_HASH].cloudfront.net/css/[ANOTHER_HASH]/assets/style.css

[SimonEast]

Adding another vote for this. I realise that people want to use different pipelines, but since SASS and JS bundling is so common, it would be great to have a recommended starting point for newcomers where they can easily drop in to their project and get up and running fast.

I've always found webpack pretty slow and unwieldy, so am hoping Vite.js overtakes it at some point (from the creators of Vue JS). Whether it's mature enough to use as the default, I'm not sure, but it is designed to be pre-configured for most typical tasks, which is the kind of thing that would be great to have in Eleventy.

[zomars]

I found using esbuild it's pretty simple and straightforward:

// /assets/js/main.11ty.js
module.exports = class {
  data() {
    return {
      layout: '',
      permalink: false,
      eleventyExcludeFromCollections: true,
    };
  }

  async render(data) {
    /* https://esbuild.github.io/getting-started/#build-scripts */
    require('esbuild')
      .build({
        entryPoints: ['_includes/js/main.js'],
        bundle: true,
        outfile: '_site/assets/js/main.js',
      })
      .catch(() => process.exit(1));
  }
};

And for CSS:

// /assets/css/style.11ty.js
const path = require('path')
const sass = require('node-sass-promise')
const CleanCSS = require('clean-css')

const inputFile = path.join(__dirname, '../../_includes/scss/main.scss')
const outputFile = path.join(__dirname, '../../assets/css/style.css')

module.exports = class {
  data() {
    return {
      layout: '',
      permalink: 'assets/css/style.css',
      eleventyExcludeFromCollections: true
    }
  }

  async render() {
    const { css } = await sass.render({ file: inputFile })
    const output = new CleanCSS({}).minify(css.toString()).styles

    return output
  }
}

[mandrasch]

Just my two cents: I found nice and interesting examples as well, e.g. https://hipsterbrown.com/musings/musing/esbuild-with-11ty/ or https://github.com/jamshop/eleventy-plugin-esbuild (and so many others as mentioned here). Very exciting to see so many examples!

While this is a very nice challenge for nerds to test on which packages they want to rely - in my personal opinion this could lead to "decision fatigue" and a robust starter would be good for many users of 11ty.

A good and simple test case could be bootstrap5 I guess, because scss and js compiling is needed as well PostCSS autoprefixer is recommended (https://getbootstrap.com/docs/5.0/getting-started/download/#source-files).

npm install bootstrap @popperjs/core --save-dev

/* app.js */
import 'bootstrap';

/* app.scss,
   see: https://getbootstrap.com/docs/5.0/customize/sass/ */
@import "~bootstrap/scss/bootstrap";

I recently found https://github.com/bergwerk/11ty-mix (Laravel Mix => Webpack) for this and got it working by modifying the package.json as suggested for my own projects. As far as I understand https://laravel-mix.com/ was built for the same challenge - keeping asset building simple for devs.

(Here is a simple guide for bootstrap5 and laravel mix in general https://5balloons.info/setting-up-bootstrap-5-workflow-using-laravel-mix-webpack/, laravel mix also has autoprefixer automatically activated https://laravel-mix.com/docs/6.0/autoprefixer)

But the most readable and accessible way for (new) 11ty users would probably be using the 'afterBuild' hook in eleventy config? :-)

[SimonEast]

a robust starter would be good for many users of 11ty ... A good and simple test case could be bootstrap5 I guess ...

Yes, very much agreed. A robust quickstart guide to enabling things like Bootstrap, SCSS and JS would be much appreciated, otherwise it takes a fair bit of research to read through the options of varying quality to work out which is going to work best.

Even Jekyll has better SCSS support out-of-the-box than Eleventy at the moment.

[simplerethink]

@mandrasch @SimonEast - have you tried the Vite plugin and associated youtube video?

[mandrasch]

@mandrasch @SimonEast - have you tried the Vite plugin and associated youtube video?

No, haven't seen or tried. Thanks very much, exciting times ahead! 👍 (Although we'd have to wait for stable release of Eleventy 2.0+ to use this in production if I understand it correctly. So for Eleventy 1.0 a simple starter would be still a nice improvement)

[simplerethink]

@mandrasch I've been building a site on eleventy 2.0 for a few weeks now (not production yet) and haven't come across any issues preventing me from continuing.

[mandrasch]

@mandrasch I've been building a site on eleventy 2.0 for a few weeks now (not production yet) and haven't come across any issues preventing me from continuing.

Thanks for sharing your experience on this!

Optional information for the interested readers regarding Eleventy v1:
Got laravel-mix [webpack / nodejs] working for Bootstrap 5 based on the https://github.com/bergwerk/11ty-mix example. Integrated it into https://github.com/mandrasch/11ty-plain-bs5. 🥳

[kitschpatrol]

Just wanted to share that I recently wrote a plugin integrating Parcel in Eleventy's build pipeline. It also lets you (optionally) use the Parcel dev server as middleware.

It's templated after Zach's new eleventy-plugin-vite, and similarly requires Eleventy 2.x.

Vite's looking really promising, but I was curious about Parcel as well!

https://github.com/kitschpatrol/eleventy-plugin-parcel

[djwebdroid]

I always install webc plugin, and in my base.webc, in the head. I do this:

<link rel="stylesheet" href="/css/@layer.css">
<link rel="stylesheet" href="/css/reset.css">
<link rel="stylesheet" href="/css/vars.css">
<link rel="stylesheet" href="/css/layout.css">
<link rel="stylesheet" href="/css/site.css">
<link rel="stylesheet" :href="getBundleFileUrl('css')" webc:keep>

Result:

<link rel="stylesheet" href="/bundle/e0N9x6dUth.css">

By adding the at layer rule on top, you get the natural cascade you want and later, in your components, you add the @layer rule you need in order to have the css in it's proper place in the cascade.

I'm still experimenting with this technique to see if it holds true.

Voila, your css is bundled in one neat file!

Ok. Maybe only a small part of the asset pipeline ... :)