Parallel Serialization

[thescientist13]

Type of Change

• New Feature Request
• Documentation / Website
• Improvement / Suggestion
• Bug
• Other (please clarify below)

Summary

Currently the serialization phase runs serially, one page at a time, each page as a Promise.

const runBrowser = async (compilation) => {

  try {
    return await Promise.all(compilation.graph.map(({ route, label }) => {
      return browserRunner(`http://127.0.0.1:${PORT}${route}`, label, route, compilation.context.publicDir);
    }));
  } catch (err) {
    // eslint-disable-next-line no-console
    console.log(err);
    return false;
  }
};

It would be good to see what options there are to support running these in parallel so multiple pages could be serialized at once.

Otherwise this happens locally 😬

Details

Something to consider is how to manage the "firehose" of creating and serializing pages. It would probably make sense to support a queue so for very large applications, hundreds and thousands of pages won't get spun up at once and potentially stall / crash the build server or local development workstation.

It should probably be a configurable value from greenwood.config.js.

[thescientist13]

Also, could we replace JSDOM entirely? 🤔

[thescientist13]

Seems like this would help out #141 and would allow us to remove process. setMaxListeners

[thescientist13]

Wonder what our options in regard to an actual renderToString method being worked on for lit-html.

• Render to a stream lit/lit#187
• lit-html-server lit/lit#461

Could be an alternative to using puppeteer, though full browser rendering is pretty cool as long as we can make this change.

[thescientist13]

So looking at this again through the lens of #213 , I notice we're launching a new instance of puppeteer for every page but and then never closing it.

It would probably make sense to launch puppeteer once, and run every page through it, then close it down. This could be dragging our tests down or causing memory leaks or even just wasting system resources.

Maybe a ➕ 1 for #15 and or some sort of benchmarking?