[RFC] Plugins: Transforms

[thescientist13]

Type of Change

• New Feature Request
• Documentation / Website
• Improvement / Suggestion
• Bug
• Other (please clarify below)

Summary

Taken from a comment in #17 (comment) in regards to a general discussion around the API and capturing the specific action item re: the lifecycle plugin type here.

Details

This would essentially be enabling potentially different source file types in addition to the default markdown support, like JavaScript or maybe YAML?

At minimum, should probably make sure to support at least .html files out of the box.

[thescientist13]

Looks like we have a new paradigm and opportunity to solve this as part of the work being done #417 with Koa. I think just trying to find the common denominator in terms of things in common when need to resolve / transform files. Things like

• filename - e.g. a .ts file would need to become a .js file for the browser
• source / contents - typescript would need to become javascript
• others?

(I wouldn't worry about ctx.redirect for example, leave that to the outer server functions)

Rather than pass the whole app / ctx object around, specify a (testable) API between the Greenwood CLI and consumers that can hopefully account for 90% of use cases. I think some shared code will likely come up between them (like referencing context or DOM / string manipulation) so feel free to come up with those. Plugin developers could be encouraged to use those as well for their own plugin offering!

I would say out of the box Greenwood should support all web languages and some friends * out of the box, everything else would be community plugins / packages

• HTML
• CSS
• JavaScript
• PNG, JPG, etc image formats
• SVG
• Markdown *
• node_modules *

Ideally plugins should be decoupled as much as possible. So given the above, I would expect at least 6 different transforms.

[thescientist13]

Roadmap as part of #418

1. Default Tranform Plugins - Rfc/issue 185 Transforms API (pt. 1) #422
2. External existing plugins (Google Analytics, Polyfills) and documentation
3. Play around with TypeScript
4. Make additional external plugins for non standard transforms - tracked in Roadmap To 1.0 And Beyond #418
   • split out images from fonts transform
   • import CSS, JSON as external plugins
   • import.meta.url for images and general assets?
5. refactor develop vs prod server and plugin usage (if applicable) - - tracked in Roadmap To 1.0 And Beyond #418

[thescientist13]

Had a couple more thoughts on technical direction here that we could achieve as part of trying to integrate our plugins into this system

TransformInterace Constructor

I think we should change the signature of the TransformInterface constructor to accept a request and compilation object, and then a third object with all the transform options, e.g.

// tranform.interface.js
module.exports = class TransformInterface {

  constructor(request, compilation, { extensions = [], contentType = '' }) {
    const { context, config } = compilation;
    this.extensions = extensions; // ['.foo', '.bar'] 
    this.workspace = context.userWorkspace; // greenwood
    this.scratchDir = context.scratchDir;
    this.request = request;
    this.config = config;
    this.contentType = contentType;
  }

  shouldTransform() {
    // do stuff
  }

  applyTransform() {
    // do stuff with path
  }
};

And extended like this

class JSTransform extends TransformInterface {

  constructor(req, compilation) {
    super(req, compilation, { 
      extensions: ['.js'], 
      contentType: 'text/javascript'
    });
  }

  shouldTransform() {
    // stuff
  }

  ...

}

And used like this

    try {
      // default transforms 
      const defaultTransforms = [
        new HTMLTransform(request, compilation),
        new MarkdownTransform(request, compilation),
        new CSSTransform(request, compilation),
        new JSTransform(request, compilation),
        new JSONTransform(request, compilation),
        new AssetTransform(request, compilation)
      ];

Reduce

Instead of a map, I think we could get what we're looking for using a reduce! This will allow us to continually pass in iterated-on response object that can just build up as transforms operate on it, ex.

      await Promise.all(defaultTransforms.reduce(async (response, plugin) => {
        if (plugin instanceof Transform && plugin.shouldTransform()) {

          const transformedResponse = await plugin.applyTransform();

          response = { 
            ...transformedResponse
          };
        }
      }response));

Ordering

I think we should try and work our way up on this, and least try support these two kinds of cases:

• pre-processors: e.g. TypeScript, SCSS
• regular transforms: e.g. HTML, CSS, JS (just like our defaults)

I think we can follow a similar model as how we are doing rehype / remark customizations. I think we could do something where we

1. Get an array of all of Greenwood's default transform extensions
2. Check that against all custom transform extensions
3. If it doesn't exist, consider it a pre-processor
4. Otherwise, put it in after our default transforms

So in a loose code example, something like

const defaultTransforms = [
  new HTMLTransform(request, compilationCopy),
  new MarkdownTransform(request, compilationCopy),
  new CSSTransform(request, compilationCopy),
  new JSTransform(request, compilationCopy),
  new JSONTransform(request, compilationCopy),
  new AssetTransform(request, compilationCopy)
];

const defaultExtensions = [].concat(defaultTransforms.map(transform => return transform.getExtension());
const preProcessTransforms = compilation.config.plugins.filter(plugin => {
  return plugin.type === 'transform'
}).filter(plugin => {
  !defaultExtensions.includes(return.plugin.getExtension());
});
const postProcessTransforms = compilation.config.plugins.filter(plugin => {
  return plugin.type === 'transform'
}).filter(plugin => {
  return defaultExtensions.includes(return.plugin.getExtension());
});

const orderedTransforms = [
  ...preProcessTransforms,
  ...defaultTransforms,
  ...postProcessTransforms
]

await Promise.all(orderedTransforms.reduce(async (response, plugin) => {
  // reduce response against all transforms
}), response);