[RFC] Parcel 2: On-disk transformers

[rtsao]

Note: I had proposed this in a comment in #3477 but decided to break this out into a separate RFC to avoid derailing the discussion specifically related to query parameters

Problem statement

It can convenient to be able to consume non-JS files as JS modules. A common way to do this is via an import statement, which implicitly or explicitly signals some kind of transformation that occurs during bundling.

Two popular use cases are:

• Importing scoped css classes from a .css file
• Importing a React component from a .svg file

For example, with webpack, this can be expressed explicitly as

import Icon from "svgr-loader!./icon.svg";
import {foo, bar} from "css-loader!./styles.css";

or implicitly as:

import Icon from "./icon.svg";
import {foo, bar} from "./styles.css";

There a several problems with this, including:

• Leaking bundler-specific details
• Other tools don't understand the syntax

Goals

• Be able to transform non-JS assets into importable JS modules
• Only a single build process and/or file watcher
• Full support for JS type checkers such as TypeScript and Flow
• Vanilla ESM source code

Proposed Solution

Simply by writing these transformed JS modules to disk, the magic involved in resolving these transformed JS modules is eliminated. In fact, once these files are generated, any tool will work.

Suppose we have the following file tree:

src/
├── index.js
├── icon.svg
└── styles.css

If using Node.js resolver, we import:

import Icon from "./icon.svg";
import {foo, bar} from "./styles.css";

// (If not using Node.js resolver, "./icon.svg.js" and `./icon.css.js` must be used).

Parcel config

{
  "transforms": {
    "**/*.svg": ["parcel-transformer-svgr"],
    "**/*.css": ["parcel-transformer-typed-css-modules"]
  }
}

Compiler process

1. When resolving dependencies, neither ./icon.svg.js nor ./styles.css.js exist. However, because matching transformers exist, they are executed against the matching .svg and .css files
2. When executed, the transformers write the following files to disk:

styles.css.js

// @flow

export const foo = "foo_5cfec4";
export const bar = "bar_d3206a";

icon.svg.js

// @flow

import * as React from "react";

export default function Icon(props) {
  return (
    <svg width="1em" height="1em" viewBox="0 0 48 1" {...props}>
      <path d="M0 0h48v1H0z" fill="currentColor" fillRule="evenodd" />
    </svg>
  );
}

3. After being written to disk, Parcel continues with the compilation process
4. If running in watch-compile mode, icon.svg and styles.css are added into the dependency graph, so any time they are modified, the corresponding transform is re-run and the JS is regenerated on disk. And in watch-compile mode, any time the asset is deleted, so is the generated JS file.

Nice properties

• Jest "just works." No need to configure Jest transformers (Parcel does all the heavy lifting).
• Type checking "just works." Types for each generated module can be different (not limited to stubs merely based on file extensions). This is particularly important in the context of GraphQL, where a single type stub is woefully inadequate.
• Other tooling "just works." All files are valid ESM and import valid ESM files. Assuming the transformed files have already been generated by Parcel, one could run Rollup and it would bundle correctly.

Open Questions

• Transform chaining
• JS → JS transformations and non-JS → non-JS transformations (this proposal only deals with non-JS→ JS transformations)
• Passing arguments to transforms (query params?)