Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(@jest/transform)!: require process() and processAsync() methods to always return structured data #12638

Merged
merged 12 commits into from
Apr 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
diff --git a/jest/preprocessor.js b/jest/preprocessor.js
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yarn placed this file inside .yarn directory. Perhaps a good idea?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, this is where it should be (new in yarn v3 there's a built in --save)

index 5920c0a6f23c056f27366fabf32dd13c6f86465b..2658e52f9127ac58849e7f830f6342d8b683672c 100644
index f5e34763d840193e243a974e448b619f8f635095..cc6e05ab4c9c25a3d257379a1bf19c21fb9ef5ca 100644
--- a/jest/preprocessor.js
+++ b/jest/preprocessor.js
@@ -64,8 +64,6 @@ module.exports = {
@@ -38,7 +38,7 @@ module.exports = {
sourceType: 'script',
...nodeOptions,
ast: false,
- }).code;
+ });
}

const {ast} = transformer.transform({
@@ -66,8 +66,6 @@ module.exports = {
[require('@babel/plugin-transform-flow-strip-types')],
[
require('@babel/plugin-proposal-class-properties'),
Expand All @@ -11,3 +20,12 @@ index 5920c0a6f23c056f27366fabf32dd13c6f86465b..2658e52f9127ac58849e7f830f6342d8
],
[require('@babel/plugin-transform-computed-properties')],
[require('@babel/plugin-transform-destructuring')],
@@ -112,7 +110,7 @@ module.exports = {
sourceMaps: true,
},
src,
- ).code;
+ );
},

getCacheKey: (createCacheKeyFunction([
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
- `[jest-runtime]` [**BREAKING**] `Runtime.createHasteMap` now returns a promise ([#12008](https://github.com/facebook/jest/pull/12008))
- `[jest-runtime]` Calling `jest.resetModules` function will clear FS and transform cache ([#12531](https://github.com/facebook/jest/pull/12531))
- `[@jest/schemas]` New module for JSON schemas for Jest's config ([#12384](https://github.com/facebook/jest/pull/12384))
- `[jest-transform]` [**BREAKING**] Make it required for `process()` and `processAsync()` methods to always return structured data ([#12638](https://github.com/facebook/jest/pull/12638))
- `[jest-test-result]` Add duration property to JSON test output ([#12518](https://github.com/facebook/jest/pull/12518))
- `[jest-watcher]` [**BREAKING**] Make `PatternPrompt` class to take `entityName` as third constructor parameter instead of `this._entityName` ([#12591](https://github.com/facebook/jest/pull/12591))
- `[jest-worker]` [**BREAKING**] Allow only absolute `workerPath` ([#12343](https://github.com/facebook/jest/pull/12343))
Expand Down
46 changes: 34 additions & 12 deletions docs/CodeTransformation.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,36 @@ id: code-transformation
title: Code Transformation
---

Jest runs the code in your project as JavaScript, but if you use some syntax not supported by Node.js out of the box (such as JSX, types from TypeScript, Vue templates etc.) then you'll need to transform that code into plain JavaScript, similar to what you would do when building for browsers.
Jest runs the code in your project as JavaScript, but if you use some syntax not supported by Node out of the box (such as JSX, TypeScript, Vue templates) then you'll need to transform that code into plain JavaScript, similar to what you would do when building for browsers.

Jest supports this via the [`transform` configuration option](Configuration.md#transform-objectstring-pathtotransformer--pathtotransformer-object).
Jest supports this via the [`transform`](Configuration.md#transform-objectstring-pathtotransformer--pathtotransformer-object) configuration option.

A transformer is a module that provides a synchronous function for transforming source files. For example, if you wanted to be able to use a new language feature in your modules or tests that aren't yet supported by Node, you might plug in one of many compilers that compile a future version of JavaScript to a current one.
A transformer is a module that provides a method for transforming source files. For example, if you wanted to be able to use a new language feature in your modules or tests that aren't yet supported by Node, you might plug in a code preprocessor that would transpile a future version of JavaScript to a current one.

Jest will cache the result of a transformation and attempt to invalidate that result based on a number of factors, such as the source of the file being transformed and changing configuration.

## Defaults

Jest ships with one transformer out of the box - `babel-jest`. It will automatically load your project's Babel configuration and transform any file matching the following RegEx: `/\.[jt]sx?$/` meaning any `.js`, `.jsx`, `.ts` and `.tsx` file. In addition, `babel-jest` will inject the Babel plugin necessary for mock hoisting talked about in [ES Module mocking](ManualMocks.md#using-with-es-module-imports).
Jest ships with one transformer out of the box – [`babel-jest`](https://github.com/facebook/jest/tree/main/packages/babel-jest#setup). It will load your project's Babel configuration and transform any file matching the `/\.[jt]sx?$/` RegExp (in other words, any `.js`, `.jsx`, `.ts` or `.tsx` file). In addition, `babel-jest` will inject the Babel plugin necessary for mock hoisting talked about in [ES Module mocking](ManualMocks.md#using-with-es-module-imports).

If you override the `transform` configuration option `babel-jest` will no longer be active, and you'll need to add it manually if you wish to use Babel.
:::tip

Remember to include the default `babel-jest` transformer explicitly, if you wish to use it alongside with additional code preprocessors:

```json
"transform": {
"\\.[jt]sx?$": "babel-jest",
"\\.css$": "some-css-transformer",
}
```

:::

## Writing custom transformers

You can write your own transformer. The API of a transformer is as follows:

```ts
// This version of the interface you are seeing on the website has been trimmed down for brevity
// For the full definition, see `packages/jest-transform/src/types.ts` in https://github.com/facebook/jest
// (taking care in choosing the right tag/commit for your version of Jest)

interface TransformOptions<OptionType = unknown> {
supportsDynamicImport: boolean;
supportsExportNamespaceFrom: boolean;
Expand All @@ -41,6 +48,11 @@ interface TransformOptions<OptionType = unknown> {
transformerConfig: OptionType;
}

type TransformedSource = {
code: string;
map?: RawSourceMap | string | null;
};

interface SyncTransformer<OptionType = unknown> {
canInstrument?: boolean;

Expand Down Expand Up @@ -111,6 +123,12 @@ type TransformerFactory<X extends Transformer> = {
};
```

:::note

The definitions above were trimmed down for brevity. Full code can be found in [Jest repo on GitHub](https://github.com/facebook/jest/blob/main/packages/jest-transform/src/types.ts) (remember to choose the right tag/commit for your version of Jest).
mrazauskas marked this conversation as resolved.
Show resolved Hide resolved

:::

There are a couple of ways you can import code into Jest - using Common JS (`require`) or ECMAScript Modules (`import` - which exists in static and dynamic versions). Jest passes files through code transformation on demand (for instance when a `require` or `import` is evaluated). This process, also known as "transpilation", might happen _synchronously_ (in the case of `require`), or _asynchronously_ (in the case of `import` or `import()`, the latter of which also works from Common JS modules). For this reason, the interface exposes both pairs of methods for asynchronous and synchronous processes: `process{Async}` and `getCacheKey{Async}`. The latter is called to figure out if we need to call `process{Async}` at all. Since async transformation can happen synchronously without issue, it's possible for the async case to "fall back" to the sync variant, but not vice versa.

So if your code base is ESM only implementing the async variants is sufficient. Otherwise, if any code is loaded through `require` (including `createRequire` from within ESM), then you need to implement the synchronous variant. Be aware that `node_modules` is not transpiled with default config.
Expand All @@ -125,7 +143,9 @@ Note that [ECMAScript module](ECMAScriptModules.md) support is indicated by the

:::tip

Make sure `TransformedSource` contains a source map, so it is possible to report line information accurately in code coverage and test errors. Inline source maps also work but are slower.
Make sure `process{Async}` method returns source map alongside with transformed code, so it is possible to report line information accurately in code coverage and test errors. Inline source maps also work but are slower.

During the development of a transformer it can be useful to run Jest with `--no-cache` to frequently [delete cache](Troubleshooting.md#caching-issues).

:::

Expand All @@ -143,8 +163,10 @@ Importing images is a way to include them in your browser bundle, but they are n
const path = require('path');

module.exports = {
process(src, filename, config, options) {
return `module.exports = ${JSON.stringify(path.basename(filename))};`;
process(sourceText, sourcePath, options) {
return {
code: `module.exports = ${JSON.stringify(path.basename(sourcePath))};`,
};
},
};
```
Expand Down
25 changes: 12 additions & 13 deletions docs/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1564,27 +1564,26 @@ Default timeout of a test in milliseconds.

Default: `{"\\.[jt]sx?$": "babel-jest"}`

A map from regular expressions to paths to transformers. A transformer is a module that provides a synchronous function for transforming source files. For example, if you wanted to be able to use a new language feature in your modules or tests that aren't yet supported by node, you might plug in one of many compilers that compile a future version of JavaScript to a current one. Example: see the [examples/typescript](https://github.com/facebook/jest/blob/main/examples/typescript/package.json#L16) example or the [webpack tutorial](Webpack.md).
A map from regular expressions to paths to transformers. Optionally, a tuple with configuration options can be passed as second argument: `{filePattern: ['path-to-transformer', {options}]}`. For example, here is how you can configure `babel-jest` for non-default behavior: `{'\\.js$': ['babel-jest', {rootMode: 'upward'}]}`.

Examples of such compilers include:
Jest runs the code of your project as JavaScript, hence a transformer is needed if you use some syntax not supported by Node out of the box (such as JSX, TypeScript, Vue templates). By default, Jest will use [`babel-jest`](https://github.com/facebook/jest/tree/main/packages/babel-jest#setup) transformer, which will load your project's Babel configuration and transform any file matching the `/\.[jt]sx?$/` RegExp (in other words, any `.js`, `.jsx`, `.ts` or `.tsx` file). In addition, `babel-jest` will inject the Babel plugin necessary for mock hoisting talked about in [ES Module mocking](ManualMocks.md#using-with-es-module-imports).

- [Babel](https://babeljs.io/)
- [TypeScript](http://www.typescriptlang.org/)
- To build your own please visit the [Custom Transformer](CodeTransformation.md#writing-custom-transformers) section

You can pass configuration to a transformer like `{filePattern: ['path-to-transformer', {options}]}` For example, to configure babel-jest for non-default behavior, `{"\\.js$": ['babel-jest', {rootMode: "upward"}]}`
See the [Code Transformation](CodeTransformation.md) section for more details and instructions on building your own transformer.

:::tip

A transformer is only run once per file unless the file has changed. During the development of a transformer it can be useful to run Jest with `--no-cache` to frequently [delete Jest's cache](Troubleshooting.md#caching-issues).

When adding additional code transformers, this will overwrite the default config and `babel-jest` is no longer automatically loaded. If you want to use it to compile JavaScript or TypeScript, it has to be explicitly defined by adding `{"\\.[jt]sx?$": "babel-jest"}` to the transform property. See [babel-jest plugin](https://github.com/facebook/jest/tree/main/packages/babel-jest#setup).
Keep in mind that a transformer only runs once per file unless the file has changed.

:::
Remember to include the default `babel-jest` transformer explicitly, if you wish to use it alongside with additional code preprocessors:

A transformer must be an object with at least a `process` function, and it's also recommended to include a `getCacheKey` function. If your transformer is written in ESM you should have a default export with that object.
```json
"transform": {
"\\.[jt]sx?$": "babel-jest",
"\\.css$": "some-css-transformer",
}
```

If the tests are written using [native ESM](ECMAScriptModules.md) the transformer can export `processAsync` and `getCacheKeyAsync` instead or in addition to the synchronous variants.
:::

### `transformIgnorePatterns` \[array&lt;string&gt;]

Expand Down
25 changes: 13 additions & 12 deletions docs/Webpack.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,16 +83,16 @@ Then all your className lookups on the styles object will be returned as-is (e.g
}
```

> Notice that Proxy is enabled in Node 6 by default. If you are not on Node 6 yet, make sure you invoke Jest using `node --harmony_proxies node_modules/.bin/jest`.

If `moduleNameMapper` cannot fulfill your requirements, you can use Jest's [`transform`](Configuration.md#transform-objectstring-pathtotransformer--pathtotransformer-object) config option to specify how assets are transformed. For example, a transformer that returns the basename of a file (such that `require('logo.jpg');` returns `'logo'`) can be written as:

```js title="fileTransformer.js"
const path = require('path');

module.exports = {
process(src, filename, config, options) {
return `module.exports = ${JSON.stringify(path.basename(filename))};`;
process(sourceText, sourcePath, options) {
return {
code: `module.exports = ${JSON.stringify(path.basename(sourcePath))};`,
};
},
};
```
Expand All @@ -112,16 +112,19 @@ module.exports = {

We've told Jest to ignore files matching a stylesheet or image extension, and instead, require our mock files. You can adjust the regular expression to match the file types your webpack config handles.

_Note: if you are using babel-jest with additional code preprocessors, you have to explicitly define babel-jest as a transformer for your JavaScript code to map `.js` files to the babel-jest module._
:::tip

Remember to include the default `babel-jest` transformer explicitly, if you wish to use it alongside with additional code preprocessors:

```json
"transform": {
"\\.js$": "babel-jest",
"\\.css$": "custom-transformer",
...
"\\.[jt]sx?$": "babel-jest",
"\\.css$": "some-css-transformer",
}
```

:::

### Configuring Jest to find our files

Now that Jest knows how to process our files, we need to tell it how to _find_ them. For webpack's `modulesDirectories`, and `extensions` options there are direct analogs in Jest's `moduleDirectories` and `moduleFileExtensions` options.
Expand Down Expand Up @@ -186,8 +189,7 @@ That's it! webpack is a complex and flexible tool, so you may have to make some

webpack 2 offers native support for ES modules. However, Jest runs in Node, and thus requires ES modules to be transpiled to CommonJS modules. As such, if you are using webpack 2, you most likely will want to configure Babel to transpile ES modules to CommonJS modules only in the `test` environment.

```json
// .babelrc
```json title=".babelrc"
{
"presets": [["env", {"modules": false}]],

Expand All @@ -203,8 +205,7 @@ webpack 2 offers native support for ES modules. However, Jest runs in Node, and

If you use dynamic imports (`import('some-file.js').then(module => ...)`), you need to enable the `dynamic-import-node` plugin.

```json
// .babelrc
```json title=".babelrc"
{
"presets": [["env", {"modules": false}]],

Expand Down
10 changes: 6 additions & 4 deletions e2e/__tests__/dependencyClash.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,12 @@ test('does not require project modules from inside node_modules', () => {
if (!threw) {
throw new Error('It used the wrong invariant module!');
}
return script.replace(
'INVALID CODE FRAGMENT THAT WILL BE REMOVED BY THE TRANSFORMER',
''
);
return {
code: script.replace(
'INVALID CODE FRAGMENT THAT WILL BE REMOVED BY THE TRANSFORMER',
'',
),
};
},
};
`,
Expand Down
4 changes: 2 additions & 2 deletions e2e/__tests__/multiProjectRunner.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -450,7 +450,7 @@ test('Does transform files with the corresponding project transformer', () => {
};`,
'project1/transformer.js': `
module.exports = {
process: () => 'module.exports = "PROJECT1";',
process: () => ({code: 'module.exports = "PROJECT1";'}),
getCacheKey: () => 'PROJECT1_CACHE_KEY',
}
`,
Expand All @@ -465,7 +465,7 @@ test('Does transform files with the corresponding project transformer', () => {
};`,
'project2/transformer.js': `
module.exports = {
process: () => 'module.exports = "PROJECT2";',
process: () => ({code: 'module.exports = "PROJECT2";'}),
getCacheKey: () => 'PROJECT2_CACHE_KEY',
}
`,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@ export default {

return {code: outputText, map: sourceMapText};
}
return sourceText;
return {code: sourceText};
},
};
2 changes: 1 addition & 1 deletion e2e/coverage-provider-v8/no-sourcemap/cssTransform.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@

module.exports = {
getCacheKey: () => 'cssTransform',
process: () => 'module.exports = {};',
process: () => ({code: 'module.exports = {};'}),
};
2 changes: 1 addition & 1 deletion e2e/coverage-remapping/typescriptPreprocessor.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ module.exports = {
map: JSON.parse(result.sourceMapText),
};
}
return src;
return {code: src};
},
};
4 changes: 2 additions & 2 deletions e2e/global-setup-custom-transform/transformer.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ const fileToTransform = require.resolve('./index.js');
module.exports = {
process(src, filename) {
if (filename === fileToTransform) {
return src.replace('hello', 'hello, world');
return {code: src.replace('hello', 'hello, world')};
}

return src;
return {code: src};
},
};
4 changes: 2 additions & 2 deletions e2e/snapshot-serializers/transformer.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
module.exports = {
process(src, filename) {
if (/bar.js$/.test(filename)) {
return `${src};\nmodule.exports = createPlugin('bar');`;
return {code: `${src};\nmodule.exports = createPlugin('bar');`};
}
return src;
return {code: src};
},
};
23 changes: 14 additions & 9 deletions e2e/stack-trace-source-maps-with-coverage/preprocessor.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,19 @@
const tsc = require('typescript');

module.exports = {
process(src, path) {
return tsc.transpileModule(src, {
compilerOptions: {
inlineSourceMap: true,
module: tsc.ModuleKind.CommonJS,
target: 'es5',
},
fileName: path,
}).outputText;
process(sourceText, fileName) {
if (fileName.endsWith('.ts') || fileName.endsWith('.tsx')) {
const {outputText, sourceMapText} = tsc.transpileModule(sourceText, {
compilerOptions: {
inlineSourceMap: true,
module: tsc.ModuleKind.CommonJS,
target: 'es5',
},
fileName,
});

return {code: outputText, map: sourceMapText};
}
return {code: sourceText};
},
};
23 changes: 14 additions & 9 deletions e2e/stack-trace-source-maps/preprocessor.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,19 @@
const tsc = require('typescript');

module.exports = {
process(src, path) {
return tsc.transpileModule(src, {
compilerOptions: {
inlineSourceMap: true,
module: tsc.ModuleKind.CommonJS,
target: 'es5',
},
fileName: path,
}).outputText;
process(sourceText, fileName) {
if (fileName.endsWith('.ts') || fileName.endsWith('.tsx')) {
const {outputText, sourceMapText} = tsc.transpileModule(sourceText, {
compilerOptions: {
inlineSourceMap: true,
module: tsc.ModuleKind.CommonJS,
target: 'es5',
},
fileName,
});

return {code: outputText, map: sourceMapText};
}
return {code: sourceText};
},
};
2 changes: 1 addition & 1 deletion e2e/transform-linked-modules/preprocessor.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@

module.exports = {
process() {
return 'module.exports = "transformed"';
return {code: 'module.exports = "transformed"'};
},
};
Loading