unified is an interface for processing text using syntax trees. It’s what powers remark, retext, and rehype, and allows for processing between formats.

Intro

unified enables new exciting projects like Gatsby to pull in Markdown, MDX to embed JSX, and Prettier to format it. It’s used in about 200k projects on GitHub and has about 10m downloads each month on npm: you’re probably using it. Some notable users are Node.js, ZEIT, Netlify, GitHub, Mozilla, WordPress, Adobe, Facebook, Google, and many more.

To read about what we are up to, follow us on Medium and Twitter
For a less technical and more practical introduction to unified, visit unified.js.org and try its introductory Guides
Browse awesome unified to find out more about the ecosystem
Questions? Get help on our Spectrum community!
Check out Contribute below to find out how to help out, or become a backer or sponsor on Open Collective

Sponsors

🥇 ZEIT

🥇 Gatsby

🥉 Compositor

Holloway

You?

Read more about the unified collective on Medium »

Install

npm:

npm install unified

This package comes with types. If you’re using TypeScript, make sure to also install @types/unist and @types/vfile.

Use

var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')
var report = require('vfile-reporter')

unified()
  .use(markdown)
  .use(remark2rehype)
  .use(doc, {title: '👋🌍'})
  .use(format)
  .use(html)
  .process('# Hello world!', function(err, file) {
    console.error(report(err || file))
    console.log(String(file))
  })

Yields:

no issues found

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>👋🌍</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <h1>Hello world!</h1>
  </body>
</html>

Table of Contents

Description
API
- processor()
- processor.use(plugin[, options])
- processor.parse(file)
- processor.stringify(node[, file])
- processor.run(node[, file][, done])
- processor.runSync(node[, file])
- processor.process(file[, done])
- processor.processSync(file|value)
- processor.data([key[, value]])
- processor.freeze()
Plugin
- function attacher([options])
- function transformer(node, file[, next])
Preset
Contribute
Acknowledgments
License

Description

unified is an interface for processing text using syntax trees. Syntax trees are a representation understandable to programs. Those programs, called plugins, take these trees and inspect and modify them. To get to the syntax tree from text, there is a parser. To get from that back to text, there is a compiler. This is the process of a processor.

| ........................ process ........................... |
| .......... parse ... | ... run ... | ... stringify ..........|

          +--------+                     +----------+
Input ->- | Parser | ->- Syntax Tree ->- | Compiler | ->- Output
          +--------+          |          +----------+
                              X
                              |
                       +--------------+
                       | Transformers |
                       +--------------+

Processors

Every processor implements another processor. To create a processor, call another processor. The new processor is configured to work the same as its ancestor. But when the descendant processor is configured in the future it does not affect the ancestral processor.

When processors are exposed from a module (for example, unified itself) they should not be configured directly, as that would change their behaviour for all module users. Those processors are frozen and they should be called to create a new processor before they are used.

Syntax trees

The syntax trees used in unified are unist nodes. A node is a plain JavaScript objects with a type field. The semantics of nodes and format of syntax trees is defined by other projects.

There are several utilities for working with nodes.

hast — HTML
mdast — Markdown
nlcst — Natural language

List of Processors

The following projects process different syntax tree formats. They parse text to a syntax tree and compile that back to text. These processors can be used as is, or their parser and stringifier can be mixed and matched with unified and plugins to process between different syntaxes.

rehype (hast) — HTML
remark (mdast) — Markdown
retext (nlcst) — Natural language

List of Plugins

The below plugins work with unified, on all syntax tree formats:

unified-diff — Ignore messages for unchanged lines in Travis

See remark, rehype, and retext for their lists of plugins.

File

When processing a document, metadata is often gathered about that document. vfile is a virtual file format that stores data, metadata, and messages about files for unified and its plugins.

There are several utilities for working with these files.

Configuration

Processors are configured with plugins or with the data method.

Integrations

unified can integrate with the file system with unified-engine. CLI apps can be created with unified-args, Gulp plugins with unified-engine-gulp, and Atom Linters with unified-engine-atom.

unified-stream provides a streaming interface.

Programming interface

The API provided by unified allows multiple files to be processed and gives access to metadata (such as lint messages):

var unified = require('unified')
var markdown = require('remark-parse')
var styleGuide = require('remark-preset-lint-markdown-style-guide')
var remark2retext = require('remark-retext')
var english = require('retext-english')
var equality = require('retext-equality')
var remark2rehype = require('remark-rehype')
var html = require('rehype-stringify')
var report = require('vfile-reporter')

unified()
  .use(markdown)
  .use(styleGuide)
  .use(
    remark2retext,
    unified()
      .use(english)
      .use(equality)
  )
  .use(remark2rehype)
  .use(html)
  .process('*Emphasis* and _stress_, you guys!', function(err, file) {
    console.error(report(err || file))
    console.log(String(file))
  })

Yields:

  1:16-1:24  warning  Emphasis should use `*` as a marker                                  emphasis-marker  remark-lint
  1:30-1:34  warning  `guys` may be insensitive, use `people`, `persons`, `folks` instead  gals-men         retext-equality

⚠ 2 warnings

<p><em>Emphasis</em> and <em>stress</em>, you guys!</p>

Processing between syntaxes

Processors can be combined in two modes.

Bridge mode transforms the syntax tree from one format (origin) to another (destination). Another processor runs on the destination tree. Finally, the original processor continues transforming the origin tree.

Mutate mode also transforms the syntax tree from one format to another. But the original processor continues transforming the destination tree.

In the previous example (“Programming interface”), remark-retext is used in bridge mode: the origin syntax tree is kept after retext is done; whereas remark-rehype is used in mutate mode: it sets a new syntax tree and discards the origin tree.

API

`processor()`

Processor describing how to process text.

Returns

Function — New unfrozen processor that is configured to work the same as its ancestor. When the descendant processor is configured in the future it does not affect the ancestral processor.

Example

The following example shows how a new processor can be created (from the remark processor) and linked to stdin(4) and stdout(4).

var remark = require('remark')
var concat = require('concat-stream')

process.stdin.pipe(concat(onconcat))

function onconcat(buf) {
  var doc = remark()
    .processSync(buf)
    .toString()

  process.stdout.write(doc)
}

`processor.use(plugin[, options])`

Configure the processor to use a plugin and optionally configure that plugin with options.

Signatures

processor.use(plugin[, options])
processor.use(preset)
processor.use(list)

Parameters

plugin (Attacher)
options (*, optional) — Configuration for plugin
preset (Object) — Object with an optional plugins (set to list), and/or an optional settings object
list (Array) — List of plugins, presets, and pairs (plugin and options in an array)

Returns

processor — The processor that use was called on.

Note

use cannot be called on frozen processors. Call the processor first to create a new unfrozen processor.

Example

There are many ways to pass plugins to .use(). The below example gives an overview.

var unified = require('unified')

unified()
  // Plugin with options:
  .use(plugin, {})
  // Plugins:
  .use([plugin, pluginB])
  // Two plugins, the second with options:
  .use([plugin, [pluginB, {}]])
  // Preset with plugins and settings:
  .use({plugins: [plugin, [pluginB, {}]], settings: {position: false}})
  // Settings only:
  .use({settings: {position: false}})

function plugin() {}
function pluginB() {}

`processor.parse(file)`

Parse text to a syntax tree.

Parameters

file (VFile) — File, any value accepted by vfile()

Returns

Node — Parsed syntax tree representing file.

Note

parse freezes the processor if not already frozen.

parse performs the parse phase, not the run phase or other phases.

Example

The below example shows how parse can be used to create a syntax tree from a file.

var unified = require('unified')
var markdown = require('remark-parse')

var tree = unified()
  .use(markdown)
  .parse('# Hello world!')

console.log(tree)

Yields:

{ type: 'root',
  children:
   [ { type: 'heading',
       depth: 1,
       children: [Array],
       position: [Position] } ],
  position:
   { start: { line: 1, column: 1, offset: 0 },
     end: { line: 1, column: 15, offset: 14 } } }

`processor.Parser`

A parser handles the parsing of text to a syntax tree. Used in the parse phase and called with a string and VFile representation of the text to parse.

Parser can be a function, in which case it must return a Node: the syntax tree representation of the given file.

Parser can also be a constructor function (a function with a parse field, or other fields, in its prototype), in which case it’s constructed with new. Instances must have a parse method that is called without arguments and must return a Node.

`processor.stringify(node[, file])`

Stringify a syntax tree to text.

Parameters

node (Node) — Syntax tree to stringify
file (VFile, optional) — File, any value accepted by vfile()

Returns

string (see notes) — Textual representation of the syntax tree

Note

stringify freezes the processor if not already frozen.

stringify performs the stringify phase, not the run phase or other phases.

Be aware that compilers typically, but not always, return string. Some compilers, such as the one configured with rehype-react, return other values (in this case, a React tree). When using TypeScript, cast the type on your side.

Example

The below example shows how stringify can be used to stringify a syntax tree.

var unified = require('unified')
var html = require('rehype-stringify')
var h = require('hastscript')

var tree = h('h1', 'Hello world!')

var doc = unified()
  .use(html)
  .stringify(tree)

console.log(doc)

Yields:

<h1>Hello world!</h1>

`processor.Compiler`

A compiler handles the compiling of a syntax tree to text. Used in the stringify phase and called with a Node and VFile representation of syntax tree to compile.

Compiler can be a function, in which case it should return a string: the textual representation of the syntax tree.

Compiler can also be a constructor function (a function with a compile field, or other fields, in its prototype), in which case it’s constructed with new. Instances must have a compile method that is called without arguments and should return a string.

`processor.run(node[, file][, done])`

Run transformers on a syntax tree.

Parameters

node (Node) — Syntax tree to run on
file (VFile, optional) — File, any value accepted by vfile()
done (Function, optional) — Callback

Returns

Promise if done is not given. The returned promise is rejected with a fatal error, or resolved with the transformed syntax tree.

Note

run freezes the processor if not already frozen.

run performs the run phase, not other phases.

`function done(err[, node, file])`

Callback called when transformers are done. Called with either an error or results.

Parameters

err (Error, optional) — Fatal error
node (Node, optional) — Transformed syntax tree
file (VFile, optional) — File

Example

The below example shows how run can be used to transform a syntax tree.

var unified = require('unified')
var references = require('remark-reference-links')
var u = require('unist-builder')

var tree = u('root', [
  u('paragraph', [
    u('link', {href: 'https://example.com'}, [u('text', 'Example Domain')])
  ])
])

unified()
  .use(references)
  .run(tree, function(err, tree) {
    if (err) throw err
    console.log(tree)
  })

Yields:

{ type: 'root',
  children:
   [ { type: 'paragraph', children: [Array] },
     { type: 'definition',
       identifier: '1',
       title: undefined,
       url: undefined } ] }

`processor.runSync(node[, file])`

Run transformers on a syntax tree.

An error is thrown if asynchronous plugins are configured.

Parameters

node (Node) — Syntax tree to run on
file (VFile, optional) — File, any value accepted by vfile()

Returns

Node — Transformed syntax tree.

Note

runSync freezes the processor if not already frozen.

runSync performs the run phase, not other phases.

`processor.process(file[, done])`

Process the given file as configured on the processor.

Parameters

file (VFile) — File, any value accepted by vfile()
done (Function, optional) — Callback

Returns

Promise if done is not given. The returned promise is rejected with a fatal error, or resolved with the processed file.

The parsed, transformed, and stringified value is exposed on file.contents.

Note

process freezes the processor if not already frozen.

process performs the parse, run, and stringify phases.

Be aware that compilers typically, but not always, return string. Some compilers, such as the one configured with rehype-react, return other values (in this case, a React tree). When using TypeScript, cast the type of file.contents on your side.

Example

The below example shows how process can be used to process a file, whether transformers are asynchronous or not, with promises.

var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')

unified()
  .use(markdown)
  .use(remark2rehype)
  .use(doc, {title: '👋🌍'})
  .use(format)
  .use(html)
  .process('# Hello world!')
  .then(
    function(file) {
      console.log(String(file))
    },
    function(err) {
      console.error(String(err))
    }
  )

Yields:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>👋🌍</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <h1>Hello world!</h1>
  </body>
</html>

`function done(err, file)`

Callback called when the process is done. Called with a fatal error, if any, and a file.

Parameters

err (Error, optional) — Fatal error
file (VFile) — Processed file

Example

The below example shows how process can be used to process a file, whether transformers are asynchronous or not, with a callback.

var unified = require('unified')
var parse = require('remark-parse')
var stringify = require('remark-stringify')
var github = require('remark-github')
var report = require('vfile-reporter')

unified()
  .use(parse)
  .use(github)
  .use(stringify)
  .process('@wooorm', function(err, file) {
    console.error(report(err || file))
    console.log(String(file))
  })

Yields:

no issues found

[**@wooorm**](https://github.com/wooorm)

`processor.processSync(file|value)`

Process the given file as configured on the processor.

An error is thrown if asynchronous plugins are configured.

Parameters

file (VFile) — File, any value accepted by vfile()

Returns

(VFile) — Processed file

Note

processSync freezes the processor if not already frozen.

processSync performs the parse, run, and stringify phases.

Be aware that compilers typically, but not always, return string. Some compilers, such as the one configured with rehype-react, return other values (in this case, a React tree). When using TypeScript, cast the type of file.contents on your side.

Example

The below example shows how processSync can be used to process a file, if all transformers are synchronous.

var unified = require('unified')
var markdown = require('remark-parse')
var remark2rehype = require('remark-rehype')
var doc = require('rehype-document')
var format = require('rehype-format')
var html = require('rehype-stringify')

var processor = unified()
  .use(markdown)
  .use(remark2rehype)
  .use(doc, {title: '👋🌍'})
  .use(format)
  .use(html)

console.log(processor.processSync('# Hello world!').toString())

Yields:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>👋🌍</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <h1>Hello world!</h1>
  </body>
</html>

`processor.data([key[, value]])`

Configure the processor with information available to all plugins. Information is stored in an in-memory key-value store.

Typically, options can be given to a specific plugin, but sometimes it makes sense to have information shared with several plugins. For example, a list of HTML elements that are self-closing, which is needed during all phases of the process.

Signatures

processor = processor.data(key, value)
processor = processor.data(values)
value = processor.data(key)
info = processor.data()

Parameters

key (string, optional) — Identifier
value (*, optional) — Value to set
values (Object, optional) — Values to set

Returns

processor — If setting, the processor that data is called on
value (*) — If getting, the value at key
info (Object) — Without arguments, the key-value store

Note

Setting information cannot occur on frozen processors. Call the processor first to create a new unfrozen processor.

Example

The following example show how to get and set information:

var unified = require('unified')

var processor = unified().data('alpha', 'bravo')

processor.data('alpha') // => 'bravo'

processor.data() // {alpha: 'bravo'}

processor.data({charlie: 'delta'})

processor.data() // {charlie: 'delta'}

`processor.freeze()`

Freeze a processor. Frozen processors are meant to be extended and not to be configured directly.

Once a processor is frozen it cannot be unfrozen. New processors working the same way can be created by calling the processor.

It’s possible to freeze processors explicitly by calling .freeze(). Processors freeze implicitly when .parse(), .run(), .runSync(), .stringify(), .process(), or .processSync() are called.

Returns

processor — The processor that freeze was called on.

Example

The following example, index.js, shows how rehype prevents extensions to itself:

var unified = require('unified')
var parse = require('rehype-parse')
var stringify = require('rehype-stringify')

module.exports = unified()
  .use(parse)
  .use(stringify)
  .freeze()

The below example, a.js, shows how that processor can be used and configured.

var rehype = require('rehype')
var format = require('rehype-format')
// …

rehype()
  .use(format)
  // …

The below example, b.js, shows a similar looking example that operates on the frozen rehype interface because it does not call rehype. If this behaviour was allowed it would result in unexpected behaviour so an error is thrown. This is invalid:

var rehype = require('rehype')
var format = require('rehype-format')
// …

rehype
  .use(format)
  // …

Yields:

~/node_modules/unified/index.js:440
    throw new Error(
    ^

Error: Cannot invoke `use` on a frozen processor.
Create a new processor first, by invoking it: use `processor()` instead of `processor`.
    at assertUnfrozen (~/node_modules/unified/index.js:440:11)
    at Function.use (~/node_modules/unified/index.js:172:5)
    at Object.<anonymous> (~/b.js:6:4)

`Plugin`

Plugins configure the processors they are applied on in the following ways:

They change the processor: such as the parser, the compiler, or configuring data
They specify how to handle syntax trees and files

Plugins are a concept. They materialise as attachers.

Example

move.js:

module.exports = move

function move(options) {
  var expected = (options || {}).extname

  if (!expected) {
    throw new Error('Missing `extname` in options')
  }

  return transformer

  function transformer(tree, file) {
    if (file.extname && file.extname !== expected) {
      file.extname = expected
    }
  }
}

index.md:

# Hello, World!

index.js:

var unified = require('unified')
var parse = require('remark-parse')
var remark2rehype = require('remark-rehype')
var stringify = require('rehype-stringify')
var vfile = require('to-vfile')
var report = require('vfile-reporter')
var move = require('./move')

unified()
  .use(parse)
  .use(remark2rehype)
  .use(move, {extname: '.html'})
  .use(stringify)
  .process(vfile.readSync('index.md'), function(err, file) {
    console.error(report(err || file))
    if (file) {
      vfile.writeSync(file) // Written to `index.html`.
    }
  })

Yields:

index.md: no issues found

index.html:

<h1>Hello, World!</h1>

`function attacher([options])`

Attachers are materialised plugins. An attacher is a function that can receive options and configures the processor.

Attachers change the processor, such as the parser, the compiler, configuring data, or by specifying how the syntax tree or file are handled.

Context

The context object (this) is set to the processor the attacher is applied on.

Parameters

options (*, optional) — Configuration

Returns

transformer — Optional.

Note

Attachers are called when the processor is frozen, not when they are applied.

`function transformer(node, file[, next])`

Transformers handle syntax trees and files. A transformer is a function that is called each time a syntax tree and file are passed through the run phase. If an error occurs (either because it’s thrown, returned, rejected, or passed to next), the process stops.

The run phase is handled by trough, see its documentation for the exact semantics of these functions.

Parameters

node (Node) — Syntax tree to handle
file (VFile) — File to handle
next (Function, optional)

Returns

Error — Fatal error to stop the process
node (Node) — New syntax tree. If returned, the next transformer is given this new tree
Promise — Returned to perform an asynchronous operation. The promise must be resolved (optionally with a Node) or rejected (optionally with an Error)

`function next(err[, tree[, file]])`

If the signature of a transformer includes next (the third argument), the transformer may perform asynchronous operations, and must call next().

Parameters

err (Error, optional) — Fatal error to stop the process
node (Node, optional) — New syntax tree. If given, the next transformer is given this new tree
file (VFile, optional) — New file. If given, the next transformer is given this new file

`Preset`

Presets are sharable configuration. They can contain plugins and settings.

Example

preset.js:

exports.settings = {bullet: '*', emphasis: '*', fences: true}

exports.plugins = [
  require('remark-preset-lint-recommended'),
  require('remark-preset-lint-consistent'),
  require('remark-comment-config'),
  [require('remark-toc'), {maxDepth: 3, tight: true}],
  require('remark-license')
]

readme.md:

# Hello, World!

_Emphasis_ and **importance**.

## Table of Contents

## API

## License

index.js:

var remark = require('remark')
var vfile = require('to-vfile')
var report = require('vfile-reporter')
var preset = require('./preset')

remark()
  .use(preset)
  .process(vfile.readSync('readme.md'), function(err, file) {
    console.error(report(err || file))

    if (file) {
      vfile.writeSync(file)
    }
  })

Yields:

readme.md: no issues found

readme.md now contains:

# Hello, World!

*Emphasis* and **importance**.

## Table of Contents

*   [API](#api)
*   [License](#license)

## API

## License

[MIT](license) © [Titus Wormer](https://wooorm.com)

Contribute

See contributing.md in unifiedjs/.github for ways to get started. See support.md for ways to get help. Ideas for new plugins and tools can be posted in unifiedjs/ideas.

A curated list of awesome unified resources can be found in awesome unified.

This project has a Code of Conduct. By interacting with this repository, organisation, or community you agree to abide by its terms.

Acknowledgments

Preliminary work for unified was done in 2014 for retext and inspired by ware. Further incubation happened in remark. The project was finally externalised in 2015 and published as unified. The project was authored by @wooorm.

Although unified since moved its plugin architecture to trough, thanks to @calvinfo, @ianstormtaylor, and others for their work on ware, as it was a huge initial inspiration.

License

MIT © Titus Wormer

License

JamesMessinger/unified

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages