Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@jescalan
Griko Nibras authored and jescalan committed Sep 11, 2020
1 parent b044dad commit dfbda44
Show file tree
Hide file tree
Showing 2 changed files with 104 additions and 63 deletions.
167 changes: 104 additions & 63 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,68 +1,57 @@
# Next MDX Remote
<!-- markdownlint-disable-file MD033 MD041 -->

<!--
# next-mdx-remote
A set of light utilities allowing mdx to be loaded within `getStaticProps` or `getServerSideProps` and hydrated correctly on the client.
### Background & Theory
-->

If you are using mdx within a nextjs app, you are probably using the webpack loader. This means that you have your mdx files locally and are probably using [next-mdx-enhanced](https://github.com/hashicorp/next-mdx-enhanced) in order to be able to render your mdx files into layouts and import their front matter to create index pages. This workflow is fine, but introduces a few limitations that we aim to remove with `next-mdx-remote`:
[![next-mdx-remote](./header.png)](.)

- The file content must be local. You cannot store mdx files in another repo, a database, etc. For a large enough operation, there will end up being a split between those authoring content and those working on presentation of the content. Overlapping these two concerns in the same repo makes a more difficult workflow for everyone.
- You are bound to filesystem-based routing. Your pages are generated with urls according to their locations. Or maybe you remap them using `exportPathMap`, which creates confusion for authors. Regardless, moving pages around in any way breaks things -- either the page's url or your `exportPathMap` configuration.
- You will end up running into performance issues. Webpack is a javascript bundler, forcing it to load hundreds/thousands of pages of text content will blow out your memory requirements - webpack stores each page as a distinct object with a large amount of metadata. One of our implementations with a couple hundred pages hit more than 8gb of memory required to compile the site. Builds took more than 25 minutes.
- You will be limited in the ways you are able to structure relational data. Organizing content into dynamic, related categories is difficult when your entire data structure is front matter parsed into javascript objects and held in memory.
---

So, `next-mdx-remote` changes the entire pattern so that you load your mdx content not through an import, but rather through `getStaticProps` or `getServerProps` -- you know, the same way you would load any other data. The library provides the tools to serialize and hydrate the mdx content in a manner that is performant. This removes all of the limitations listed above, and does so at a significantly lower cost -- `next-mdx-enhanced` is a very heavy library with a lot of custom logic and [some annoying limitations](https://github.com/hashicorp/next-mdx-enhanced/issues/17). Early testing has shown build times reduced by 50% or more.
- [Background & Theory](#background--theory)
- [Installation](#installation)
- [Example Usage](#example-usage)
- [APIs](#apis)
- [Frontmatter & Custom Processing](#frontmatter--custom-processing)
- [Caveats](#caveats)
- [Security](#security)
- [License](#license)

### Installation
---

```
npm i next-mdx-remote
```
## Background & Theory

### Usage
If you are using MDX within a Next.js app, you are probably using the Webpack loader. This means that you have your MDX files locally and are probably using [`next-mdx-enhanced`](https://github.com/hashicorp/next-mdx-enhanced) in order to be able to render your MDX files into layouts and import their front matter to create index pages.

This library exposes two functions, `renderToString` and `hydrate`, much like `react-dom`. These two are purposefully isolated into their own files -- `renderToString` is intended to be run **server-side**, so within `getStaticProps`, which runs on the server/at build time. `hydrate` on the other hand is intended to be run on the client side, in the browser.
This workflow is fine, but introduces a few limitations that we aim to remove with `next-mdx-remote`:

First let's break down each function's signature in some pseudo-typescript-y format, then we'll look at an example:

```typescript
renderToString(
// raw mdx contents as a string
source: String,
options?: {
// the `name` is how you will invoke the component in your mdx
components: { name: React.ComponentType },
// mdx's available options at time of writing
// pulled directly from https://github.com/mdx-js/mdx/blob/master/packages/mdx/index.js
mdxOptions: {
remarkPlugins: []any,
rehypePlugins: []any,
hastPlugins: []any,
compilers: []any,
filepath: String
},
// variable names and values which can be consumed by components
scope: { [key:string]: any }
}
)
```
- **The file content must be local.** You cannot store MDX files in another repo, a database, etc. For a large enough operation, there will end up being a split between those authoring content and those working on presentation of the content. Overlapping these two concerns in the same repo makes a more difficult workflow for everyone.
- **You are bound to filesystem-based routing.** Your pages are generated with urls according to their locations. Or maybe you remap them using `exportPathMap`, which creates confusion for authors. Regardless, moving pages around in any way breaks things -- either the page's url or your `exportPathMap` configuration.
- **You will end up running into performance issues.** Webpack is a JavaScript bundler, forcing it to load hundreds/thousands of pages of text content will blow out your memory requirements. Webpack stores each page as a distinct object with a large amount of metadata. One of our implementations with a couple hundred pages hit more than 8GB of memory required to compile the site. Builds took more than 25 minutes.
- **You will be limited in the ways you are able to structure relational data.** Organizing content into dynamic, related categories is difficult when your entire data structure is front matter parsed into javascript objects and held in memory.

So, `next-mdx-remote` changes the entire pattern so that you load your MDX content not through an import, but rather through `getStaticProps` or `getServerProps` -- you know, the same way you would load any other data. The library provides the tools to serialize and hydrate the MDX content in a manner that is performant. This removes all of the limitations listed above, and does so at a significantly lower cost -- `next-mdx-enhanced` is a very heavy library with a lot of custom logic and [some annoying limitations](https://github.com/hashicorp/next-mdx-enhanced/issues/17). Early testing has shown build times reduced by 50% or more.

## Installation

```typescript
hydrate(
// the direct return value of `renderToString`
source: CompiledMdxSourceType,
// should be the exact same components that were passed to renderToString
options?: {
components: { name: React.ComponentType }
}
)
```sh
# using npm
npm i next-mdx-remote

# using yarn
yarn add next-mdx-remote
```

Ok with that out of the way, let's look at an example for the normal use case:
## Example Usage

```jsx
import renderToString from 'next-mdx-remote/render-to-string'
import hydrate from 'next-mdx-remote/hydrate'

import Test from '../components/test'

const components = { Test }
Expand All @@ -73,30 +62,76 @@ export default function TestPage({ source }) {
}

export async function getStaticProps() {
// mdx text - can be from a local file, database, anywhere
// MDX text - can be from a local file, database, anywhere
const source = 'Some **mdx** text, with a component <Test />'
const mdxSource = await renderToString(source, { components })
return { props: { source: mdxSource } }
}
```

While it may seem strange to see these two in the same file, this is one of the cool things about next.js -- `getStaticProps` and `TestPage`, while appearing in the same file, run in two different places. Ultimately your browser bundle will not include `getStaticProps` at all, or any of the functions only it uses, so `renderToString` will be removed from the browser bundle entirely.
While it may seem strange to see these two in the same file, this is one of the cool things about Next.js -- `getStaticProps` and `TestPage`, while appearing in the same file, run in two different places. Ultimately your browser bundle will not include `getStaticProps` at all, or any of the functions only it uses, so `renderToString` will be removed from the browser bundle entirely.

## APIs

Let's break down each function:
This library exposes two functions, `renderToString` and `hydrate`, much like `react-dom`. These two are purposefully isolated into their own files -- `renderToString` is intended to be run **server-side**, so within `getStaticProps`, which runs on the server/at build time. `hydrate` on the other hand is intended to be run on the client side, in the browser.

- **`renderToString(source: string, components: object, options?: object, scope?: object)`**

**`renderToString`** consumes a string of MDX along with any components it utilizes in the format `{ ComponentName: ActualComponent }`. It also can optionally be passed options which are [passed directly to MDX](https://mdxjs.com/advanced/plugins), and a scope object that can be included in the mdx scope. The function returns an object that is intended to be passed into `hydrate` directly.

```ts
renderToString({
// Raw MDX contents as a string
source: '# hello, world',
// Optional parameters
options: {
// The `name` is how you will invoke the component in your MDX
components: { name: React.ComponentType },
// MDX's available options at time of writing pulled directly from
// https://github.com/mdx-js/mdx/blob/master/packages/mdx/index.js
mdxOptions: {
remarkPlugins: [],
rehypePlugins: [],
hastPlugins: [],
compilers: [],
filepath: '/some/file/path',
},
},
scope: {},
})
```

Visit <https://github.com/mdx-js/mdx/blob/master/packages/mdx/index.js> for available `mdxOptions`.

- **`hydrate(source: object, components: object)`**

**`hydrate`** consumes the output of `renderToString` as well as the same components argument as `renderToString`. Its result can be rendered directly into your component. This function will initially render static content, and hydrate it when the browser isn't busy with higher priority tasks.

```ts
hydrate(
// The direct return value of `renderToString`
source,
// Should be the exact same components that were passed to `renderToString`
{
components: { name: React.ComponentType },
}
)
```

- `renderToString(source: string, components: object, options?: object, scope?: object)` - This function consumes a string of mdx along with any components it utilizes in the format `{ ComponentName: ActualComponent }`. It also can optionally be passed options which are [passed directly to mdx](https://mdxjs.com/advanced/plugins), and a scope object that can be included in the mdx scope. The function returns an object that is intended to be passed into `hydrate` directly.
- `hydrate(source: object, components: object)` - This function consumes the output of `renderToString` as well as the same components argument as `renderToString`. Its result can be rendered directly into your component. This function will initially render static content, and hydrate it when the browser isn't busy with higher priority tasks.
## Frontmatter & Custom Processing

### Frontmatter & Custom Processing
Markdown in general is often paired with frontmatter, and normally this means adding some extra custom processing to the way markdown is handled. Luckily, this can be done entirely independently of `next-mdx-remote`, along with any extra custom processing necessary.

Markdown in general is often paired with frontmatter, and normally this means adding some extra custom processing to the way markdown is handled. Luckily, this can be done entirely independently of `next-mdx-remote`, along with any extra custom processing necessary. Let's walk through an example of how we could process frontmatter out of our mdx source.
Let's walk through an example of how we could process frontmatter out of our MDX source:

```jsx
import renderToString from 'next-mdx-remote/render-to-string'
import hydrate from 'next-mdx-remote/hydrate'
import Test from '../components/test'
import matter from 'gray-matter'
import Test from '../components/test'
const components = { Test }
export default function TestPage({ source, frontMatter }) {
Expand All @@ -110,13 +145,15 @@ export default function TestPage({ source, frontMatter }) {
}
export async function getStaticProps() {
// mdx text - can be from a local file, database, anywhere
const source = `---
// MDX text - can be from a local file, database, anywhere
const source = `
---
title: Test
---

Some **mdx** text, with a component <Test name={title}/>
`
`
const { content, data } = matter(source)
const mdxSource = await renderToString(content, { components, scope: data })
return { props: { source: mdxSource, frontMatter: data } }
Expand All @@ -125,14 +162,18 @@ Some **mdx** text, with a component <Test name={title}/>

Nice and easy - since we get the content as a string originally and have full control, we can run any extra custom processing needed before passing it into `renderToString`, and easily append extra data to the return value from `getStaticProps` without issue.

### Caveats
## Caveats

There's only one caveat here, which is that `import` cannot be used **inside** an mdx file. If you need to use components in your mdx files, they should be provided through the second argument to the `hydrate` and `renderToString` functions.
There's only one caveat here, which is that `import` cannot be used **inside** an MDX file. If you need to use components in your MDX files, they should be provided through the second argument to the `hydrate` and `renderToString` functions.

Hopefully this makes sense, since in order to work, imports must be relative to a file path, and this library allows content to be loaded from anywhere, rather than only loading local content from a set file path.

### Security
## Security

This library evaluates a string of JavaScript on the client side, which is how it hydrates the MDX content. Evaluating a string into javascript can be a dangerous practice if not done carefully, as it can enable XSS attacks. It's important to make sure that you are only passing the `mdxSource` input generated by the `render-to-string` function to `hydrate`, as instructed in the documentation. **Do not pass user input into `hydrate`.**

If you have a CSP on your website that disallows code evaluation via `eval` or `new Function()`, you will need to loosen that restriction in order to utilize the `hydrate` function, which can be done using [`unsafe-eval`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src#common_sources). It's also worth noting that you do not _have_ to use `hydrate` on the client side, but without it, you will get a server-rendered result, meaning no ability to react to user input, etc.

This library evaluates a string of javascript on the client side, which is how it hydrates the mdx content. Evaluating a string into javascript can be a dangerous practice if not done carefully, as it can enable XSS attacks. It's important to make sure that you are only passing the `mdxSource` input generated by the `render-to-string` function to `hydrate`, as instructed in the documentation. _Do not pass user input into `hydrate`._
## License

If you have a CSP on your website that disallows code evaluation via `eval` or `new Function()`, you will need to loosen that restriction in order to utilize the `hydrate` function, which can be done using [`unsafe-eval`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src#common_sources). It's also worth noting that you do not _have_ to use `hydrate` on the client side, but without it, you will get a server-rendered result, meaning no ability to react to user input etc.
[Mozilla Public License Version 2.0](./LICENSE)
Binary file added header.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit dfbda44

Please sign in to comment.