[Feature Request] MDX Support

[swyxio]

See #2064 for technical requirements.

Is your feature request related to a problem? Please describe.

I want to use MDX with NetlifyCMS. But I can't.

Describe the solution you'd like

NetlifyCMS should:

• detect my .mdx files
• not attempt to WYSIWYG my mdx files, but it can render in the right pane

Describe alternatives you've considered

nil. vuepress was mentioned in another issue

Additional context

some background chat done between me and @erquhart on the netlify internal slack as well as cms channel: https://netlify.slack.com/archives/C1C30FQFM/p1537919621000100

so i converted the gatsby netlifycms template to use mdx: https://gatsby-starter-netlify-cms-with-mdx.netlify.com/
source: https://github.com/sw-yx/gatsby-starter-netlify-cms-1
however netlifycms just totally fails to recognize mdx files instead of md files. i think the problem comes in config.yml where it is just hardcoded to parse markdown
in particular this line https://github.com/sw-yx/gatsby-starter-netlify-cms-1/blob/master/static/admin/config.yml#L11
what do you folks think we should do here?

@verythorough:

The collection field accepts an extension sub-field, but mdx is not a valid value for that right now:
https://www.netlifycms.org/docs/configuration-options/#collections
If it was added, we'd also need to add an mdx parser of some kind. Markdown is currently parsed/generated with Slate & Remark

@erquhart:

However we do it, MDX will require a fair amount of effort within the CMS itself.
Your work made me realize mdx has it's own extension - if that's always the case, we may be able to tackle a lot of this at the format level, before the file content ever reaches the editor.
To wit, the CMS reads in files and parses them before storing them in state, so maybe mdx files get some extra state pointing to the components they import, and those are then made available to the editor, but the markdown string that's passed to the editor is stripped of those import/export statements.

[swyxio]

one alternative i realized overnight is that we can write MDX files with an MD extension: https://gatsby-mdx.netlify.com/guides/mdx-file-extensions

so i did that to my own post: with these commits: https://github.com/sw-yx/gatsby-starter-netlify-cms-1/commit/07f6bd01d5f2133450c833deffba00a3107f9d99

and it worked! https://gatsby-starter-netlify-cms-with-mdx.netlify.com/blog/2017-01-04-this-is-an-mdx-post

Here's how it looks in netlify cms in markdown view

and the rich text view:

I can probably put in more work to fix the rendering on the right panel.

is this what we want? treat MDX as MD files? is that too hacky or do we want to have a special flow for MDX files?

[erquhart]

Hmm yeah I guess if the editor doesn't know about MDX it won't mess with any of it (already treats xml style markup as html and ignores). Really cool that MDX isn't 100% blocked for folks that really want it.

To announce support, however, we'll need preview support in place.

[tech4him1]

one alternative i realized overnight is that we can write MDX files with an MD extension

@sw-yx
Technically, you should be able to set on the collection:

extension: mdx
format: frontmatter

You can then leave the files with an mdx extension, and they will be treated like a regular MD file.

The collection field accepts an extension sub-field, but mdx is not a valid value for that right now:
https://www.netlifycms.org/docs/configuration-options/#collections

@verythorough That's not exactly correct -- the extension field will take any value (that's how people can use it for en.md, for example). The only difference if it is not one of those listed values is that the format must be set as well, since it cannot be inferred. I'm thinking we should update the linked documentation to make that more clear. Thoughts?

[verythorough]

@tech4him1 Right. The docs do actually say that in the two paragraphs that follow, but people (like me! and I wrote it!) might not read that far.

My Friday-afternoon brain isn't coming up with a clearer wording. Maybe we should file a separate issue for it. I'll drop it on my to-do list to look at later.

Another semi-related thing I think I'll make a feature request for is a "raw" format - something that does no parsing or previewing, but passes through the raw text, allows editing, and saves as typed. It's obviously not full support, but could be a nice option for allowing the authentication/browsing/saving flow of the CMS for formats that aren't supported (like rST, for example).

[swyxio]

yes. my kind of ideal (and i know i dont speak for everyone) is the dev.to workflow - where there is ONLY "raw" and then the preview pane. this way it's fast (because no wysiwyg) and extensible (mdx is a cinch). sorry i know this is classic scope creep :)

[swyxio]

inserting ok-mdx is an option https://github.com/jxnblk/ok-mdx

[talves]

@sw-yx is the only issue we have so far is to fix the preview (right pane)? or do you want full support from the editor also?

[swyxio]

yup, preview is fine.basically a dev.to workflow would be seamless to add this. but also there’s probably config stuff we need to work out.

[erquhart]

The big issue is MDX import and export statements.

ok-mdx looks awesome, great find @sw-yx! It may face similar issues as codemirror getting integrated into slate, but if nothing else it's a good starting point.

Another note for folks following this: MDX is now under the Unified umbrella along with Remark and Rehype, the libraries our markdown parsing is already driven by. We should be able to support MDX without wrapping in a purpose built editor, but hard to say for sure until someone digs in.

[talves]

I dug into it this weekend and rendering MDX during runtime for the preview is going to be a big ask.

[erquhart]

I expect it will look like this:

• pluck out import/exports before parsing the file, and prepare references to the imported/exported components
• add serialization logic to Remark for recognizing MDX imported React components
• I think that's all

From there, the markdown-turned-html and React components can all be static rendered via react-dom and live updated with any changes (which is what we already do). The question is where to register the React components that the MDX is importing. Probably really simple to solve, but I haven't spent any time considering it yet.

[karolis-sh]

I'm building an MDX based CMS at work, tried a few things and as @erquhart mentioned one of the main issues for me was the imports.
I made a small wrapper mdx-scoped-runtime around @mdx-js/runtime package that strips down the imports and layout export syntax using babel standalone. The gotcha is that you have to provide the possible importable components via scope prop for the pseudo-dynamic imports to work.

Also made a widget with said runtime. It's limited with the pre-defined scope prop, but the preview seems to work as expected.

I'd like to expand the mdx-scoped-runtime with ability to analyse the imports and provide some validation maybe, but the gatsby cms build has some issues (gatsbyjs/gatsby#10326) with certain babel plugins.

EDIT: I actually refactored the solution, turns out you can remove the imports via remark plugins quite easily, so no @babel/standalone required.

[damassi]

@buz-zard - the widget is working well 👌 It would be nice to find a way to make it work with gatsby-mdx (different package) global scope ChristopherBiscardi/gatsby-mdx#226, but for the time being the issue can be worked around by duplication.