Custom components for mdast nodes

[jstcki]

remarkReactComponents and createElement are nice but I would like to be able to customize rendering a few levels lower. By the time data gets to those extension points, all semantic information beyond the HTML element is lost.

For example, I would like to use my own React components for footnote references and footnote definitions. After some digging (phew 😅 ), I think the point to do that would be at the MDAST level, i.e. through my own plugin which I use() before this plugin.

Now, I didn't understand how to modify the footnoteDefinition node in a way (e.g. by adding a component prop) that gets preserved by mdast-util-to-hast because it turns unknown nodes into divs and somehow when I set properties on a node they don't show up in the createElement props.

Maybe this all sounds a bit confusing but currently I am confused 😁 (mainly by the multiple layers of abstraction). I'll try to summarize:

• What I want is to associate a footnoteDefinition node with a React component which receives the information about footnote definitions as data.
• What I don't want is to construct some intermediary dom-like representation with paragraphs and list items and reverse-engineer that in createElement.
• What I'd prefer to avoid is to have to re-build everything above the MDAST level myself.

Maybe @wooorm can shed some light on this?

[jstcki]

By the way, somehow properties seem get lost along the way. For example the className: 'footnote' from mdast-util-to-hast/lib/footer doesn't show up in my rendered elements. Same goes for the language- class name of fenced code blocks.

[wooorm]

Thanks for writing this out in such a detailed issue, it helps a lot :)

So, I introduced the recent changes to ensure less stuff is in this project, but that does mean there’s more abstraction, which of course makes things harder.

On the other hand, if you create a remark plug-in which sets a data.hName property on strong nodes to b, that creates a <b> elements in both remark-react and in remark-vdom. I believe that, as you’re compiling to React, setting Component would instantiate the proper React class (but then I don’t know too much about React).

Did you see the info at the bottom of mdast-util-to-hast? If not, I believe that should do most of what you’re after.

Maybe this isn’t enough, and we should implement some other mapping as well, such as somehow passing the original mdast/hast node to a function before passing it to React.createElement?

P.S. Whoops, thanks for letting me know, I’ll dig into what’s going on with footer and language-

[wooorm]

@herrstucki Ah, I forgot that of course sanitation is now on by default, defaulting to GitHub’s mechanism. That’s why all classes are stripped (take this for example: <code class="language-js">foo()</code>, foo(), the class is removed).
You can pass a new schema in though, which allows the className prop!

Maybe we should add a note that the sanitation is quite strict?

[jstcki]

So, if I do:

const footnoteRenderer = (processor, options) => {
  const transformer = (node) => {
    const footnoteDefs = node.children.filter(n => n.type === 'footnoteDefinition');

    footnoteDefs.forEach(n => {
      n.data = {hName: 'Footnote'};
    });
  };

  return transformer;
};

const markdownRenderer = remark().use(footnoteRenderer).use(reactRenderer, {
  createElement(name, props, children) {
    // Shouldn't `name` be 'Footnote' at some point?
    console.log(name, props)
    return React.createElement(name, props, children);
  }
})

… it doesn't work. Any pointers?

[jstcki]

Regarding sanitation: shouldn't this schema allow className again?

{sanitize: {attributes: {'*': ['className']}}}

(that doesn't work too 😅 )

[wooorm]

It doesn’t? I get the correct results:

var react = require('react');
var remark = require('remark');
var react = require('remark-react');
var doc = '```js\nfoo()\n```';

var vdom = remark().use(react, {
    createElement: react.createElement,
    sanitize: {attributes: {'*': ['className']}}
}).process(doc).contents;

console.log(require('util').inspect(vdom, {depth: null}));

Yields:

{ '$$typeof': Symbol(react.element),
  type: 'div',
  key: 'h-1',
  ref: null,
  props: 
   { children: 
      [ { '$$typeof': Symbol(react.element),
          type: 'pre',
          key: 'h-2',
          ref: null,
          props: 
           { children: 
              [ { '$$typeof': Symbol(react.element),
                  type: 'code',
                  key: 'h-3',
                  ref: null,
                  props: { className: 'language-js', children: [ 'foo()\n' ] },
                  _owner: null,
                  _store: {} } ] },
          _owner: null,
          _store: {} } ] },
  _owner: null,
  _store: {} }

[jstcki]

Ah, I passed the sanitize option to process() instead of use() 😅

[wooorm]

That being said, this disallows all other attributes. I suggest using a deep cloning mechanism instead, and using github.json as a base like so:

// ...
var clone = require('clone');
var schema = clone(require('hast-util-sanitize/lib/github.json'))
// ...

schema.attributes['*'].push('className');

var vdom = remark().use(react, {
    // ...
    sanitize: schema
}).process(doc).contents;

// ...

[jstcki]

A quick update on this. I ended up just using unified and remark-parse, and wrote my own compiler which generates React elements directly from the MDAST. This way, I can retain the full semantics of my Markdown documents, including footnoteDefinitions, inlineCode etc. 👌

This has two additional advantages:

• If I want to include the parser in my client code, this saves a few KB from the compiled bundle because I can skip the libraries for intermediary representations.
• And if I don't need the parser in the client, I can wrap it into a webpack loader (since the MDAST can be serialized to JSON), and just include the compiler.

So, thanks for creating such a modular system for text processing @wooorm! It's not easy to find the right integration point at first, but once I found it, it was very straight-forward 😄

[wooorm]

@herrstucki Well, first of all I’m very happy that the way I set it all up allows you to quite easily set this up. 👍

But on the other hand I’m quite unhappy that the changes I made in 4b2e674 caused you to have to work around it. I’d love to see your code; maybe the current set-up already allows that, and the docs need to be changed, or if not, those features should be included!

I also wanted to show that the below example shows how remark can be split across server/client:

node.js:

var unified = require('unified');
var parse = require('remark-parse');
var doc = require('fs').readFileSync('example.md', 'utf8');
var proc = unified().use(parse);
var tree = proc.parse(doc);
// sendToClient(JSON.stringify(tree));

...and browser.js:

var unified = require('unified');
var react = require('remark-react');
var tree = // JSON.parse(getFromClient())
var proc = unified().use(react);
var vdom = proc.stringify(tree);
// doSomethingWith(vdom);

[jstcki]

Things are still kind of in flux but I'll try to share what I came up with when it's settled a bit.

I'm (almost) doing what you suggested in your example. I just don't use remark-react but instead compile directly to React elements from the tree. I use a map of visitors similar to this:

{
  heading: (node, index /*, parent */) => React.createElement(`h${node.depth}`, {key: index}, visitChildren(node)),
  //...
}

Kinda off-topic for this repo, but when I split parser and compiler like this, I struggle with plugins which expect Parser and Compiler both to be present (e.g. https://github.com/wooorm/mdast-zone).

[wooorm]

OK, I’d love to see what you come up with an allow similar behaviour in this project as well, or update the docs accordingly!

Regarding failing modules, I’d love to know about those. Could you create issues on their respective projects? Or send me a message on Gitter!

[bkniffler]

Hey, I'm struggling with a similar issue. I'd like to have remark render a react component. More specifically, I'd like for this input:

## My gallery
@gallery(["image1.jpg", "image2.jpg"])

to render

<Gallery value={["image1.jpg", "image2.jpg"]} />

I think I will need a custom tokenization plugin and make use of remarkReactComponents. But I couldn't manage to get this to work yet. Would you happen to have some guidance or an example on this @wooorm? Thanks!

[wooorm]

This is a quite different issue, as you’re not really dealing with markdown (which remark is all about), but with stuff embedded in markdown.

First, to render @gallery(["image1.jpg", "image2.jpg"]) as something, you need to define how it should be parsed by extending the parser (probably remark-parse).
Second, you want to render XML / JSX, there’s probably tools for that, but I don’t know them.

Finally, you could also expect all HTML to be in fact JSX. Is there a reason to not allow <Gallery value={["image1.jpg", "image2.jpg"]} /> in the source document?

[bkniffler]

Thanks for the quick response and sorry for not being clear. I mean I'd like for the said input to see the rendered gallery, so the evaluated component.

Lets say I use the code from the remark-parse extension example. What I fail to understand is, how do I really introduce a new entity type (e.g. component) and map this to my own react component?

function plugin() {
  var Parser = this.Parser;
  var tokenizers = Parser.prototype.inlineTokenizers;
  var methods = Parser.prototype.inlineMethods;
  tokenizers.mention = tokenizeMention;
  methods.splice(methods.indexOf('text'), 0, 'mention');
}
function tokenizeMention(eat, value, silent) {
  var match = /^@(\w+)/.exec(value);
  if (match) {
    if (silent) return true;
    return eat(match[0])({
      'type': 'component',
      'url': 'https://social-network/' + match[1],
      'children': [{
        'type': 'text',
        'value': match[0]
      }],
    });
  }
};
tokenizeMention.notInLink = true;
tokenizeMention.locator = locateMention;
function locateMention(value, fromIndex) {
  return value.indexOf('#', fromIndex);
}

 const remark = remark().use(plugin).use(reactRenderer, {
      sanitize: false,
      remarkReactComponents: {
        component: props => <span style={{ fontWeight: 'bold' }} {...props} />
      },
    });
  }

I'd expect this to, on input #xyz, to be rendered bold.

[bkniffler]

Ah, nvm, got it!

function plugin() {
  var Parser = this.Parser;
  var tokenizers = Parser.prototype.inlineTokenizers;
  var methods = Parser.prototype.inlineMethods;
  tokenizers.mention = tokenizeMention;
  methods.splice(methods.indexOf('text'), 0, 'mention');
}
const tryParseJson = (str) => {
  try {
    return JSON.parse(str);
  } catch(ex) {
    return undefined;
  }
}
function tokenizeMention(eat, value, silent) {
  var match = /^@(\w+)(\(([^)]+)\))?/.exec(value);
  if (match) {
    if (silent) return true;
    return eat(match[0])({
      type: 'component',
      name: match[1],
      value: match[3] ? tryParseJson(match[3]) : undefined,
      'children': [{
        'type': 'text',
        'value': match[0]
      }],
    });
  }
};
tokenizeMention.notInLink = true;
tokenizeMention.locator = locateMention;
function locateMention(value, fromIndex) {
  return value.indexOf('#', fromIndex);
}
const components = {
  Default: (props) => {
    return <div>No component of type {props.name} found</div>;
  },
  Gallery: (props) => {
    return <img src="http://placehold.it/350x150" alt={props.value} />;
  }
}
this.remark = remark().use(plugin).use(reactRenderer, {
      sanitize: false,
      remarkReactComponents: components,
      toHast: {
        handlers: {
          component: (h, node) => {
            var props = { name: node.name, value: node.value };
            return h(node, components[node.name] ? node.name : 'Default', props, all(h, node));
          }
        }
      }
    });

[wooorm]

Probably because tokenizeMention matches on @, not #?

Hey, I don’t mind helping you, but this issue is not the place. I think the remark channel on Gitter is.
Or create issues on specific remark packages for specific issues, or PRs for doc updates (I think the docs could be better, and I can help guide that)

EDIT: I now see your comment, glad it worked.