Use the recipe to drive the renderer

[wbamberg]

This PR is an attempt to implement something like #68 (comment).

It changes the build-json stuff to emit something like:

{
  "title": "<bdi>: The Bidirectional Isolate element",
  "mdn_url": "https://developer.mozilla.org/docs/Web/HTML/Element/bdi",
  "related_content": [...],
  "body": [
    {
      "type": "prose",
      "value": {
        "title": "Short description",
        "id": "short_description",
        "content": "..."
      }
    },
    {
      "type": "interactive_example",
      "value": "https://interactive-examples.mdn.mozilla.net/pages/tabbed/bdi.html"
    },
    {
      "type": "prose",
      "value": {
        "title": "Overview",
        "id": "overview",
        "content": "..."
      }
    },
    {
      "type": "attributes",
      "value": [...]
    },
    {
      "type": "additional_prose",
      "value": [
        {
          "type": "prose",
          "value": {
            "title": "Some custom section",
            "content": "..."
          }
        }
      ]
    },
    {
      "type": "examples",
      "value": [...]
    },
    {
      "type": "info_box",
      "value": "info_box-value"
    },
    {
      "type": "browser_compatibility",
      "value": {...}
    },
    {
      "type": "prose",
      "value": {
        "title": "See also",
        "id": "see_also",
        "content": "..."
      }
    }
  ],
  "contributors": "..."
}

That is, is has title, mdn-url, then related_content. Then body which is an array of the bits of the page.

Each element in the body array is an object where:

• type is the type of thing, either prose for a simple prose section or an identifier like interactive_example
• value is the value of the thing and its syntax is dependent on the value of type.

The elements in body are listed in the order we expect them to be in the page (which is also the order they're listed in the recipe.

Features of this are that build-json is pretty generic: it has to add code when we invent a new atomic piece of a page, but not just for a new recipe (so for example this should be able to handle the proposed input element type without any changes). And then the renderer is generic: again it has to know how to render each atomic piece, but to render any page it just has to walk through the array rendering each piece in order.

Another feature of course is that the JSON output is much closer to the MDN page than the current one in, and we'd have to decide, eventually, whether that's a direction we want to go in.

This PR also comes with the change proposed for info_box (#106). It would be possible to keep the original syntax, although it would make the processing more complicated. But I don't think keeping the old syntax would be the right thing to do. I'm still not sure what we should do with "info boxes" - most reference pages have them in one form or another and it would be good to invent a generic syntax for them.

[ddbeck]

This looks reasonable to me. 👍

[peterbe]

I haven't attempted to use this in stumptown-renderer but if I think about it, I think it'll work nicely and be easy to do.

And my prototype I have on the renderer for syntax highlighting would work when the packaged content is like this.

Almost all of my comments are nits but I really think we should avoid console.error and throw a proper error instead because swallowing problems are dangerous.

[peterbe]

Suggested change

	console.error(`Error: Unrecognized ingredient: ${ingredient}`);
	throw new Error(`Unrecognized ingredient: ${ingredient}`);

[peterbe]

If you like this suggestion, the next line would need to be nixed.

[peterbe]

In the else case, should it at least return null or perhaps throw an error? Either way, looks a bit "naked" without dealing with this.

[wbamberg]

I think it is permissible for there not to be any matches, if the prose ingredient is optional and this item omits it. But I agree that we should have an explicit return branch here.

[peterbe]

Shouldn't this be...

throw new Error(`Unrecognized ingredient type: ${ingredientType}`);

Also, perhaps it's overkill but I could see how this error might be very cryptic if it happens in a CI build. We could either do something like this:

console.warn(`Ingredient processing problem in ${elementPath}`);
throw new Error(`Unrecognized ingredient type: ${ingredientType}`);

...or...

throw new Error(`Unrecognized ingredient type: ${ingredientType} in ${elementPath}`);

[wbamberg]

Thanks for the reviews! @peterbe , I think all your comments should be addressed?

@Elchi3 , since this PR affects the external interface I'd love it if you could take a look but I appreciate you are busy. So let me know if you expect to get a chance to look at this, and if you would like me to wait until you can find time.

[peterbe]

Seems my few things are fully addressed. Yay!

[peterbe]

Perhaps I'm missing something. The packaged file packaged/html/guides/Applying_color.json now looks like this:

    {
      "type": "prose",
      "content": "<h2>Letting the user pick a color</h2>\n<p>There are many situations in whic...ve.</p>"
    },

What we need to achieve is a DOM so that links like this work.
I.e. that <h2>Letting the user pick a color</h2> needs to become <h2 id="Letting_the_user_pick_a_color">Letting the user pick a color</h2>.

Is that supposed to be done in the renderer now?

[peterbe]

In packaged/html/reference/elements/video.json it's a bit more obvious. The JSON contains:

    {
      "type": "prose",
      "value": {
        "title": "See also",
        "id": "see_also",
        "content": "<ul>\n<li><a href=\"https
    ...

So, a React component would know what to do. E.g.

return <div>
  <h2 id={section.id}>{section.title}</h2>
  <div dangerouslyInsertHTML={{__html: section.content}}/>
</div>

[wbamberg]

that <h2>Letting the user pick a color</h2> needs to become <h2 id="Letting_the_user_pick_a_color">Letting the user pick a color</h2>.

The id attributes in the HTML we scrape from MDN don't survive the conversion to Markdown, because Markdown doesn't support attributes. It would be possible of course to reinstate them at some point (either this side or in the renderer) but we don't currently do this.

In packaged/html/reference/elements/video.json it's a bit more obvious.

There's a basic difference between the reference doc JSON and the guides JSON. In the reference docs, prose elements are always top-level sections of a doc, with an H2 heading. In the guides, prose elements are just chunks of HTML and there's no expectation that they are aligned with any particular part of the doc outline. They are just aligned with calls to insert examples and BCD tables. So there wouldn't really be any sense in having a prose.id property in the JSON for that element.

For example, a guide doc could have:

1. an H2
2. a paragraph
3. a paragraph
4. a live sample
5. a paragraph
6. an H2
7. a paragraph
8. a live sample
9. a paragraph

....this would get processed into 5 pieces:

1. an H2
2. a paragraph
3. a paragraph
---
4. a live sample
---
5. a paragraph
6. an H2
7. a paragraph
---
8. a live sample
---
9. a paragraph

The third piece is a prose section starting with a <p> element, and it doesn't seem sensible to give that an id and not, say, the <p> at 2 or 3 let along the <h2> at 6.

But then, even for reference docs, the id property is not intended particularly for the benefit of the MDN renderer to add id attributes to the rendered HTML. For one thing, you only get them for H2 headings, so if you wanted to add id to H3s, it wouldn't help. The id is intended mostly for consumers to be able to select particular bits of the documentation (like the short descriptions) from the JSON.

I feel like if it's an important use case to have id attributes for headings, we should treat that separately from the JSON structures as far as possible, and maybe have a custom step that adds them when we generate HTML from the Markdown.

[peterbe]

would be possible of course to reinstate them at some point (either this side or in the renderer)

My gut tells me this should be done in stumptown-content. Perhaps we'll get 99% mileage from doing it with the jsdom with a simple conversion of the <h2> content transformed. I can file a new issue about that.

[Elchi3]

@Elchi3 , since this PR affects the external interface I'd love it if you could take a look but I appreciate you are busy. So let me know if you expect to get a chance to look at this, and if you would like me to wait until you can find time.

Sorry for not being involved. I'm bad at context switching and BCD has been my focus this sprint.

I've read the discussion in the issue and in this PR and from a conceptual point of view, this makes sense to me and I'm looking forward to seeing this new public interface in action. I haven't reviewed details or the code. It seems others have done that already.

(Part of me wishes to start a side project that uses stumptown-content, so that I could give you way better feedback from a consumers point of view that is not stumptown-renderer aka "mdn the website")

[peterbe]

project that uses stumptown-conten

mdn/yari#85 hint hint :)

[peterbe]

Wohoo!! I'll try to take a stab at using this in the renderer tomorrow.