build tree

[peterbe]

This is quite experimental. But it works. Now you get a file called client/build/tree.json (and client/build/titles.json) which you can potentially do interesting stuff with.

For example, you can incorporate this tree when doing the React rendering. (Note! I haven't even started on a prototype to do this). At the moment, we just dump the string SIDE BAR but if you read from the tree you could do something like this:

function RenderSideBar({ uri, tree }) {
    // E.g. `uri' is '/docs/Web/HTML/elements/video'
   const siblings = tree['docs']['Web']['HTML']['elements'];
   return <div className="side-bar">
      <h3>Related Content</h3>
      <ul>
         {siblings.map(doc => {
            return <li key={doc.uri} className={doc.uri === uri ? "current" : null}>
                <Link to={doc.uri}>{doc.title}</Link>
            </li>
         })};
      </ul>
   </div>
}

Another thing I'd love to build, just because I love search, is to do something like this:

let loadingAllTitles = false;
let fullTextIndex = null;
<input type="search" name="q" 
   onFocus={event => {
    if (!loadingAllTitles) {
       loadingAllTitles = true;
       fetch('/titles.json').then(r => r.json()).then(titles => {
         fullTextIndex = new FlexSearch();
         titles.forEach(doc => {
            fullTextIndex.add({uri: doc.uri, title: doc.title});
         })
       });
    }
  onChange={event => {
    if (fullTextIndex) {
      const foundTitles = fullTextIndex.search(event.target.value);
     ...do something with foundTitles...
    }
  }}
}}/>

What do you think? Is it a good start or premature?

[peterbe]

@wbamberg I made some changes from the original commit. In particular, the tree.json looks like this now:

{
  "date": "2019-06-27T12:38:56.205Z",
  "tree": {
    "docs": {
      "Web": {
        "HTML": {
          "Element": {
            "abbr": {
              "title": "<abbr>: The Abbreviation element",
              "uri": "/docs/Web/HTML/Element/abbr",
              "hash": "54336b1cd82e6c125c3fdb0d9bca3435"
            },
            "address": {
              "title": "<address>: The Contact Address element",
              "uri": "/docs/Web/HTML/Element/address",
              "hash": "c9ee9766ded5e9731999605621e7d0d8"
            },
  ...

(What's new is that everything that used to be in there is now moved under a new key called tree and in the root there's a new key called date)

And the titles.json that gets built from that looks like this instead:

{
  "date": "2019-06-27T12:38:56.211Z",
  "titles": {
    "/docs/Web/HTML/Element/abbr": "<abbr>: The Abbreviation element",
  ...

The "date" is meant to kill two birds with one stone. It's informational for human eyes (i.e. debugging) and it can be used as a hash. Suppose you have a piece of code that uses state, you can now use the date string to see if it is different. For example;

let tree = localStorage.getItem('tree');
fetch('/tree.json').then(r => r.json()).then(xhrTree => {
   if (!tree || xhrTree.date !== tree.date) {
     // server version different than local version!
    tree = xhrTree;
    localStorage.setItem('tree', JSON.stringify(xhrTree));
   }
});

[peterbe]

Note 1: The above example code is prehaps silly. Once we have thousands or tens of thousands of pages moved to Stumptown it might not make sense to store that in localStorage in the browser.

Note 2: It's implied that the file tree.json is en-US. The day the localization settles we might rename this to en-US/tree.json or tree/en-US.json or something. Doesn't matter yet.

[wbamberg]

What do you think? Is it a good start or premature?

I think this is interesting but I'm not quite sure what we would want to do with it yet. For sidebars (and their less space-constrained relatives, landing pages) I think we need to start by figuring out what kinds of sidebars we want to support, then building whatever infrastructure is needed for that.

Sidebars are super-complicated in MDN at the moment. There are ~30 different sidebar implementations, and the tools for defining sidebar content are very powerful. We have sidebars that do complicated things in JS, use the rather powerful Kuma API, use document tags and URLs and titles, use JSON data sources baked into KS, use JSON data sources loaded as npm modules.

I'd like them to be a lot less complicated than this in future, but still there are going to be some basic features which they want. We made a start on this in Whistler which I have written up here: https://github.com/mdn/stumptown-content/wiki/Notes-from-Whistler-All-Hands,-June-2019#html-sidebars. I'd like us to try to implement something like this for HTML to see how it feels (and to think some more about how we would implement something for more complex sidebars like those used in the JS docs).

[peterbe]

Yeah, with the new sidebars in stumptown-content, there is less incentive to bother with this branch.

Another idea was to use this tree to be able to transform slugs into titles. But, if we ever do something like that (i.e. some sort of macro expansion), considering how powerful the related_content is, we should do that in stumptown-content too.

I still things there's great value in generating the titles.json so that one can have that autocomplete widget for quickly jumping to a document from search. However, I can generate that without first looping through all possible content.

What I might do, when I revisit the work on the autocomplete is that I add a thing to the cli so that it generates the titles.json as a side-effect of looping.