Parse emoji shortcodes in navbar and table of contents

[jasper-zanjani]

It appears that emoji shortcodes are only parsed in page bodies but not in the navbar and table of contents. If parsing was done on these other sections then emoji could be used to make them more navigable. I for one would love to be able to have the ability to put icons to make relevant section headings and documents more distinct.

Keep in mind that although I am actually using the material theme and emoji support appears to be poor in general in the default mkdocs theme (the emoji are rendered excessively large, occupying the full width of the enclosing container), I am at least able to reproduce the error in both themes, which means it is not an error specific to material but rather a feature missing in the mkdocs package.

• The red box on the right shows that an emoji shortcode was not parsed when it occurred in the first-order (h1) heading of the document. This shortcode was parsed correctly in the page body (again, excessively large but correct nonetheless).
• The red box on the left shows a second-order (h2) heading that was not parsed in the table of contents. Again, the page body did render the emoji.

Note that the emoji that do appear in other headings are literals ("⭐") and not shortcodes (":star:").

mkdocs.yml

site_name: 🧠
theme:
    name: material
    shortcuts:
        search: 191

markdown_extensions:
    - pymdownx.tabbed
    - pymdownx.superfences
    - pymdownx.emoji:
        emoji_index: !!python/name:materialx.emoji.twemoji
        emoji_generator: !!python/name:materialx.emoji.to_svg

Versions

OS: Ubuntu 20.04 (WSL)

• Python 3.8.5
• mkdocs 1.1.2
• mkdocs-material 6.2.3
• pymdown-extensions 8.1

[waylan]

First of all, MkDocs does not provide any native support for rendering emoji shortcodes. As can be seen in your config file the pymdownx.emoji Markdown extension is responsible for rendering the emoji. I have a few things to note about that:

• Themes may or may not include CSS for emoji. If a theme does not, then it is your responsibility as the user who is enabling the extension to provide the CSS (perhaps via the extra_css config option).
• As you note, this does not present an issue for Unicode characters. However, emoji and unicode characters are two different things, so yeah, they work differently. As Unicode characters are just plain text, they work everywhere plain text works. However, emoji shortcodes explicitly need to be processed.
• The pymdownx.emoji Markdown extension is only applied to the body of pages, which is the only text that is run through the Markdown parser. It would not be appropriate to run the navigation text (or other text not part of a page body) through the Markdown parser. Therefore, to support emoji shortcodes for that text would require some other method outside of the Markdown extension.
• The TOC (box on the left in your screenshot) has all markup stripped from it by Markdown's toc extension. Therefore, we have no way of preserving any emoji (or bold, or italic, or anything else) in the TOC.
• The search plugin indexes plain text, not the parsed HTML. Therefore, we cannot expect emoji shortcodes to be rendered there either.

In short, to support emoji shortcodes outside of the body text of a page, MkDocs would need to explicitly support it. Although I suppose that could be supported through a third party plugin and/or theme. A plugin could step through the nav items and parse each for shortcodes. Or a plugin could provide a template filter to the Jinja env, which a theme could use to parse shortcodes. Another option might be for a theme to include a JavaScript library which parses shortcodes upon page load.

The problem with all of these is that they may generate slightly different results that pymdownx.emoji Markdown extension. Therefore, in the end you would probably need to choose one option to maintain consistency.

[jasper-zanjani]

I appreciate your comprehensive response, which has taught me some things I did not know about the architecture of mkdocs.

It makes sense that the TOC and navbar should not be parsed for markdown generally, since they must be generated dynamically from the markdown of the page bodies. So it is an unfortunate consequence that these shortcodes can never be parsed if they are parsed along with markdown.

Even though I'm not a developer or maintainer, if mkdocs introduced native emoji rendering it would not seem out of place. The other solutions involving themes and extensions are too kludgy. Emoji are being found in more and more places, and I'm positive that the pymdown.emoji extension must be very popular among users of vanilla mkdocs anyway, since it is probably used mostly to organize software documentation, so I would welcome the decision to implement native emoji shortcode parsing.

[polarathene]

Just so that you're aware, the emoji short codes are generally intended to output a common graphical representation, that is an image/vector graphic. This differs from raw emoji unicode, which is treated as text not an image.

When emoji is treated as text, it's rendered based on the application or OS handling and support for emoji, this is not always consistent, especially with newer emoji released in recent years and can render quite badly (as in multiple separate emoji that compose a single emoji, a common example is skin tone or gender along with some invisibles like VS15/VS16 (for glyphs that support solid/single colour or colour forms) or ZWJ).

You'll find that Github, Twitter, Facebook and others actually normalize emoji to a specific image set regardless of viewing device, the image is selectable as it's inline and has an alt attribute providing the raw emoji for copy/paste (or a short code, sometimes that doesn't exist and the emoji isn't copied).

Some short codes do not have equivalent emoji (non-standard), the is an example of that. When there is no equivalent text variant, you would have to choose either '' (empty) or :octocat: (short code).

---

For what it's worth emoji input is worse still. There have been cases of emoji being used for passwords, but different devices/inputs would represent that encoding differently (such as including the VS15/VS16 or not), some emoji changing or being removed officially over time, some users being able to use emoji to set a password for their OS, but the OS not supporting emoji input at the login/lock screen.

---

You can probably use raw emoji despite this if you want them to appear in nav sidebars, just know that not every user will see the emoji rendered the same as you do which may impact the UX/rendering of the nav or look buggy depending on context.

An alternative is to build an SVG font and use ligatures of short codes (demo), alternatively, for more image like and colourful versions, you could leverage CSS but it'd be more of a hassle to maintain.

One more method is post-processing. You could technically write a webpage scraper (or some text processing logic with regex) and match the shortcodes in the sidebars of HTML pages/files, to replace for the emoji or image representation of your choosing. That could be a generic utility to run after a build (or I guess with mkdocs serve during local dev, if the plugin system allows for that).

[waylan]

@polarathene you make good point. Given the variety of emoji images, and the already limited time I have, trying to take on emoji in MkDocs proper is not something I have the time to maintain. I'm leaving this in the domain of third-party plugins.