Register extension from plugin

[pawamoy]

Hi! Thanks for mkdocs, I love it 🙂

I am building a plugin to support API documentation generation: mkdocstrings (inspired by mkautodoc and mkdocs-autoreflinks-plugin).

My code is currently very ugly because I'm parsing markdown lines like ::: text manually, in Python, with a for loop on lines (so without any markdown processor), then converting Python docstrings to markdown in on_page_markdown (using lots of markdown extensions features), and then finally tweaking the HTML (to add divs) in on_page_content.

Since I'm using other markdown extensions features in the markdown I generate, the end user needs to add all these extensions in mkdocs.yml.

It could be much, much cleaner to implement it with a single markdown extension (with a few dependencies to other extensions like pymdownx.inlinehilite), as the end-user would only need to set this extension in mkdocs.yml.

The thing is, the extension would still need to have access to some context of mkdocs, particularly the pages, in order to support objects references in Python docstrings, like Sphinx's rst+autodoc :class: or :meth: functionality.

For this, I need a way to pass data from the plugin to the extension. Is this possible? A maybe very similar question, but formulated differently, is: is it possible to register a markdown extension from a plugin (without having to add the extension to mkdocs.yml)? Or: do we have access to markdown extensions from a plugin, and can we modify them?

I'm planning to inspect contents of the plugin at runtime when I get some free time, but maybe you'll have some hints for me 🙂

[waylan]

You could use the on_config event to add one or more global extensions to the config.extensions setting, without requiring the user to do so.

However, if you need to pass page specific data to your extensions, it gets more complicated. The problem stems from the fact that Python-Markdown does not anticipate page specific information being passed to extensions. Sure it is doable as demonstrated by our private extension which handles relative internal links. Note that we recreate a new instance of that extensions and add it to a copy of the global list of extensions separately for each page:

mkdocs/mkdocs/structure/pages.py

Lines 170 to 184 in c9032bd

	def render(self, config, files):
	"""
	Convert the Markdown source file to HTML as per the config.
	"""
	extensions = [
	_RelativePathExtension(self.file, files)
	] + config['markdown_extensions']
	md = markdown.Markdown(
	extensions=extensions,
	extension_configs=config['mdx_configs'] or {}
	)
	self.content = md.convert(self.markdown)
	self.toc = get_toc(getattr(md, 'toc', ''))

For a page specific extension in a plugin. what I might consider doing is using the on_page_content event. In my event function, I would ignore (discard) the html passed in and take the Markdown source text in page.markdown and pass it to my own custom Markdown instance which is created specifically for that page. That custom Markdown instance will need to have a _RelativePathExtension extension, plus config['markdown_extensions'] plus your custom extension. It would basically be a re-implementation of the Page.render method with your custom stuff added. Of course, it would return the HTML generated by the custom Markdown instance.

You also seem to have some questions regarding the internal working of Python-Markdown and its extension API. That would be something to take upstream to Python-Markdown/markdown. MkDocs simply uses the library as it is provided.

[pawamoy]

Hmmm, thank you for your answer, it helps a lot.

I think config.extensions might be the key. If I can append an instance of the extension containing a reference to the plugin (self), then it's done. The extension will be able to access the plugin attributes. I don't really need to know what page is currently being built from within the extension as long as I can access the plugin attributes that were already populated during the on_nav event (which I realize I forgot to mention in the issue description).

The trick using the on_page_content to discard the generated HTML and regenerate it from page.markdown is also neat, didn't think of that before.

I'll play with that tomorrow, thank you very much @waylan !

[pawamoy]

However, if you need to pass page specific data to your extensions, it gets more complicated.

I see what you mean now. I thought the extensions would be instantiated and available through the config object, just like the plugins objects, but they are not. The config object only contains the list of extensions as strings, not Python objects. So there's no way to instantiate my extension with a reference to self and add it to the config extensions 😢

Next try is the HTML discarding trick. Maybe I'll also open a new issue/feature request to ask for exposing the instantiated markdown extensions through the config object, just like plugins.

[waylan]

You can pass an extension instance to Markdown. From the on_config event, you certainly could add an "instance" to the list of extensions rather than a string. However, that is a global instance which gets reused for all pages. There is no way to get a new instance for each page.

I am certainly open to suggestions for how to address this. However, the current system is built with the YAML config file in mind, which expects only strings. So any solution needs to keep that working consistently.

[pawamoy]

Ah, indeed, I was able to append the instantiated extension to config["markdown_extensions"]! Then I think I got all I need 🙂

Thank you again @waylan!