Plugin API.

[erichulser]

Hey Tom,

I've worked on my own documentation generator for a while and used that but am in the process of upgrading it, so I wanted to see what was out there besides Sphinx (which is way to bloated). I came across this project and I really like the simplicity of it. One of the requirements I have for documentation is auto-module/class docs as a means to augment the prose style.

I know you've already stated that you have no interest in that, but I was wondering if you'd be interested in allowing me to work with you on this project. I have all the code for generating docs that output a format like the ones used in Qt (http://qt-project.org/doc/qt-4.8/qdialog.html) from the python docs. I would propose simply adding a plugin architecture to mkdocs (similar to django/pyramid) that would allow a developer to run additional addons so as to not pollute the core of the project. Something like:

site_name: my_site
extensions:
       mkdocs_autodoc
pages:
       - ['index.md', 'Introduction']
       - ['user-guide/creating.md', 'User Guide', 'Creating a new Marshmellow project']
       - ['autodoc:my_project', 'API', 'My Project']

Where there would be a render hook that would dynamically generate the content from the my_project package as markdown files that could be then used and referenced from the prose style. I would be happy to work with you on the structure and provide the work I've been doing for the auto-class documentation as a plugin.

Please let me know if this is something that you'd be interested in discussing, otherwise I may end up forking this project and implementing that as an option.

[d0ugal]

So, I'm generally +1 on the 'plugin architecture' idea. It's something I have been pondering a bit.

[erichulser]

Awesome, is that something that you guys have a plan/roadmap for? Do you need help implementing?

[d0ugal]

I'll probably have to deferr to @tomchristie for this.

However, it is something that I do think it a good idea and I would like to see. To make MkDocs work for more people and in more situations we should allow them to be able to extend it. This does however go against a few of the goals:

• We want to create standalone installers for different platforms (like Windows) so users don't need to know they are using Python necessarily. Making a plugin architecture work in this situation would be tricky.
• So far we have tried to avoid exposing any form of Python API. The closest we have is markdown_extensions in the config.

I'm sure I had a third in mind but I can't think of it now.

[tomchristie]

Certainly be interesting to see someone work on this, but I'd be wary of pulling anything in when there's still a bunch of work ahead in order to get to a solid 1.0 release.

What would be most likely to convince me on something like this would be documentation pull requests that demonstrate the API design work, and show exactly how these plugins would work from a user perspective.

Personally I think we're probably still too early in the day to start thinking about maintaince-increasing features like a plugin architecture, but that's not to say that I wouldn't be interested in seeing how it might be tackled.

[d0ugal]

I think we can work towards it by defining the build pipeline better and exposing API's to use internally (at first). Internal only use means we don't need to worry even slightly about supporting external plugins until it settles.

I have been working on support for local searching which requires a few things: The search index needs to be created during the page builds, I need to figure out if the theme supports it (by looking for the js lib, I think, not sure yet) and probably some other stuff.

To make this work I need to hook into the process in a number of places, doing this manually is a bit nasty and each section is slowly getting more complicated.

tl;dr - improving the abractions and build processes internally would make it much easier to support a plugin api and add other features we do want to support in core.

[tomchristie]

All sounds reasonable.

[SvenDowideit]

I would use a plugin API that allows us to add pre and post process steps, for brute force changes (though given the experience we have with TWiki/Foswiki, I would also suggest you don't do this.

mmmm, actually, the safest thing you could add, would be a mechanism that allows the registration of a tag ie some begin and end marker, and a class which will be given only the pre-parsed sub-section of markup that applies to it.

contextless markup processing causes much less in the way of lock in - and if you use a pre-built parse tree (ish), you're not totally tied to the original formatting (and can validate too)

but it does depend on the cleverness of the parser you use - regexp based ones are generally less robust.

[SvenDowideit]

moby/moby#8976 (comment) is one example, where we really would prefer to have one section of information (what kernel versions and features are needed) be transcluded into many documents. Ideally, I'd use a foswiki style %INCLUDE{"topicname" section="paragraph identifier"}% - so the original maintained text is inline in an existing document.