Improve markdown compiler performance #2661
The CompileMarkdown.compile_string() method was always adding the configured markdown extensions (from MARKDOWN_EXTENSIONS) to self.extensions. This meant that each time it was called, self.extensions grew. As a result, the markdown conversion grew slower each time due to the increasing number of extensions. Fix by adding the configured extensions once in the set_site() method instead.
In the PR description, I included some timings I measured (basically more than 11 minutes before my changes, then under 55 seconds with just the first commit, and finally under 34 seconds with both commits). Are you looking for something more than that? If so, could you please describe what you would like to see? I'm happy to try to accomodate
Stop using the markdown() convenience function in the CompileMarkdown.compile_string() method, which creates a new Markdown object each time it is called. Instead, create a new class, ThreadLocalMarkdown, which is a subclass of threading.local. This class has a single attribute, markdown, which is a per-thread Markdown instance. It also has a single method, convert(), which converts the data using the Markdown instance's convert() method, and then resets the Markdown instance's internal state. Then in the CompileMarkdown.set_site() method, instantiate a new ThreadLocalMarkdown instance (unless the markdown module is unavailable) and assign it to the plugin's converter attribute. In the compile_string() method, use that ThreadLocalMarkdown instance to perform the conversion. The end result is one Markdown object created per thread, instead of one for each markdown file compiled.