Support latest realise of Markdown library

[Samreay]

I believe there has been some update to the Markdown library and how it internally records its version that is breaking things.

With a brand new environment and a fresh install of mkdocs, a mkdocs build --strict --verbose fails my project with this error:

DEBUG    -  Loading configuration file: /Users/sh/Projects/dataportalapiclient/mkdocs.yml
ERROR    -  Config value: 'markdown_extensions'. Error: module 'markdown' has no attribute 'version_info'

At this point, mkdocs has a dependency on Markdown==3.4.1, which was released three days ago.

After running a pip install Markdown==3.3.7 to downgrade the version, rerunning the build is successful:

DEBUG    -  Loading configuration file: /Users/sh/Projects/dataportalapiclient/mkdocs.yml
...
DEBUG    -  mkdocstrings: Tearing handlers down
INFO     -  Documentation built in 3.45 seconds

I notice in this commit from May 27th on the Markdown repository, the deprecated version_info info object was removed, and replaced with the __version_info__ object, as per this table:

Deprecated Object	Replacement Object
markdown.version	markdown.__version__
markdown.version_info	markdown.__version_info__
markdown.util.etree	xml.etree.ElementTree
markdown.util.string_type	str
markdown.util.text_type	str
markdown.util.int2str	chr
markdown.util.iterrange	range
markdown.util.isBlockLevel	markdown.Markdown.is_block_level
markdown.util.Processor().markdown	markdown.util.Processor().md
markdown.util.Registry().__setitem__	markdown.util.Registry().register
markdown.util.Registry().__delitem__	markdown.util.Registry().deregister
markdown.util.Registry().add	markdown.util.Registry().register

Hopefully the fix is a simple change to this dunder object! Whether this repository is the right place for the packaged markdown extension or not, I'm unsure, I couldn't quite see where that config gets run either here or in the Python Markdown library.

If this isn't the place, I'd appreciate if you can please point me towards the right repo.

[lyz-code]

I'm also facing errors with this update, when I try to run mkdocs build I get the next error:

ERROR    -  Config value: 'markdown_extensions'. Error: extendMarkdown() missing 1 required positional
            argument: 'md_globals'

Rolling back works as expected

[ultrabug]

@oprypin maybe we could first limit the Markdown version in a "hotfix" release?

See #2893

Else, we can also work on directly updating mkdocs to support newer Markdown itself of course. Thoughts?

[oprypin]

Hi. I could still be missing something, but my first point is, I am detecting absolutely no issue with MkDocs. Tests pass, random websites that I had laying around build without problems. That's one basic check that I'd hope to see. And another one is that if there's no mention of markdown.version_info and no mention of md_globals (also as a parameter) in the whole repository, then there's no reasonable way to make a blanket bug report on this repository claiming that everything is broken and that something should done about it (about what exactly?).

That is, this would require at least the list of plugins that you're using (mkdocs.yml) to be able to conclude anything. Or better, a full minimal website source that causes this error.

I am only now able to attend to this, but all I can do is look through every Python-Markdown plugin (possibly all old versions, too, because who knows..?) to guess which ones it might have been that caused these errors. (So far I found one but the error looks differently still.) But let's just step back and go to the previous paragraph.

Actually if anyone is running into such an issue, for all such cases I'd like to figure out which plugin is causing that for them. For all we know, it could be just 1 or 2 plugins in common use that should just be fixed.

Speaking of which, that is actually my main point - I do believe that it's some Markdown plugins that are breaking with the version update, not MkDocs breaking, at all. Sure, the error report comes from MkDocs and in a way that conceals which plugin is at fault. But in the end, I'll still want to see the list of those plugins to make a decision.

[oprypin]

And yes indeed, it is the Python-Markdown release 3.4 that causes all these breakages of extensions. I do believe that the best course of action would be for them to backtrack on the breakages (the PR itself doesn't even sound certain that the breakages were OK to do), but I have no influence there.

As a workaround (which I admit would immediately help people), to punt this issue some time into the future (where we will presumably be back at square one), we could indeed pin the version within MkDocs, even if that is technically clearly wrong, as MkDocs has no such limitation. And pinning deps like that is in itself a ticking time bomb. So I am not so sure about this.

What we will have to do in any case, it seems, is to track down the Python-Markdown extensions that break with this change in Python-Markdown. Concealing this issue now (as in the previous paragraph) might hinder the progress of that.

[oprypin]

And I did just mention that MkDocs does poorly here in how it reports these breakages. Absolutely all information about where this breakage comes from is concealed. Actually a big part of the "concealment" already happens in Python-Markdown itself but then MkDocs doesn't even keep the name of the extension that broke (I still haven't dug into where this happens).
This is absolutely a high priority to improve as well. I might provide a patch soon so people can at least see which extension breaks.

[ojacques]

That is, this would require at least the list of plugins that you're using (mkdocs.yml) to be able to conclude anything. Or better, a full minimal website source that causes this error.

My mkdocs sites, with extensions, broke also since the introduction of markdown-3.4.
I am able to reproduce with mdx-truly-sane-lists or mkdocs-featherlight.

This example mkdocs.yml is enough to reproduce (starting from mkdocs new .):

site_name: My Docs
markdown_extensions:
  - mdx_truly_sane_lists

The error I get for both plugin:

$ mkdocs build
ERROR    -  Config value: 'markdown_extensions'. Error: Failed loading extension "mdx-truly-sane-lists".
Aborted with 1 Configuration Errors!

Not the same as the version info from @Samreay, but I can link it to markown extension.

I also used the <3.4.0 version constraint in the meantime as a workaround.

[ultrabug]

Hi. I could still be missing something, but my first point is, I am detecting absolutely no issue with MkDocs.

I agree with you on the strict sense.
But this can become arguable in a more general sense where people and projects rely on the mkdocs requirement and thus on its listed dependencies, which is now clearly broken by Python-Markdown.

So yes, Python-Markdown broke us but I have the feeling that we ought to preserve our community from a cascade of potential failures now that we're aware of the situation.

Now, unless I'm mistaken I also would like to point out that MkDocs itself is affected by this problem when building our own mkdocs.yml website:

(fresh_virtualenv) ➜  mkdocs git:(master) mkdocs build                
ERROR    -  Config value: 'markdown_extensions'. Error: Failed loading extension "mdx_gh_links".

[ultrabug]

And I did just mention that MkDocs does poorly here in how it reports these breakages.
This is absolutely a high priority to improve as well.

I agree with you and if you feel like it, ping me on IRC so we can maybe join forces to tackle this at the same time.

[oprypin]

@ultrabug Ah at that point I was already finishing it up. That would be #2894

[oprypin]

For anyone affected by extension breakages, please install this patch (with the below command) and provide the output of MkDocs with it.

pip install git+https://github.com/mkdocs/mkdocs.git@refs/pull/2894/head

This output will also basically be ready as a bug report for whichever extension is affected, that you certainly should report in the respective repositories in any case.