[Python][Docs] API docs are broken for dev version

[AlenkaF]

Describe the bug, including details regarding any error messages, version, and platform.

Development version of the docs seems to be broken for generated Python API Reference documentation.

Example: https://arrow.apache.org/docs/dev/python/generated/pyarrow.Schema.html

Did a hard refresh, hope it is not just me?

Component(s)

Documentation, Python

[AlenkaF]

cc @raulcd

[jorisvandenbossche]

This started with the update apache/arrow-site@05cb617 from the nightly build of 2024-03-12, so that matches the timeframe of potential candidate cause #40442

[kou]

#40442 may be related...: #40442 (comment)

But it has different result:

[jorisvandenbossche]

That looks like the same issue

[jorisvandenbossche]

Starting to run the doc build locally, it seems that the issue is that the generated API files are now generated as .md files (but there content is restructuredtext).

With https://www.sphinx-doc.org/en/master/man/sphinx-autogen.html it seems there is an option to specify which extension to use, but not sure how that can be tweaked from conf.py

It seems that adding myst_parser to the list of extensions has (unintended) effect on the default extension that sphinx uses for this ..

[kou]

Oh... Does this solve?

diff --git a/docs/source/conf.py b/docs/source/conf.py
index 7915e2c2c4..b67833112b 100644
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -208,8 +208,8 @@ templates_path = ['_templates']
 #
 
 source_suffix = {
-    '.md': 'markdown',
     '.rst': 'restructuredtext',
+    '.md': 'markdown',
 }
 
 autosummary_generate = True

[kou]

Hmm. Sphinx may have a problem:

diff --git a/sphinx/ext/autosummary/__init__.py b/sphinx/ext/autosummary/__init__.py
index 22e616c3f..9a469d23e 100644
--- a/sphinx/ext/autosummary/__init__.py
+++ b/sphinx/ext/autosummary/__init__.py
@@ -770,7 +770,7 @@ class AutoLink(SphinxRole):
 
 def get_rst_suffix(app: Sphinx) -> str | None:
     def get_supported_format(suffix: str) -> tuple[str, ...]:
-        parser_class = app.registry.get_source_parsers().get(suffix)
+        parser_class = app.registry.get_source_parsers().get(suffix.removeprefix('.'))
         if parser_class is None:
             return ('restructuredtext',)
         return parser_class.supported

[jorisvandenbossche]

Oh... Does this solve?

That indeed seems to do the trick ..

[kou]

• autosummary may output .md files with MyST-Parser sphinx-doc/sphinx#12147
• [autosummary] fix suffix detection sphinx-doc/sphinx#12149

[kou]

The fix was merged.

[AlenkaF]

Issue resolved by pull request 40739
#40739