Pandas and Requests classes not linked

[akaihola]

I have intersphinx configured properly so that the following markup all link correctly to the relevant documentation:

:class:`pandas.DataFrame`
:class:`requests.Request`
:class:`our_in_house_package:OurClass`

However, all of these type hints fail to render as links to Pandas documentation, even though the type annotation rendering using sphinx_autodoc_typehints looks otherwise just fine:

import pandas as pd
import pandas

def test1() -> pd.DataFrame:
    ...

def test2() -> pandas.DataFrame:
    ...

def test3() -> pd.core.frame.DataFrame:
    ...

def test4() -> pandas.core.frame.DataFrame:
    ...

The same goes for classes inside the Requests package.

This is from our conf.py:

intersphinx_mapping = {
    'pandas': ('http://pandas.pydata.org/pandas-docs/stable/', None),
    'our_in_house_package': ('http://our_host/path/', None),
    'requests': ('http://docs.python-requests.org/en/master/', None)}

[agronholm]

This library looks at the module in which the annotation was defined and generates the Sphinx markup based on that. How would you prefer me to fix this?

[kristofve]

I think the problem is that pd.DataFrame always gets expanded to :py:class:`~pandas.core.frame.DataFrame` . pandas.core.frame.DataFrame is not listed in the objects.inv file, whereas pandas.DataFrame is, so intersphinx doesn't find it. I encountered the same issue with things like plt.Axes (becomes matplotlib.axes._axes.Axes), while plt.Figure works fine because matplotlib.figure.Figure has an entry.

I am not sure of where the problem should be fixed. Is it possible to only resolve the import alias (e.g. pd.DataFrame --> pandas.DataFrame) while keeping the rest of the type hint as is? Maybe as an optional flag, because I can see advantages and disadvantages to that approach.

[agronholm]

Is it possible to only resolve the import alias (e.g. pd.DataFrame --> pandas.DataFrame) while keeping the rest of the type hint as is?

How do you propose to do the resolution?

[kristofve]

I have taken a quick look at how the autodoc-process-signature and autodoc-process-docstring callback functions work. It seems like the signature and obj parameters only contain the fully expanded python module names. Without a proper reference to the parsed file, it looks very tricky to properly fix this issue from within sphinx-autodoc-typehints.

An easier possible solution could be to do a search for the module name in the relevant objects.inv file and replace the signature text with a match, but this approach would be slow and probably out of scope for this extension as it's only useful for intersphinx users.

[agronholm]

Then you have arrived at the exact same conclusion as I.
This issue could be reopened if someone finds a workable solution.

[rmorshea]

I'm presently working on a sphinx extension at my company that resolves this and other annoyances with missing references. It will take a bit of time to open source though as I've been pretty busy lately.

[brenns10]

As a note, although this does result in a bit of repetition, you can simply include :rtype: pandas.DataFrame in your docstring for the function, and this will override the autodoc suggestion, so you get a correct link.

[hagenw]

Could you maybe reopen this issue and mark as wontfix or something like this as the issue itself is still present and very annoying. This would then make it easier to find it again.

[rmorshea]

I haven't had to time to test out the release yet, but here's the extension I mentioned above:

https://github.com/23andMe/sphinx-resolve-py-references

pip install sphinx-resolve-py-references

And here's what documentation I've been able to write about it...

---

Fixes or warns about missing references to Python objects in the docs

The extension is useful because it's often annoying to write out the full path to an object in your docs which you know is imported in the current module. This is also true for neighboring modules which would be easier referenced with
a relative name.

This extension addresses these issues by allowing you to reference functions or classes which are imported in the current module as if they had been defined there. This is done by analyzing the AST to find and trace import statements back to the place where the target was defined. This is more robust than looking at the __module__ and __name__ attributes of classes and functions because it won't work for module attributes that have been documented (e.g. global variables).

from some_package import MyClass

def my_function():
    '''Uses :class:`MyClass`'''
    # The above reference would normally fail because `MyClass`
    # is not defined in this module. However this extension
    # automatically resolves the reference

[brenns10]

Throwing my hat into the ring, I came up with my own fix for this which is also a Sphinx extension:
https://github.com/brenns10/sphinx-reference-rename

It just lets you define a dictionary mapping the names which didn't resolve, to the correct ones. See the readme in that repository for instructions, the example should fix the original problem at the top of this issue.

I didn't actually try out @rmorshea's option first, so it may be better than mine for all I know. Mine is very quick and simple, the whole thing is like 20 lines of Python, mostly imports :)

[hagenw]

The issue described here is no longer occurring to me, at least links to pd.DataFrame are working.

[brenns10]

Glad to hear it! Wonder what fixes it for you. The only thing that helps me is my workaround, otherwise pd.DataFrame and other types don't link properly.

[hagenw]

Don't know. I use the following entry in intersphinx_mapping:

'pandas': ('https://pandas.pydata.org/pandas-docs/stable/', None),

compare https://github.com/audeering/audformat/blob/v0.15.0/docs/conf.py
which results in the following page with working pandas links:
https://audeering.github.io/audformat/api.html