Foo = NewType(...) is treated as data and not as a class

[orenbenkiki]

Describe the bug

Saying Foo = NewType(...) stored Foo as data and not as class. When using Foo in a type annotation, sphinx searches for it with the objtypes class and exception but fails to find it since it is stored as data, so it generates a warning that the reference target is not found.

How to Reproduce

$ git clone https://github.com/orenbenkiki/sphinx-bug
$ cd sphinx-bug/docs
$ make html

The docs/ERRORS file contains sample result, including the warning line:

/home/obk/sphinx-bug/bug.py:docstring of bug.foo_to_str:: WARNING: py:class reference target not found: bug.Foo

Expected behavior

sphinx should be able to figure out the type references and emit the right links without warnings when using NewType.

Your project

https://github.com/orenbenkiki/sphinx-bug

Screenshots

No response

OS

Linux

Python version

3.10

Sphinx version

4.5.0 (latest PyPI version)

Sphinx extensions

sphinx.ext.autodoc sphinx.ext.autosectionlabel sphinx.ext.viewcodesphinx.ext.todo

Extra tools

No response

Additional context

One brutal way to fix this is to add data to the objtypes when searching for a class, that is, in line 1196 of domains/python.py, change the line:

'data': ObjType(_('data'), 'data', 'obj')

To:

'data': ObjType(_('data'), 'data', 'class', 'obj')

But that is not the right way to solve the problem - it was helpful in figuring out what the problem was, though.

[AA-Turner]

Simplified single file reproducer (run python reproducer.py)

from pathlib import Path
import shutil

from sphinx.cmd.make_mode import run_make_mode

Path("bug.py").write_text('''\
"""Module docstring."""
from typing import NewType

Foo = NewType("Foo", str)


def foo_to_str(foo: Foo) -> str:
    """Convert to string."""
    return foo
''', encoding="utf-8")

Path("conf.py").write_text('''\
extensions = ['sphinx.ext.autodoc']
nitpicky = True
''', encoding="utf-8")

Path("index.rst").write_text('''\
Bug
---

.. automodule:: bug
    :members:
''', encoding="utf-8")

shutil.rmtree("_build", ignore_errors=True)
run_make_mode(["html", ".", "_build"])

[AA-Turner]

@tk0miya in #8465 you made NewType types py:data rather than py:class. sphinx.domains.python.type_to_xref() assumes that all annotation parameters are types that can be annotated with py:class, but doesn't have access to the actual Python object so we can't introspect.

It seems it would be easier to update autodoc to generate py:class for NewType types, but this may have unintended consequences?

A

[tk0miya]

I agree that autodoc should generate py:class for NewType objects.

[anselor]

After a few hours of digging through what's happening with my documentation generation I finally tracked it down to this problem and then found this.
Is there any work-around that can be done to temporarily force it to generate py:class for NewType objects?

[anselor]

Ok, so adding this hack to my conf.py seems to make NewTypes resolve and gets rid of all of the warnings referencing NewType definitions:

sphinx.ext.autodoc.NewTypeDataDocumenter.directivetype = 'class'

I've having the same problem with TypeVar declarations - it's also being treated as data.

[anselor]

@AA-Turner Since it's the same type of issue, can TypeVar also be changed to be py:class instead of py:data in the next release?