Investigate how documentation tooling treats indents in docstrings

[konstin]

In the formatter, we currently normalize docstring indentation in the same way as black and cpython do. For example this is what cpython does, using str.expandtabs with a fixed width of 8 on python 3.13:

>>> exec('def f():\n\t"""Computes the value.\n\tx = 1\n\t\ty = x + 1\n\t"""')
>>> f.__doc__
'Computes the value.\nx = 1\n        y = x + 1\n'

This becomes a problem when using the formatter with tab indentation, esp. with a tab width that is not 8, because we're either inconsistent with tab indentation or mismatch what cpython does. We should investigate how the various docs tools handle docstring indents. If the indent size or kind doesn't matter to them, we have more freedom in e.g. converting to tabs inside docstrings, otherwise we need to know which properties we need to preserve.

[Mutantpenguin]

Is there any way to suppress formatting of docstrings for now?

[charliermarsh]

You can add # fmt: skip to the end:

def func():
    """abc
def
    ghi
    """  # fmt: skip

But there is no dedicated setting to enforce this globally (and, candidly, I think we're unlikely to add one).

[Mutantpenguin]

Thanks, but frankly that's just not feasible for any bigger codebase.

I added some examples to #8430 to show what the current behaviour does to our codebase. That just looks wrong, regardless if E101 is activated or not.

If E101 would ignore multiline strings and the formatter would use the value of indent-width then this might be a valid compromise.

[charliermarsh]

Makes sense! I'm sure we'll make some improvements to the tab-in-docstring behavior (hence this issue). It just hasn't been prioritized yet. But I'd rather spend time fixing the root cause than investing in other workarounds.

[sciyoshi]

@konstin I don't see the same behavior as you:

Python 3.11.6 (main, Oct 23 2023, 22:48:54) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exec('def f():\n\t"""Computes the value.\n\tx = 1\n\t\ty = x + 1\n\t"""')
>>> f.__doc__
'Computes the value.\n\tx = 1\n\t\ty = x + 1\n\t'

Sphinx seems to be configurable using the tab-width option.

[sciyoshi]

(As pointed out on Discord, this behavior will be changing in Python 3.13.)

[MichaReiser]

• pydoc uses cleandoc, which expands tabs to 8 spaces (source)
• pydoc uses getdoc which uses cleandoc (source)
• doxygen has a tab size option. But what I understand is that it is only used for code snippets (it uses a DSL for the documentation)
• sphinx has a tab width option

[MichaReiser]

I'm closing this issue. Most documentation tools use a version of inspect.cleandoc. I leave a more in depth reply in

#8430