add support for formatting reStructuredText code snippets #9003

This fixes a bug where if the very last line in a docstring is a doctest, then it wasn't getting formatted. We do a little more than fix that bug in this commit. In particular, we industrialize how we handle actions by using a queue. Instead of trying to do too much with each action, we make each one a bit simpler and build the infrastructure necessary to permit more arbitrary composition. This in particular lets us handle weird corner cases like "the code example and the docstring end at the same time" without much fuss. In essence, actions like Format and Reset no longer carry the "original" line. Instead of doing that, we generate two actions: the first to format and the second (if necessary) to print the original line. This infrastructure will become more apparently useful when dealing with reStructuredText blocks.

There was really no good reason to have it panic if the code block given is empty. Instead, we can just return `None` and everything will get handled correctly. (It will turn into a `Reset` with zero lines, which will in turn be a no-op.) In practice I do believe empty code blocks are not possible, but I felt like this makes the code a bit more robust.

The comment in the source code should explain, but basically, if the user has tab indentation enabled, then this winds up screwing with code snippet formatting. The essential problem seems to be that docstring normalization strips tabs, and this interplay between code snippet reformatting with tabs winds up making formatting non-idempotent. We justify this hard-coded option by pointing out that tabs get stripped as part of docstring normalization anyway.

Small style tweak. . o O { too bad `imports_granularity` isn't stable in rustfmt yet }

This commit makes use of the refactoring done in prior commits to slot in reStructuredText support. Essentially, we add a new type of code example and look for *both* literal blocks and code block directives. Literal blocks are treated as Python by default because it seems to be a common practice[1]. That is, literal blocks like this: ``` def example(): """ Here's an example:: foo( 1 ) All done. """ pass ``` Will get reformatted. And code blocks (via reStructuredText directives) will also get reformatted: ``` def example(): """ Here's an example: .. code-block:: python foo( 1 ) All done. """ pass ``` When looking for a code block, it is possible for it to become invalid. In which case, we back out of looking for a code example and print the lines out as they are. As with doctest formatting, if reformatting the code would result in invalid Python or if the code collected from the block is invalid, then formatting is also skipped. A number of tests have been added to check both the formatting and resetting behavior. Mixed indentation is also tested a fair bit, since one of my initial attempts at dealing with mixed indentation ended up not working. Closes #8859 [1]: adamchainz/blacken-docs#195

This is what seems consistent with the prevailing code.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for formatting reStructuredText code snippets #9003

add support for formatting reStructuredText code snippets #9003

Commits on Dec 5, 2023