config: add new docstring-code-format knob
#8854
Conversation
|
|
There's also a question of whether this ought to be a boolean option at all. @konstin raised this here: #8811 (comment) That is, we might want the formatter to specifically raise an error if it comes across a snippet that is invalid Python. I somewhat think this would be better as a lint, but it would likely be much easier to implement in the formatter. If it were a lint, then we'd need to refactor the code snippet logic out of the formatter into something more generalized. |
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
from
d6148b6
to
1a75fdd
Compare
There was a problem hiding this comment.
The change itself looks good to me but I would prefer changing the option name (also on PyFormatOptions).
Rustfmt uses format_code_in_doc_comments which we could change to format_code_in_docstrings.
I prefer the name because
- It's unclear to me what
snippetsmeans without prefixing it withcode-which I find to verbose format-embeddedmight be ambiguous if we start formatting embedded language, let's say SQL or CSS in your Python code (the other way round)- I'm okay with
format-docstring-code, although I find rustfmt's naming clearer.
I'm okay with using a boolean. I don't think we should bail formatting just because of an incorrect example. Ideally we would emit a warning, but that's tracked as a separate issue.
The other question: Should this be gated behind preview mode (sorry if this is part of your other PR)? If so, should we emit a warning if it is enabled but the formatter is run without preview mode?
|
I like
The idea is opt-in without preview opt-out (i.e. on by default) with preview. Having a "double" opt-in seems overkill. |
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
from
1a75fdd
to
0071e7d
Compare
|
We have a winner. |
|
Works for me :) |
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
from
0071e7d
to
78e88c3
Compare
BurntSushi
changed the title
format-code-in-docstrings knob
(This is not possible to actually use until #8854 is merged.) ruff_python_formatter: add reStructuredText docstring formatting support This commit makes use of the refactoring done in prior commits to slot in reStructuredText support. Essentially, we add a new type of code example and look for *both* literal blocks and code block directives. Literal blocks are treated as Python by default because it seems to be a [common practice](adamchainz/blacken-docs#195). That is, literal blocks like this: ``` def example(): """ Here's an example:: foo( 1 ) All done. """ pass ``` Will get reformatted. And code blocks (via reStructuredText directives) will also get reformatted: ``` def example(): """ Here's an example: .. code-block:: python foo( 1 ) All done. """ pass ``` When looking for a code block, it is possible for it to become invalid. In which case, we back out of looking for a code example and print the lines out as they are. As with doctest formatting, if reformatting the code would result in invalid Python or if the code collected from the block is invalid, then formatting is also skipped. A number of tests have been added to check both the formatting and resetting behavior. Mixed indentation is also tested a fair bit, since one of my initial attempts at dealing with mixed indentation ended up not working. I recommend working through this PR commit-by-commit. There is in particular a somewhat gnarly refactoring before reST support is added. Closes #8859
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
from
78e88c3
to
f671ce1
Compare
(This is not possible to actually use until #8854 is merged.) This commit slots in support for formatting Markdown fenced code blocks[1]. With the refactoring done for reStructuredText previously, this ended up being pretty easy to add. Markdown code blocks are also quite a bit easier to parse and recognize correctly. One point of contention in #8860 is whether to assume that unlabeled Markdown code fences are Python or not by default. In this PR, we make such an assumption. This follows what `rustdoc` does. The mitigation here is that if an unlabeled code block isn't Python, then it probably won't parse as Python. And we'll end up skipping it. So in the vast majority of cases, the worst thing that can happen is a little bit of wasted work. Closes #8860 [1]: https://spec.commonmark.org/0.30/#fenced-code-blocks
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
2 times, most recently
from
3d5d433
to
a8f9c58
Compare
…#9055) ## Summary This does the light plumbing necessary to add a new internal option that permits setting the line width of code examples in docstrings. The plan is to add the corresponding user facing knob in #8854. Note that this effectively removes the `same-as-global` configuration style discussed [in this comment](#8855 (comment)). It replaces it with the `{integer}` configuration style only. There are a lot of commits here, but they are each tiny to make review easier because of the changes to snapshots. ## Test Plan I added a new docstring test configuration that sets `docstring-code-line-width = 60` and examined the differences.
BurntSushi
changed the title
format-code-in-docstrings knobdocstring-code-format knob
|
I think this was talked about internally and not here on the PR, but we ended up settling on |
|
Note: |
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
2 times, most recently
from
5405750
to
20ef1a4
Compare
BurntSushi
requested review from
charliermarsh,
MichaReiser,
konstin and
zanieb
|
All righty, I think this PR is now in good spot where another review would be good. The probable changes since the last time you looked:
|
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
2 times, most recently
from
f3e8073
to
a6510f7
Compare
|
@charliermarsh I've added prose docs and fixed up your comments. A review of the prose docs would be great! Thanks. |
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
from
a6510f7
to
cd68e6c
Compare
…namic' I do feel like this is probably the right default and perhaps will be the least surprising. It also has the benefit of avoiding the line length lint in all cases.
…th' knobs This commit does the plumbing to make a new formatting option, 'docstring-code-format', available in the configuration for end users. It is disabled by default (opt-in). It is opt-in at least initially to reflect a conservative posture. The intent is to make it opt-out at some point in the future. This also adds a compansion option, 'docstring-code-line-length', that permits setting a line length for reformatted code examples that is distinct from the global setting. Its default value is 'dynamic', which means reformatted code will respect the global line width, regardless of the indent level of the enclosing docstring.
BurntSushi
force-pushed
the
ag/fmt/user-config
branch
from
cd68e6c
to
bac1d2f
Compare
|
This feature is most excellent, thank you for working on it. |
This PR does the plumbing to make a new formatting option,
docstring-code-format, available in the configuration for end users. It is disabled by default (opt-in). It is opt-in at least initially to reflect a conservative posture. The intent is to make it opt-out at some point in the future.This was split out from #8811 in order to make #8811 easier to merge. Namely, once this is merged, docstring code snippet formatting will become available to end users. (See comments below for how we arrived at the name.)
Closes #7146
Test Plan
Other than the standard test suite, I ran the formatter over the CPython and polars projects to ensure both that the result looked sensible and that tests still passed. At time of writing, one issue that currently appears is that reformatting code snippets trips the long line lint: https://github.com/BurntSushi/polars/actions/runs/7006619426/job/19058868021