Replacing black and blue?

[warsaw]

Do you have thoughts about ruff eventually subsuming and replacing black or blue? I'm one of the maintainers of blue (because we have different opinions about a few things), and it's been pretty difficult to both keep our monkey patching of black up to date, and rely on flake8's pyproject.toml support across multiple different major versions of flake8.

I'm thinking about alternatives, and this looks like a nice framework which could be a good starting point, so I'd like to get your thoughts about this project's direction. Would something like this be in or out of scope?

[charliermarsh]

I'm really interested in this and definitely consider it to be within scope. Would love to chat about how it would work and how we'd push it forward :)

To-date, I've focused on making Ruff a Black-compatible linter and code transformation tool, but I'd like it to expand to an all-in-one code formatter (ideally, though, in a way that's totally opt-in and incrementally adoptable, so users could continue to use Black as a formatter and Ruff as a linter). I think coupling a linter (with autofix capabilities) with an auto-formatter can make both tools more effective and efficient.

Philosophically, I'm comfortable enabling a higher level of configuration than is supported by black. Though it's somewhat subjective, my general philosophy towards auto-formatted settings is that I don't care what settings are used, as long as they're consistent in a codebase and automatically applied. I think Prettier has done a reasonable job of exposing a few configuration settings, like space width and quotation style. Rustfmt goes much further, maybe further than is desired in this case. But, regardless, it's critical to me that Ruff enforces a single formatting style; rather, than it's capable of enforcing and applying formatting style.

The biggest fundamental blocker to starting on auto-formatting is that we may need a CST rather than an AST. (The lexer outputs comments, so if we don't care about preserving whitespace in code, and view auto-formatting as a function of (AST + comments) => (source code), we may not actually need a CST? I haven't done enough research to understand the requirements here.)

[andersk]

More information than just AST + comments will likely be needed. Black preserves empty lines between statements (up to 1 or 2 depending on where), extra parentheses with certain specific exceptions, the distinction between normal and raw strings ("", r"", R""), and a magic trailing comma that forces vertical layout.

But maybe an AST-like type with just that extra information would still be nicer than a CST?

There’s also # fmt: off and # fmt: skip to deal with somehow.

[ofek]

This would be excellent; all users would need would be Ruff and a type checker

[charliermarsh]

Yeah, I'd like to see what minimal set of extensions we'd need to the AST and parser to support Black-compatible formatting. Part of me wonders if we could generate any extra metadata by taking the AST and augmenting it through a second pass over the token stream... For example, we could collect comments and empty lines. We could also tag strings as normal or raw (though that's not captured in the token stream -- we'd have to look at the actual source), and so on.

(We could move to LibCST and generate code from a CST representation, but I'm really hesitant since the parser is so much slower and the representation is indeed more complex.)

[charliermarsh]

For more inspiration, there's also tokenize-rt, which implements round-tripping for Python's tokenize. However, this would require that the entire autoformatter operate on tokens rather than an AST, which seems challenging.

[charliermarsh]

Closing in favor of #1904. I'm going to start work on this.

(I wrote more in #1904, but I'd like the autoformatter to be a little more flexible than Black, which would in theory make it a suitable replacement for Blue too.)