Remove black, use ruff for generated config_models#23588
Open
Remove black, use ruff for generated config_models#23588
Conversation
…ig models - Remove explicit `black==23.12.1` from `datadog_checks_dev` - Replace `apply_black` calls in the model consumer with `ruff format -`, using the repo's centralized `[tool.ruff]` configuration - Drop the now-unused `code_formatter` plumbing through `ModelConsumer` / `build_model_file` / `validate models` - Drop the [tool.black] block from `ddev/pyproject.toml` and the matching python-version-bump logic in `update_py_config.py` (with test fixture) - Update README badge from black to ruff The root `[tool.black]` section is kept (with an explanatory comment) because `datamodel-code-generator` reads it transitively through its own internal formatter, and removing it changes line-length to 88 which breaks our list[...] -> tuple[..., ...] line-by-line transform.
Contributor
|
|
|
✨ Fix all issues with BitsAI or with Cursor
|
Codecov Report❌ Patch coverage is Additional details and impacted files🚀 New features to boost your workflow:
|
… ruff
CI installs ddev with `pip install -e ./datadog_checks_dev[cli]` and never
adds ruff to PATH. The previous helper called `shutil.which('ruff')` and
silently returned the input unchanged when ruff was missing — leaving long
lines and missing wraps in 401 generated config-model files, surfacing as
"not in sync" in the validate workflow.
- Declare `ruff>=0.11` in `datadog_checks_dev[cli]` so the package is
always installed alongside the model generator.
- Switch the helper to `sys.executable -m ruff` so the in-venv package is
used regardless of PATH.
- Raise loudly on missing ruff or non-zero ruff exit instead of silently
degrading, so any future regression fails the workflow with a clear error.
Make `_fix_types` operate on the joined document (as UTF-8 bytes) instead of line by line, so the bracket-tracking pass works regardless of how datamodel-code-generator's internal formatter wrapped `list[...]`. Place the `, ...` sentinel right after the last non-whitespace byte before the closing `]`, so output stays on the previous content line even when the parser pre-wrapped the closing bracket onto its own line. With those changes the generator no longer relies on `[tool.black]` existing in the repo, so the section and its accompanying comment are removed from `pyproject.toml`. The black-related comment near the config_models lint exclusion and the black badge in `README.md` go too. Four config_models files (kafka_actions, win32_event_log, yarn x2) regenerate with different — but semantically identical — wrapping. They were the only ones whose pre-wrapped form was sensitive to the change in upstream line-length default; future regens are stable.
AAraKKe
changed the title
- Replace remaining "code style - black" references in developer docs (`docs/developer/index.md` badge, `docs/developer/guidelines/style.md` style section, link reference) with ruff equivalents. - Update the stale `ddev test postgres -l` example output in `docs/developer/testing.md` to drop `black==22.12.0` and reflect the current lint env contents (`ruff==0.11.10`, `pydantic==2.11.5`). - Move the "ruff is not installed" install hint in `format_with_ruff` from the `FileNotFoundError` branch to the `CalledProcessError` branch and gate it on `"No module named 'ruff'"` in stderr — the previous layout was effectively dead code because `sys.executable` always resolves, so missing-ruff surfaces as a non-zero exit, not a missing binary. - Add `tests/tooling/configuration/consumers/model/test_fix_types.py` with focused coverage for the `_fix_types` post-processing pass: the multiline-wrapped `list[Literal[...]]` regression case the PR was written to fix, dict and nested-list translations, unicode in descriptions, and verbatim pass-through when no `list[`/`dict[` is present.
code_formatter.py:
- Guard `_resolve_ruff_config` with `if root_str:` so an unset
`get_root()` (returns '') doesn't fall into the `Path('').is_dir()`
branch (which is True — it resolves to CWD), and unit tests actually
walk back to the repo pyproject.toml as the docstring claims.
- Replace the loose `'[tool.ruff' in text` substring with a line-anchored
scan that only matches actual TOML table headers (`[tool.ruff]` or
`[tool.ruff.…]`), so a comment or string value can't false-positive.
- Surface argv (via `shlex.join`), stderr, and stdout in the error
message for non-missing-package failures, so a future ruff config
change emitting actionable output is debuggable from the message alone.
Tests:
- New `test_code_formatter.py` (17 tests): direct coverage for
`format_with_ruff` (line wrapping, quote-style preservation, short
passthrough, missing-ruff hint, full-context error on other failures)
and `_resolve_ruff_config` (root path success, fallback walk on empty
root, fallback walk when root has no `[tool.ruff]`, returns None when
nothing is found, parametrized header recognition for `_has_ruff_section`).
- `test_update_py_config.py`: add explicit content assertions on the
rewritten `ddev/pyproject.toml` — no `[tool.black]` block survives,
`[tool.ruff].target-version` is updated to the new pinned version, the
old token is gone. Captures the actual contract instead of relying on
the success-counter being 9.
This PR bumps `datadog-checks-dev` to 39 (`black` is dropped from `[cli]` extras, so per semver it's a major). The current `~=38.0` constraint in ddev's `pyproject.toml` would block any GitHub Action that installs both packages from the local repo — `pip install -e ./ddev` would fail to resolve the local dcd 39 against ddev's pin. Relax the pin to `>=38.0,<40` for the duration of the gap between this PR landing and the next ddev release. The release PR for ddev MUST tighten this back to `~=39.0`.
Contributor
Validation ReportAll 20 validations passed. Show details
|
domalessi
approved these changes
May 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Drops
blackas a declared dependency of this repo and removes every reference to it (root and ddevpyproject.toml, root and ddevREADME.mdbadges, and theupdate_py_configpython-version-bump path). Generated config models are now formatted viaruff formatusing the repo's centralized[tool.ruff]configuration.Motivation
ruff formatalready powers everything else in the toolchain —ddev test -fs, the hatchlintenv, and CI's lint step. The only remaining direct uses of black were:black==23.12.1indatadog_checks_dev[cli]apply_blackcalls inside the model generator[tool.black]config blocks (root +ddev/pyproject.toml) and the python-version-bump logic that touched themThis PR removes all of that.
Changes
Replace
apply_blackwith a ruff-based helperdatadog_checks_dev/.../configuration/consumers/model/code_formatter.pywithformat_with_ruffthat shells out topython -m ruff format -(uses the active interpreter so the in-venvruffis always picked up regardless ofPATH)[tool.ruff]config viaget_root()first, falling back to walking up from the module's own path so unit tests (which never callset_root) still find itruff/ non-zero ruff exit instead of silently degradingmodel_consumer.py,model_file.py) and thecode_formatterparameter plumbing throughModelConsumer.__init__,build_model_file, andvalidate modelsare all removedMake
_fix_typeswhole-document[tool.black]was removed, datamodel-code-generator's internal black fell back toline-length=88and started pre-wrappinglist[...]across multiple lines, which silently dropped the, ...ellipsis (turningtuple[X, ...]— variable-length tuple — intotuple[X]— single-element tuple). Real type-contract regression., ...sentinel is inserted after the last non-whitespace byte before], so it lands on the same line as the inner content.[tool.black]entirely.Dependency / config / cosmetic cleanup
datadog_checks_dev/pyproject.toml: dropblack==23.12.1, addruff==0.11.10(pinned to matchddev's hatch lint env, with a comment about keeping the pin in sync until commands migrate fromdatadog_checks_devintoddev)pyproject.toml: drop[tool.black]block and the related comments; remove the stale "formatted with Black" note in the lint-exclusion blockddev/pyproject.toml: drop[tool.black]blockddev/src/ddev/cli/meta/scripts/update_py_config.py: drop the black-handling branch inupdate_ddev_pyproject_fileddev/tests/cli/meta/scripts/{conftest.py,test_update_py_config.py}: update fixture and assertion (one fewer success inupdate-python-config→ 9 instead of 10)README.mdandddev/README.md: replace black badges with ruffOne-time regen of 4 config_models
kafka_actions,win32_event_log, andyarn(instance + shared) regenerate with different — but semantically identical — wrapping. They were the only files whose pre-wrapped form was sensitive to the change in upstream line-length default; future regens are stable.<pkg>/changelog.d/23588.fixedentry.Note on transitive black
datamodel-code-generator~=0.25.6still depends onblackand uses it internally duringOpenAPIParser.parse(). We can't remove that without bumping the generator (newer 0.26+ supports aformatterschoice that includes ruff). With this PR, we no longer declare black, no longer call black, and no longer configure black — but black still rides in viadatamodel-code-generator. That's a follow-up change.Verification
ddev validate models -s(zero diff against committed state on a fresh run).datadog_checks_dev/tests/tooling/configuration/: 231 passed.ddev/tests/cli/meta/: 36 passed.Run Validations / Validate✅,Check PR changelog✅, ddev/datadog_checks_dev test workflows on Linux and Windows ✅, all three affected integration test workflows ✅.Review checklist (to be filled by reviewers)