Conversation
Replace pylint with a coherent ruff-only stack (Stage 2c of the modernize
roadmap). Pylint is dropped from dev deps and CI workflows; ruff selected
ruleset expanded to ~22 families covering style, bug catchers, hygiene,
and the pylint port (PLE/PLC/PLW/PLR with the noisy "too-many-X"
complexity caps + magic-value disabled).
What's selected (with rationale in pyproject.toml):
E, W, F, I, N — style + correctness baseline + naming
UP — pyupgrade (py>=3.10 modernizations)
B, C4, C90, RET, SIM, TRY — bug catchers
PIE, ICN, T20, TC, TID, PTH, PERF — hygiene
PLE, PLC, PLW, PLR — pylint port (PLR complexity caps ignored)
RUF — ruff-native (incl. RUF100 unused-noqa)
What's removed:
- .pylintrc (root) — duplicate of the ingestion pylint config
- [tool.pylint.*] block in ingestion/pyproject.toml (~140 lines)
- ingestion/plugins/{print_checker,import_checker}.py + tests + README
(replaced by built-in T20 + TID251 banned-api respectively)
- pylint dep from ingestion/setup.py and openmetadata-airflow-apis/pyproject.toml
- `make lint` Makefile target + the pylint invocation in py_format_check
- dead pylint TODO comment + ignored test entry in noxfile.py
Cwd-stable config: ruff is invoked both from the repo root (pre-commit,
CI) and from ingestion/ (`make py_format_check`). The `src`,
`extend-exclude`, and per-file-ignores entries are listed twice — once
relative to ingestion/ and once with the `ingestion/` prefix — so
first-party isort detection and exclusions match in both invocations.
Grandfathering: ran `ruff check --add-noqa` once + format-stable
iteration. ~12,130 noqa directives across ~1,400 files. Cleanup is
deferred to follow-up PRs that drop noqas one rule at a time.
Documentation sweep: replaced `make lint` references in CLAUDE.md,
AGENTS.md, DEVELOPER.md, copilot-instructions, and 6 SKILL files with
the apply+verify shape `make py_format && make py_format_check`.
`make py_format` is NOT a strict superset of pylint — it only applies
auto-fixable violations; `make py_format_check` catches the rest.
Basedpyright baseline regenerated: ruff format reflowed multi-line
signatures in ~70 files, shifting type-error column positions. The
basedpyright baseline matches by (file path, error code, range), so
column shifts caused 19 entries to mis-align. Net diff is small
(154 lines in/out of the 13MB baseline.json) — purely positional.
Verified locally:
- make py_format_check → All checks passed
- nox --no-venv -s static-checks → 0 errors, 0 warnings, 0 notes
IceS2
requested review from
a team,
akash-jain-10,
harshach and
tutte
as code owners
github-actions
Bot
added
Ingestion
safe to test
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Three remaining stale-tooling references after Stage 2c:
- `ingestion/noxfile.py` `lint` session was still calling `black --check`,
`isort --check-only`, `pycln --diff`. Those tools aren't installed
anywhere (we dropped them from dev deps). Replace with the ruff
equivalents that mirror `make py_format_check`.
- `skills/standards/code_style.md`: stack listed as `black + isort +
pycln`; line length claimed 88 (black default). Both wrong: stack is
ruff, line length is 120.
- `skills/connector-building/SKILL.md`: `make py_format` comment said
`# black + isort + pycln`. Same swap.
Resolved conflicts in two files where main's PR #27728 swapped `logger.warning` → `logger.error` while Stage 2c added noqa markers: - ingestion/src/metadata/clients/domo_client.py:168 (TRY201 noqa preserved) - ingestion/src/metadata/ingestion/source/pipeline/spline/utils.py two log blocks (TRY400 noqa preserved on traceback.format_exc lines) Re-ran `ruff check --add-noqa` on the merged files; main's auto-merged warning→error swap introduced 94 new TRY400 violations across data-quality validators which are now grandfathered in line. Regenerated basedpyright baseline in a Linux Docker container so column positions match CI (Ubuntu) exactly. Local macOS arm64 will see column- shifted entries due to stub differences between macOS and Linux wheels; `--baselinemode=discard` mode tolerates the drift downstream. Pinned `pythonPlatform = "Linux"` in pyproject.toml so CI analysis is platform- stable across runners.
Per gitar-bot's review on PR #27774: 1. Main's PR #27728 promoted ~60 `logger.warning()` → `logger.error()` inside `except` blocks. Those changes landed on main with their own baseline updates. Our PR doesn't promote anything — the merge from origin/main brought those `error` calls along with their baseline entries. The bot interpreted the `# noqa: TRY400` we added next to those lines as us silencing the rule case-by-case. Cleaner: globally ignore TRY400 in pyproject.toml, with a comment explaining why the codebase's `logger.error(...)` + separate `logger.debug(traceback.format_exc())` pattern is intentional. Strip ~430 per-line `# noqa: TRY400` markers from source. 2. Document that `S101` in `per-file-ignores` is a forward-looking entry — flake8-bandit (`S`) is not yet selected, so the rule is no-op today; the entry stays so when `S` lands later, tests don't immediately error. Reverts the platform pin and Linux Docker–generated baseline. Keep main's baseline intact and let CI surface the exact column-shifted entries; the team will decide whether to fix in-place (revert format on affected files) or add per-line `# pyright: ignore` markers.
Main's baseline was stale relative to recently-added connectors (McpConnection, CustomDriveConnection) that lack common attributes like `hostPort`, `database`, `catalog` etc. — all sites that access those attributes via the union-typed `serviceConnection.root.config` fire `reportAttributeAccessIssue` errors that aren't baselined. 71 errors + 58 warnings absorbed. Local macOS regen; pushing to see CI's drift count. Per the basedpyright-baseline-and-ci PR experience, macOS↔Linux column drift on this size of regen has historically been 1-7 residuals.
Code Review ✅ Approved 2 resolved / 2 findingsReplaces pylint with ruff for static analysis, expanding coverage to Stage 2c. Addresses redundant S101 ignore rules and corrects log severity semantics from the bulk warning-to-error promotion. ✅ 2 resolved✅ Quality: Dead per-file-ignore: S101 not enforced since S family unselected
✅ Quality: Bulk warning→error promotion changes log severity semantics
OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
IceS2
changed the title
| "startColumn": 35, | ||
| "endColumn": 41, |
There was a problem hiding this comment.
updating the baseline that tells basedpyright where the issues are. It is generated by basedpyright by runnning:
There was a problem hiding this comment.
does it skip these existing failures ?
🟡 Playwright Results — all passed (10 flaky)✅ 3964 passed · ❌ 0 failed · 🟡 10 flaky · ⏭️ 86 skipped
🟡 10 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
3 participants
Summary
Replace pylint with a coherent ruff-only stack. Pylint is dropped from dev deps and CI workflows; ruff's selected ruleset expands to ~22 families covering style, bug catchers, hygiene, and the pylint port (
PLE/PLC/PLW/PLR, with the noisy "too-many-X" complexity caps and magic-value comparison disabled).This is Stage 2c of the modernize roadmap: tooling-only swap, all existing violations grandfathered via
# noqa: <CODE>(~12,130 markers across ~1,400 files). Cleanup is deferred to follow-up PRs that drop noqas one rule at a time.What's selected
Each rule family is annotated inline in
ingestion/pyproject.tomlwith a one-line rationale. Highlights:E,W,F,I,N,UPB,C4,C90,RET,SIM,TRYPIE,ICN,T20,TC,TID,PTH,PERFPLE,PLC,PLW,PLR(with PLR0904/0911-0917, PLR2004, PLR1711, PLR2044 ignored — noisy on connector dispatchers)RUF(incl.RUF100unused-noqa, so debt bleeds out as code is touched)What's removed
.pylintrc(root) — duplicate of the ingestion pylint config[tool.pylint.*]block iningestion/pyproject.toml(~140 lines)ingestion/plugins/{print_checker,import_checker}.py+ tests + README — replaced by built-inT20andTID251 banned-apirespectivelypylintdep fromingestion/setup.pyandopenmetadata-airflow-apis/pyproject.tomlmake lintMakefile target + the pylint invocation inmake py_format_checkingestion/noxfile.pyCwd-stable config
ruff is invoked from two locations:
ingestion/(make py_format_check)The
src,extend-exclude, andper-file-ignoresentries are listed twice — once relative toingestion/and once with theingestion/prefix — so first-party isort detection and exclusions match in both invocations.Documentation sweep
Replaced
make lintreferences inCLAUDE.md,AGENTS.md,DEVELOPER.md,.github/copilot-instructions.md, and 6 SKILL files with the apply+verify shapemake py_format && make py_format_check. Note:make py_formatis not a strict superset ofmake lint— it only applies auto-fixable violations;make py_format_checkcatches the rest.Basedpyright baseline regen
ruff format reflowed multi-line signatures in ~70 files, shifting type-error column positions. basedpyright's baseline matches by (file path, error code, column range), so column shifts caused 19 entries to mis-align. Net baseline diff is small — 154 lines in/out of the 13MB
baseline.json, purely positional.Verified locally
make py_format_check→ All checks passednox --no-venv -s static-checks→ 0 errors, 0 warnings, 0 notes (basedpyright)