[flake8-builtins] Implement import, lambda, and module shadowing

[alex-700]

…-shadowing

Summary

Extend flake8-builtins to imports, lambda-arguments, and modules to be consistent with original checker flake8_builtins.

closes #12540

Details

• Implement builtin-import-shadowing (A004)
  • Stop tracking imports shadowing in builtin-variable-shadowing (A001) in preview mode.
• Implement builtin-lambda-argument-shadowing (A005)
• Implement builtin-module-shadowing (A006)
  • Add new option linter.flake8_builtins.builtins_allowed_modules

Test Plan

cargo test

[alex-700]

@charliermarsh

In the commit there was a change about changing import-checking from VariableShadowing to ImportShadowing. To be consistent here we need to change it here, by removing this block (and another one with import_from):

ruff/crates/ruff_linter/src/checkers/ast/analyze/statement.rs

Line 602 in 44d0200

flake8_builtins::rules::builtin_variable_shadowing(

But this is the breaking change in stable rule A001. What is the right way to do this?

[charliermarsh]

I think we should gate those blocks on "preview is disabled". So if you have preview enabled, ImportShadowing will catch them, and VariableShadowing will not. (But on stable, VariableShadowing can continue to catch them.)

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+179 -1 violations, +0 -0 fixes in 12 projects; 42 projects unchanged)

Snowflake-Labs/snowcli (+8 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ performance_history_analysis.py:24:18: A004 Import `print` is shadowing a Python builtin
+ src/snowflake/cli/api/console/abc.py:1:1: A005 Module `abc` is shadowing a Python builtin module
+ src/snowflake/cli/api/console/enum.py:1:1: A005 Module `enum` is shadowing a Python builtin module
+ src/snowflake/cli/api/errno.py:1:1: A005 Module `errno` is shadowing a Python builtin module
+ src/snowflake/cli/api/output/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ src/snowflake/cli/api/utils/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ src/snowflake/cli/plugins/cortex/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
... 2 additional changes omitted for rule A005

apache/airflow (+25 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ airflow/api_connexion/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ airflow/api_internal/internal_api_call.py:106:37: A004 Import `ConnectionError` is shadowing a Python builtin
+ airflow/compat/functools.py:1:1: A005 Module `functools` is shadowing a Python builtin module
+ airflow/hooks/subprocess.py:1:1: A005 Module `subprocess` is shadowing a Python builtin module
+ airflow/io/utils/stat.py:1:1: A005 Module `stat` is shadowing a Python builtin module
+ airflow/models/operator.py:1:1: A005 Module `operator` is shadowing a Python builtin module
+ airflow/operators/datetime.py:1:1: A005 Module `datetime` is shadowing a Python builtin module
... 14 additional changes omitted for rule A005
+ airflow/providers/cncf/kubernetes/utils/pod_manager.py:41:43: A004 Import `TimeoutError` is shadowing a Python builtin
+ clients/python/test_python_client.py:36:22: A004 Import `print` is shadowing a Python builtin
+ dev/check_files.py:24:18: A004 Import `print` is shadowing a Python builtin
... 15 additional changes omitted for project

apache/superset (+13 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ superset/advanced_data_type/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ superset/dashboards/permalink/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ superset/distributed_lock/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ superset/explore/permalink/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ superset/key_value/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ superset/migrations/versions/2018-07-22_11-59_bebcf3fed1fe_convert_dashboard_v1_positions.py:228:27: A006 Lambda argument `sum` is shadowing a Python builtin
+ superset/reports/notifications/email.py:1:1: A005 Module `email` is shadowing a Python builtin module
... 7 additional changes omitted for rule A005
... 6 additional changes omitted for project

bokeh/bokeh (+22 -1 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ src/bokeh/application/handlers/code.py:1:1: A005 Module `code` is shadowing a Python builtin module
+ src/bokeh/command/subcommands/json.py:1:1: A005 Module `json` is shadowing a Python builtin module
+ src/bokeh/core/property/datetime.py:1:1: A005 Module `datetime` is shadowing a Python builtin module
+ src/bokeh/core/property/enum.py:1:1: A005 Module `enum` is shadowing a Python builtin module
+ src/bokeh/core/property/json.py:1:1: A005 Module `json` is shadowing a Python builtin module
+ src/bokeh/core/property/string.py:1:1: A005 Module `string` is shadowing a Python builtin module
... 10 additional changes omitted for rule A005
+ src/bokeh/core/validation/check.py:36:20: A004 Import `Warning` is shadowing a Python builtin
+ src/bokeh/core/validation/decorators.py:34:34: A004 Import `Warning` is shadowing a Python builtin
+ src/bokeh/core/validation/warnings.py:37:20: A004 Import `Warning` is shadowing a Python builtin
- src/bokeh/models/callbacks.py:25:42: A001 Variable `any` is shadowing a Python builtin
... 13 additional changes omitted for project

latchbio/latch (+9 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ latch/functions/secrets.py:1:1: A005 Module `secrets` is shadowing a Python builtin module
+ latch/registry/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ latch/registry/upstream_types/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ latch/types/glob.py:1:1: A005 Module `glob` is shadowing a Python builtin module
+ latch/types/json.py:1:1: A005 Module `json` is shadowing a Python builtin module
+ latch_cli/exceptions/traceback.py:1:1: A005 Module `traceback` is shadowing a Python builtin module
... 3 additional changes omitted for rule A005
+ latch_cli/main.py:440:49: A004 Import `exec` is shadowing a Python builtin

milvus-io/pymilvus (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ pymilvus/client/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ pymilvus/orm/types.py:1:1: A005 Module `types` is shadowing a Python builtin module

rotki/rotki (+24 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ rotkehlchen/accounting/debugimporter/json.py:1:1: A005 Module `json` is shadowing a Python builtin module
+ rotkehlchen/accounting/export/csv.py:1:1: A005 Module `csv` is shadowing a Python builtin module
+ rotkehlchen/accounting/structures/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/accounting/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/api/v1/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/assets/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/chain/arbitrum_one/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/chain/ethereum/interfaces/ammswap/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/chain/ethereum/modules/balancer/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ rotkehlchen/chain/ethereum/modules/liquity/statistics.py:1:1: A005 Module `statistics` is shadowing a Python builtin module
... 14 additional changes omitted for project

yandex/ch-backup (+6 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ ch_backup/compression/gzip.py:1:1: A005 Module `gzip` is shadowing a Python builtin module
+ ch_backup/logging.py:1:1: A005 Module `logging` is shadowing a Python builtin module
+ ch_backup/profile.py:1:1: A005 Module `profile` is shadowing a Python builtin module
+ ch_backup/storage/async_pipeline/stages/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ tests/integration/modules/datetime.py:1:1: A005 Module `datetime` is shadowing a Python builtin module
+ tests/integration/modules/typing.py:1:1: A005 Module `typing` is shadowing a Python builtin module

zulip/zulip (+10 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ zerver/actions/typing.py:1:1: A005 Module `typing` is shadowing a Python builtin module
+ zerver/data_import/slack.py:707:16: A006 Lambda argument `id` is shadowing a Python builtin
+ zerver/lib/profile.py:1:1: A005 Module `profile` is shadowing a Python builtin module
+ zerver/lib/queue.py:1:1: A005 Module `queue` is shadowing a Python builtin module
+ zerver/lib/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ zerver/lib/url_preview/types.py:1:1: A005 Module `types` is shadowing a Python builtin module
+ zerver/tests/test_link_embed.py:11:33: A004 Import `ConnectionError` is shadowing a Python builtin
+ zerver/tests/test_push_notifications.py:25:33: A004 Import `ConnectionError` is shadowing a Python builtin
+ zerver/tornado/django_api.py:11:31: A004 Import `ConnectionError` is shadowing a Python builtin
+ zerver/views/typing.py:1:1: A005 Module `typing` is shadowing a Python builtin module
... 1 additional changes omitted for rule A005

zanieb/huggingface-notebooks (+25 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select A,E703,F704,B015,B018,D100

+ diffusers/SDXL_DreamBooth_LoRA_.ipynb:cell 40:1:29: A004 Import `display` is shadowing a Python builtin
+ diffusers/geodiff_molecule_conformation.ipynb:cell 63:4:34: A004 Import `display` is shadowing a Python builtin
+ examples/accelerate_examples/simple_nlp_example.ipynb:cell 16:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/audio_classification.ipynb:cell 32:2:36: A004 Import `display` is shadowing a Python builtin
+ examples/language_modeling-tf.ipynb:cell 23:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/language_modeling.ipynb:cell 23:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/language_modeling_from_scratch-tf.ipynb:cell 23:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/language_modeling_from_scratch.ipynb:cell 23:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/multi_lingual_speech_recognition.ipynb:cell 25:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/multiple_choice-tf.ipynb:cell 24:4:29: A004 Import `display` is shadowing a Python builtin
... 15 additional changes omitted for project

openai/openai-cookbook (+16 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select A,E703,F704,B015,B018,D100

+ examples/Creating_slides_with_Assistants_API_and_DALL-E3.ipynb:cell 3:1:29: A004 Import `display` is shadowing a Python builtin
+ examples/Developing_hallucination_guardrails.ipynb:cell 2:2:29: A004 Import `display` is shadowing a Python builtin
+ examples/Fine_tuning_for_function_calling.ipynb:cell 10:4:29: A004 Import `display` is shadowing a Python builtin
+ examples/GPT_with_vision_for_video_understanding.ipynb:cell 2:1:29: A004 Import `display` is shadowing a Python builtin
+ examples/How_to_call_functions_for_knowledge_retrieval.ipynb:cell 3:10:29: A004 Import `display` is shadowing a Python builtin
+ examples/How_to_combine_GPT4o_with_RAG_Outfit_Assistant.ipynb:cell 5:10:36: A004 Import `display` is shadowing a Python builtin
+ examples/Named_Entity_Recognition_to_enrich_text.ipynb:cell 9:9:29: A004 Import `display` is shadowing a Python builtin
+ examples/Parse_PDF_docs_for_RAG.ipynb:cell 5:20:18: A004 Import `print` is shadowing a Python builtin
+ examples/Question_answering_using_a_search_API.ipynb:cell 3:3:21: A004 Import `display` is shadowing a Python builtin
+ examples/Reproducible_outputs_with_the_seed_parameter.ipynb:cell 6:3:29: A004 Import `display` is shadowing a Python builtin
... 6 additional changes omitted for project

... Truncated remaining completed project reports due to GitHub comment length restrictions

Changes by rule (4 rules affected)

code	total	+ violation	- violation	+ fix	- fix
A005	104	104	0	0	0
A004	73	73	0	0	0
A006	2	2	0	0	0
A001	1	0	1	0	0

[alex-700]

Thanks @charliermarsh for the idea!

I think we should gate those blocks on "preview is disabled"

Have done it this way. One tiny problem of it: if I understand the process correctly, we need to remove this check from A001 completely, once A004 will be stable. I put the TODO about this here, but if you know more robust way to remember about this during stabilization (e.g. assert that A004 still in preview, when this preview.is_disabled()is activated), feel free to change it.

[charliermarsh]

Thanks!

[charliermarsh]

@alex-700 -- I added a note to codes.rs so that we see it when we change from preview to stable.