Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Replace lint and formatting tools with ruff #1074

Merged
merged 38 commits into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
0b45c97
ci: Add initial ruff config to match existing
mitches-got-glitches Mar 5, 2024
0558f5d
ci: Replace pre-commit hooks with ruff
mitches-got-glitches Mar 5, 2024
89792e6
ci: Make ruff generate Py3.7 compatible code
mitches-got-glitches Mar 5, 2024
585eb93
ci: Replace reorder-python-imports with ruff
mitches-got-glitches Mar 5, 2024
84b0f62
ci: Add case insensitive sorting to ruff isort
mitches-got-glitches Mar 5, 2024
0db5f97
ci: Make sure the import of annotations is preserved
mitches-got-glitches Mar 5, 2024
88138fc
ci: Add isort rules to lint and put order-by-type=false
mitches-got-glitches Mar 5, 2024
7300020
ci: Run ruff over dynaconf/
mitches-got-glitches Mar 5, 2024
0ffb351
ci: Run ruff over tests/
mitches-got-glitches Mar 5, 2024
2211582
ci: Run ruff format on dynaconf/ again
mitches-got-glitches Mar 5, 2024
f6f4a8c
ci: Make test_functional ruff.toml extend main one
mitches-got-glitches Mar 5, 2024
663e5b0
ci: Run ruff over tests_functional/
mitches-got-glitches Mar 5, 2024
8b6ae7c
ci: Remove debug hook because its covered by ruff T100
mitches-got-glitches Mar 5, 2024
33d62e0
ci: Remove encoding declaration hook
mitches-got-glitches Mar 5, 2024
d0efade
ci: Remove more hooks covered by ruff
mitches-got-glitches Mar 5, 2024
b3b7470
Fix comment
mitches-got-glitches Mar 5, 2024
8786bd4
ci: Run pre-commit over all files
mitches-got-glitches Mar 5, 2024
0b01959
ci: Remove flake8 packages from requirements_dev
mitches-got-glitches Mar 5, 2024
7a4011e
ci: Add support for pep8-naming in ruff
mitches-got-glitches Mar 5, 2024
6c0121e
ci: Ruff fixing import groups
mitches-got-glitches Mar 5, 2024
1a118c6
ci: Remove .pep8speaks - no need for it with ruff
mitches-got-glitches Mar 5, 2024
039793b
ci: Remove pep8 section from Makefile
mitches-got-glitches Mar 5, 2024
b38739e
Add .python-version from pyenv to .gitingore
mitches-got-glitches Mar 5, 2024
04a052e
ci: Add dynaconf as known first party package
mitches-got-glitches Mar 5, 2024
15f9df0
Merge branch 'master' into ci/add-ruff
mitches-got-glitches Mar 6, 2024
f56f8a7
ci: Final fix from ruff
mitches-got-glitches Mar 6, 2024
f1aaece
chore: Remove flake8 test-requirements from setup.py
mitches-got-glitches Mar 6, 2024
6fb0725
chore: Remove flake8 config in setup.cfg
mitches-got-glitches Mar 6, 2024
a26a5fb
ci: Bump ruff lint to python v3.8
mitches-got-glitches Mar 8, 2024
fd750f3
Add missing import of annotations
mitches-got-glitches Mar 8, 2024
00a3c60
Replace lower level ruff.toml with per-file-ignores
mitches-got-glitches Mar 8, 2024
6220d97
ci: Move E402 ignore to tests_functional/ only and apply fixes
mitches-got-glitches Mar 8, 2024
50d430f
Move F401 to per-file-ignores and do fixes
mitches-got-glitches Mar 8, 2024
1a0a2c2
Add noqa on unused imports in TYPE_CHECKING sections
mitches-got-glitches Mar 8, 2024
09f87b1
Remove check for E401
mitches-got-glitches Mar 8, 2024
51f18cd
Add per-file-ignores to __init__.py for imports
mitches-got-glitches Mar 8, 2024
0e73881
Removed F841 - can be added to per-file-ignores as exception
mitches-got-glitches Mar 8, 2024
33013a5
Merge branch 'master' into ci/add-ruff
pedro-psb Mar 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,6 @@ dynaconf/vendor/source
.DS_Store

.helix

# pyenv
.python-version
18 changes: 0 additions & 18 deletions .pep8speaks.yml

This file was deleted.

33 changes: 10 additions & 23 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,20 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
args: [--allow-multiple-documents]
- id: check-merge-conflict
- id: debug-statements
- id: fix-encoding-pragma
args: [--remove]
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.1.0
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.3.0
hooks:
- id: reorder-python-imports
args: [--py37-plus, --add-import, 'from __future__ import annotations']
- repo: https://github.com/asottile/pyupgrade
rev: v2.34.0
hooks:
- id: pyupgrade
args: [--py38-plus]
- repo: https://github.com/ambv/black
rev: 22.3.0
hooks:
- id: black
args: [--line-length=79]
- repo: https://github.com/pycqa/flake8
rev: 3.9.2
hooks:
- id: flake8
# Sort imports.
- id: ruff
args: [--select, I, --fix]
# Run the linter.
- id: ruff
# Run the formatter.
- id: ruff-format
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.961
hooks:
Expand Down
12 changes: 2 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
SHELL := /bin/bash
.PHONY: all citest clean mypy dist docs install pep8 publish run-pre-commit run-tox setup-pre-commit test test_functional test_only test_redis test_vault help coverage-report
.PHONY: all citest clean mypy dist docs install publish run-pre-commit run-tox setup-pre-commit test test_functional test_only test_redis test_vault help coverage-report

help:
@$(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | egrep -v -e '^[^[:alnum:]]' -e '^$@$$'
Expand Down Expand Up @@ -39,7 +39,7 @@ coverage-report:
mypy:
mypy dynaconf/ --exclude '^dynaconf/vendor*'

test: pep8 mypy test_only
test: mypy test_only

citest:
py.test -v --cov-config .coveragerc --cov=dynaconf -l tests/ --junitxml=junit/test-results.xml
Expand All @@ -63,14 +63,6 @@ run-pre-commit:
rm -rf build/
pre-commit run --all-files

pep8:
# Flake8 ignores
# F841 (local variable assigned but never used, useful for debugging on exception)
# W504 (line break after binary operator, I prefer to put `and|or` at the end)
# F403 (star import `from foo import *` often used in __init__ files)
# flake8 dynaconf --ignore=F403,W504,W503,F841,E401,F401,E402 --exclude=dynaconf/vendor
flake8 dynaconf --exclude=dynaconf/vendor*

mitches-got-glitches marked this conversation as resolved.
Show resolved Hide resolved
dist: clean
@make minify_vendor
@python setup.py sdist bdist_wheel
Expand Down
16 changes: 8 additions & 8 deletions dynaconf/__init__.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
from __future__ import annotations

from dynaconf.base import LazySettings # noqa
from dynaconf.base import LazySettings
from dynaconf.constants import DEFAULT_SETTINGS_FILES
from dynaconf.contrib import DjangoDynaconf # noqa
from dynaconf.contrib import FlaskDynaconf # noqa
from dynaconf.contrib import DjangoDynaconf
from dynaconf.contrib import FlaskDynaconf
from dynaconf.utils.inspect import get_history
from dynaconf.utils.inspect import inspect_settings
from dynaconf.utils.parse_conf import add_converter # noqa
from dynaconf.utils.parse_conf import DynaconfFormatError # noqa
from dynaconf.utils.parse_conf import DynaconfParseError # noqa
from dynaconf.validator import ValidationError # noqa
from dynaconf.validator import Validator # noqa
from dynaconf.utils.parse_conf import add_converter
from dynaconf.utils.parse_conf import DynaconfFormatError
from dynaconf.utils.parse_conf import DynaconfParseError
from dynaconf.validator import ValidationError
from dynaconf.validator import Validator

settings = LazySettings(
# This global `settings` is deprecated from v3.0.0+
Expand Down
1 change: 0 additions & 1 deletion dynaconf/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -1231,7 +1231,6 @@ def load_file(
if files:
already_loaded = set()
for _filename in files:

# load_file() will handle validation later
with suppress(ValidationError):
if py_loader.try_to_load_from_py_module_name(
Expand Down
24 changes: 8 additions & 16 deletions dynaconf/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@
from dynaconf.utils import upperfy
from dynaconf.utils.files import read_file
from dynaconf.utils.functional import empty
from dynaconf.utils.inspect import builtin_dumpers
from dynaconf.utils.inspect import EnvNotFoundError
from dynaconf.utils.inspect import inspect_settings
from dynaconf.utils.inspect import KeyNotFoundError
from dynaconf.utils.inspect import OutputFormatError
from dynaconf.utils.parse_conf import parse_conf_data
from dynaconf.utils.parse_conf import unparse_conf_data
from dynaconf.validator import ValidationError
Expand All @@ -28,9 +33,8 @@
from dynaconf.vendor import toml
from dynaconf.vendor import tomllib


if TYPE_CHECKING: # pragma: no cover
from dynaconf.base import Settings
from dynaconf.base import Settings # noqa: F401

os.environ["PYTHONIOENCODING"] = "utf-8"

Expand Down Expand Up @@ -78,7 +82,7 @@ def set_settings(ctx, instance=None):
try:
# Django extension v2
from django.conf import settings # noqa
import dynaconf
import dynaconf # noqa: F401
import django

# see https://docs.djangoproject.com/en/4.2/ref/applications/
Expand All @@ -97,7 +101,6 @@ def set_settings(ctx, instance=None):
)

if settings is None:

if instance is None and "--help" not in click.get_os_args():
if ctx.invoked_subcommand and ctx.invoked_subcommand not in [
"init",
Expand Down Expand Up @@ -394,9 +397,7 @@ def init(ctx, fileformat, path, env, _vars, _secrets, wg, y, django):

if settings_path:
loader.write(settings_path, settings_data, merge=True)
click.echo(
f"🎛️ {settings_path.name} created to hold your settings.\n"
)
click.echo(f"🎛️ {settings_path.name} created to hold your settings.\n")
mitches-got-glitches marked this conversation as resolved.
Show resolved Hide resolved

if secrets_path:
loader.write(secrets_path, secrets_data, merge=True)
Expand Down Expand Up @@ -661,7 +662,6 @@ def write(to, _vars, _secrets, path, env, y):
loader = importlib.import_module(f"dynaconf.loaders.{to}_loader")

if to in EXTS:

# Lets write to a file
path = Path(path)

Expand Down Expand Up @@ -807,14 +807,6 @@ def validate(path): # pragma: no cover
sys.exit(1)


from dynaconf.utils.inspect import (
KeyNotFoundError,
builtin_dumpers,
inspect_settings,
EnvNotFoundError,
OutputFormatError,
)

INSPECT_FORMATS = list(builtin_dumpers.keys())


Expand Down
6 changes: 3 additions & 3 deletions dynaconf/contrib/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from __future__ import annotations

from dynaconf.contrib.django_dynaconf_v2 import DjangoDynaconf # noqa
from dynaconf.contrib.flask_dynaconf import DynaconfConfig # noqa
from dynaconf.contrib.flask_dynaconf import FlaskDynaconf # noqa
from dynaconf.contrib.django_dynaconf_v2 import DjangoDynaconf
from dynaconf.contrib.flask_dynaconf import DynaconfConfig
from dynaconf.contrib.flask_dynaconf import FlaskDynaconf
2 changes: 1 addition & 1 deletion dynaconf/contrib/django_dynaconf_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
DJANGO_ALLOWED_HOSTS='["localhost"]' \
python manage.py runserver
"""

from __future__ import annotations

import inspect
Expand Down Expand Up @@ -114,7 +115,6 @@ class UserSettingsHolder(dynaconf.LazySettings):

# 5) Patch django.conf.settings
class Wrapper:

# lazy_settings = conf.settings.lazy_settings

def __getattribute__(self, name):
Expand Down
3 changes: 2 additions & 1 deletion dynaconf/contrib/flask_dynaconf.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@
Config = object


import dynaconf
from importlib.metadata import EntryPoint

import dynaconf


class FlaskDynaconf:
"""The arguments are.
Expand Down
6 changes: 1 addition & 5 deletions dynaconf/hooking.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,11 @@
from functools import wraps
from typing import Any
from typing import Callable
from typing import Dict
from typing import List

from dynaconf.base import RESERVED_ATTRS
from dynaconf.base import Settings
from dynaconf.loaders.base import SourceMetadata


__all__ = [
"hookable",
"EMPTY_VALUE",
Expand All @@ -25,8 +22,7 @@
]


class Empty:
...
class Empty: ...
mitches-got-glitches marked this conversation as resolved.
Show resolved Hide resolved


EMPTY_VALUE = Empty()
Expand Down
1 change: 0 additions & 1 deletion dynaconf/loaders/base.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from __future__ import annotations

import io
import warnings
from typing import NamedTuple

Expand Down
1 change: 0 additions & 1 deletion dynaconf/loaders/ini_loader.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from __future__ import annotations

import io
from pathlib import Path

from dynaconf import default_settings
Expand Down
1 change: 0 additions & 1 deletion dynaconf/loaders/json_loader.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from __future__ import annotations

import io
import json
from pathlib import Path

Expand Down
1 change: 0 additions & 1 deletion dynaconf/loaders/py_loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import errno
import importlib
import inspect
import io
import types
from contextlib import suppress
from pathlib import Path
Expand Down
3 changes: 2 additions & 1 deletion dynaconf/loaders/vault_loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@

try:
from hvac import Client
from hvac.exceptions import InvalidPath, Forbidden
from hvac.exceptions import Forbidden
from hvac.exceptions import InvalidPath
except ImportError:
raise ImportError(
"vault package is not installed in your environment. "
Expand Down
10 changes: 4 additions & 6 deletions dynaconf/loaders/yaml_loader.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
from __future__ import annotations

import sys
from pathlib import Path
from typing import TextIO
from warnings import warn

from dynaconf import default_settings
Expand All @@ -13,10 +11,10 @@
from dynaconf.vendor.ruamel import yaml

# Add support for Dynaconf Lazy values to YAML dumper
yaml.SafeDumper.yaml_representers[
None
] = lambda self, data: yaml.representer.SafeRepresenter.represent_str(
self, try_to_encode(data)
yaml.SafeDumper.yaml_representers[None] = (
lambda self, data: yaml.representer.SafeRepresenter.represent_str(
self, try_to_encode(data)
)
)


Expand Down
7 changes: 2 additions & 5 deletions dynaconf/utils/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@
from typing import TYPE_CHECKING
from typing import TypeVar


if TYPE_CHECKING: # pragma: no cover
from dynaconf.base import LazySettings
from dynaconf.base import Settings
from dynaconf.utils.boxing import DynaBox
from dynaconf.base import LazySettings, Settings


BANNER = """
Expand Down Expand Up @@ -47,7 +47,6 @@ def object_merge(
return new

if isinstance(old, list) and isinstance(new, list):

# 726: allow local_merge to override global merge on lists
if "dynaconf_merge_unique" in new:
new.remove("dynaconf_merge_unique")
Expand Down Expand Up @@ -83,7 +82,6 @@ def safe_items(data):
should_merge = new.pop("dynaconf_merge", True)
if should_merge:
for old_key, value in safe_items(old):

# This is for when the dict exists internally
# but the new value on the end of full path is the same
if (
Expand Down Expand Up @@ -131,7 +129,6 @@ def handle_metavalues(
"""Cleanup of MetaValues on new dict"""

for key in list(new.keys()):

# MetaValue instances
if getattr(new[key], "_dynaconf_reset", False): # pragma: no cover
# a Reset on `new` triggers reasign of existing data
Expand Down
6 changes: 3 additions & 3 deletions dynaconf/utils/boxing.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ def evaluate(dynabox, item, *args, **kwargs):
settings = dynabox._box_config["box_settings"]

if getattr(value, "_dynaconf_lazy_format", None):
dynabox._box_config[
f"raw_{item.lower()}"
] = f"@{value.formatter.token} {value.value}"
dynabox._box_config[f"raw_{item.lower()}"] = (
f"@{value.formatter.token} {value.value}"
)

return recursively_evaluate_lazy_format(value, settings)

Expand Down