feat(logging): unified interception, Spring-style config & PII redaction (v26.06.02)#27
Merged
Conversation
added 21 commits
June 5, 2026 09:56
Brainstormed design for the pyfly.logging overhaul: intercept & uniformly format every logger (framework + third-party via structlog ProcessorFormatter), Spring- parity pyfly.logging.* keys (patterns, file output, rotation) + external config-file escape hatch, and PII redaction (regex on by default, optional Presidio via pyfly[pii], log records always + opt-in stdout/stderr wrapper).
… implementation plan 12 TDD tasks: expanded LoggingProperties; PII pattern registry + Luhn; redactor engine (regex default + optional presidio + build_redactor); structlog processor + stdlib RedactionFilter; opt-in RedactingTextIO; logback->Python layout mapping; rotating file-handler builder; external dictConfig/fileConfig loader; both adapters wired for unified formatting + redaction + file output + external config + opt-in stream redaction; pyfly[pii] extra; docs + full regression.
…ator; simplify partial mask
…ction, external config
…ure + lock presidio Adding the pii extra means CI's 'uv sync --all-extras' now installs presidio, but without a downloaded spaCy model PresidioRedactor()/AnalyzerEngine() raises OSError (not ImportError). build_redactor now catches any presidio construction failure (missing OR misconfigured) and falls back to the regex engine, so the default engine=auto never crashes the app/CI. uv.lock updated with the resolved presidio + spacy dependency tree.
…PII redaction
- Updated docs/modules/logging.md with new sections:
(a) Unified interception — how all loggers (framework + third-party) render
through one formatter / redaction pass for both adapters.
(b) Configuration — full pyfly.logging.* key table (level, format, pattern,
file, rolling, config external file) with YAML example and logback token
reference.
(c) PII redaction — redaction.* key table, regex default + pyfly[pii]/Presidio
upgrade, mask styles, deny-fields/allow-fields, and opt-in streams wrapper.
- Added test_bare_date_token_no_braces to tests/logging/test_layout.py:
verifies compile_pattern("%d %p - %m") returns ("%(asctime)s %(levelname)s - %(message)s", None).
- Added test_apply_fileconfig_ini to tests/logging/test_config_loader.py:
verifies apply_external_config with a minimal fileConfig-format .ini file
returns True and configures logging (root level DEBUG).
Full suite: 3613 passed, 0 failures. No pre-existing test assertions required changes.
Lint: ruff check/format clean. mypy: no issues in 607 source files.
…), clarify total-size-cap is reserved, document external-config no-handlers behavior, note unused allow_fields on RedactionFilter Addresses final-review findings.
…residio via --all-extras) CI's 'uv sync --all-extras' installs presidio, so mypy sees the lazy presidio imports as untyped (not import-not-found). Add import-untyped to the type:ignore codes (unused-ignore covers whichever code doesn't fire in a given environment), and keep the ignore on the wrapped import's opening line.
… type-checks under --strict with presidio installed CI installs presidio (--all-extras); presidio is typed, so mypy --strict flagged no-untyped-call on AnonymizerEngine() and an arg-type mismatch (presidio's analyzer and anonymizer packages use different RecognizerResult classes). Typing the engines as Any makes the body version-independent; the no-untyped-call + unused-ignore codes cover both presidio-installed and not-installed environments. Verified mypy clean both ways.
presidio (+spacy) pulled by --all-extras made CI installs minutes-long. The logging code handles presidio-absent cleanly (build_redactor regex fallback; mypy unused-ignore codes), so CI runs without it. The pyfly[pii] extra remains available to users; presidio's own path is exercised locally / can get a dedicated opt-in job.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A logging-subsystem overhaul, built spec-first (brainstorm → design spec → TDD plan → subagent-driven execution with per-task spec+quality review):
logging) now renders through one formatter (structlogProcessorFormatter+foreign_pre_chain; matching stdlibFormatterfallback). Third-party logs previously bypassed the framework format.pyfly.logging.*keys:format(console|json|logfmt),pattern.console/file(logback tokens),file.name/path,rolling.*(size rotation), andconfig(external dictConfig/fileConfig escape hatch). Existinglevel.*/formatunchanged.pyfly.logging.redaction.*config; newpyfly[pii]extra auto-upgrades to Microsoft Presidio (engine: auto) with graceful regex fallback on any Presidio failure (never crashes the app). Opt-inredaction.streams.enabledwrapsstdout/stderr(rich CLI console bypassed).Design artifacts
docs/superpowers/specs/2026-06-05-logging-unification-pii-design.mddocs/superpowers/plans/2026-06-05-logging-unification-pii.mdNew modules
pyfly/logging/{layout,handlers,config_loader}.py,pyfly/logging/redaction/{patterns,engine,processor,stream}.py; both adapters rewired;LoggingPropertiesexpanded.Test plan / verification
pytest -q→ 3613 passed (incl. ~50 new logging tests; no pre-existing test broke from the unified format)ruff check src/ tests/+ruff format --checkclean ·mypy src/pyfly --strictcleanReleased as v26.06.02.