Skip to content

Releases: kidoz/structguru

v1.5.0

Choose a tag to compare

@github-actions github-actions released this 13 Oct 19:19

structguru v1.5.0

A resilience and security release. A retired sink's destructor could deadlock the dispatcher during removal or reconfiguration; a stalled sink could hold flush(), shutdown(), and reconfiguration open forever; and the warnings those failures emit could print the exception's text — including a record's secrets — into diagnostics that bypass redaction by design. The minor version carries two behavior changes to read before you upgrade: adding the same logging.Handler twice now raises ValueError, and sink-failure and fork-restart diagnostics no longer include the exception text. Free-threaded CPython 3.15t joins the 3.14t wheels. No breaking API changes; upgrade in place from any v1.x.

Behavior changes

  • Adding the same logging.Handler twice raises ValueError until it has been removed. A second add() used to register the handler again and deliver every rendered record to it twice; the first registration's delivery and ownership are now preserved. A handler removed through any Logger can be added again, and a failed registration rolls its root-logger reservation back instead of leaving the handler unusable.

  • Sink-failure and fork-restart diagnostics never stringify the exception, reporting only its type: structguru callable sink raised 1 time(s), latest ConnectionError. Diagnostic warnings bypass record redaction, so an exception whose text embedded the record — or a credential — it was writing could put the unredacted value into a warning, and a __str__ with side effects ran on the delivery thread and on the fork path. The error is still counted in callable_errors and stream_errors, and the fork diagnostic still names the type.

Features

  • CPython 3.15 support. The abi3 wheel already covers 3.15 unchanged, and free-threaded 3.15t wheels join the 3.14t ones for Linux x86_64, macOS arm64, and Windows x64. The 3.15 source-gate leg returns once PyYAML ships cp315 wheels — it has none for 3.15 and its sdist build crashes the CI runners today — while the free-threaded 3.15t gate already runs on 3.15.

Fixes

Lifecycle

  • Removing or replacing a callable sink releases retired Python callbacks after unlocking the dispatcher. remove(), configure(), sync(), and disable() dropped the dispatcher's fork snapshot — and with it the last reference to a retired callback — while holding the registry lock. A callback whose __del__ re-enters the dispatcher, as a sink that calls metrics() or add() from its destructor does, deadlocked the process: all eight transition/cache combinations hung before the fix and complete cleanly after it, pinned by a subprocess regression test.

  • flush(), shutdown(), and reconfiguration no longer block forever when the native writer's sink stops making progress — a full stdout pipe whose reader has stopped, for example. A writer stuck for 10 seconds is abandoned with a warning naming the number of undrained records, matching the stall limit callable sinks already had.

  • A panic inside a native sink no longer kills the writer thread silently, which used to leave flush() waiting forever and producers enqueueing into a queue nobody drained. The interruption is counted in sink_errors, the writer closes, and blocked flushers and producers wake up.

Docs

  • The documentation is restructured into tutorials, how-to guides, reference, and explanation (Diátaxis), and the site is built in CI and through make docs.
  • The Sentry how-to no longer reads as a redaction guarantee: the raw exception passed to sentry_sdk.capture_exception lets the SDK attach unredacted frame locals unless it is initialised with include_local_variables=False, and the docs now say so.

Test coverage

  • 19 new Python test cases and 4 new Rust tests since v1.4.0 — 977 Python tests and 99 Rust tests in total, with line coverage at 93.8%. The dispatcher fix is pinned by a subprocess test that hangs on the previous build; the diagnostics change by a test whose exception's __str__ records being called and never is.

v1.4.0

Choose a tag to compare

@kidoz kidoz released this 25 Sep 08:12

structguru v1.4.0

A correctness, security, and resilience release, from a deep review of v1.3.1 that turned up about fifty defects — each now pinned by a regression test that fails against the previous build. A forked child could hang forever logging to stdout; secrets reached log lines through exception locals, message placeholders, mapping keys, and Celery headers; callable sinks could be called from two threads at once, lose records during a reconfiguration, or be shut off while they were still delivering. Free-threaded CPython 3.14t gets its own wheels. The minor version reflects a handful of behavior changes that align structguru with loguru or close a silent failure — read them before you upgrade.

Behavior changes

  • logger.remove(handler_id) raises for an id that names no sink. An unknown or already-removed id raises ValueError, and a non-integer TypeError, with loguru's messages. Handler ids are now process-wide, so any Logger removes a sink added through another; a sink added through Logger(name=...) previously could not be removed through logger at all, and a mistyped id was ignored silently.

  • The message argument is positional-only, as in loguru, and so is self on bind() and contextualize(). logger.info("login", message=..., self=...) now logs message and self as fields instead of raising TypeError; passing the message itself by keyword, logger.info(message="login"), now raises TypeError.

  • logger.add(handler) leaves your handler alone. It used to set the handler's level to 0 and its formatter to %(message)s for good, changing its output for every other logger it was attached to, even after remove(). The handler now keeps both, its own formatter wraps the rendered line, and only add(level=...) gates what the sink receives. add(level=...) accepts integers and raises ValueError for an unknown level, like configure(), instead of warning and falling back to INFO.

  • A record rejected by a closed writer reaches no other destination. During reconfiguration or shutdown the synchronous stream_sink still received a record the native writer had already rejected. The stream, callable sinks, and Sentry now all follow the writer's outcome, and the rejection is counted in lifecycle_metrics() as before.

  • Console output escapes more and drops less. Quoted values escape " and \, keys containing spaces, =, quotes, or backslashes are quoted, and U+2028, U+2029, and bidirectional controls are escaped everywhere, so a request-controlled key or value can no longer imitate another field or start a new line. Floats use the JSON renderer's text, a formatted exception follows as an indented block like the stack, and user service and stack fields are shown instead of dropped. Ordinary string fields render exactly as before.

    before: 2026-09-25T08:00:00.000000Z [INFO    ] login  user="bob"  admin="true"  ratio=1
    after:  2026-09-25T08:00:00.000000Z [INFO    ] login  user="bob\"  admin=\"true"  ratio=1.0
    
  • Stdout is written straight to file descriptor 1 on Unix, one write per record with no user-space buffer, and keeps following fd 1 after a later dup2(). A closed stdout now counts as sink_errors instead of silently counting every record as written.

  • Integration timing. Flask and Django log a streamed response's summary when its body finishes, so with Werkzeug's test client the Flask summary appears after get_data() or close(). The ASGI duration_ms stops when the final response body is sent, not after Starlette background tasks.

Features

  • Free-threaded CPython 3.14t wheels for Linux x86_64, macOS arm64, and Windows x64. pip never selects an abi3 wheel on a free-threaded interpreter, so pip install structguru on 3.14t compiled the sdist and needed a Rust toolchain.

  • writer_metrics()["callable_errors"] counts exceptions raised by callable sinks, which used to be discarded without a trace; the first failure and every 1000th also emit a UserWarning from the delivery thread.

  • Numeric levels in the environment. LOG_LEVEL=10 or STRUCTGURU_LEVEL=25 is a threshold instead of an import-time error.

Fixes

Redaction

  • Exception locals no longer leak secrets through repr(). With exception_include_locals=True a local was redacted only when its own name was sensitive, so creds={"password": ...}, a dataclass with a password field, headers={"Authorization": ...}, or a bound Logger was written as its plain repr(). Values under a sensitive key are now redacted at any depth inside mappings, sequences, sets, named tuples, dataclasses, SimpleNamespace, and argparse.Namespace, without calling those containers' own __repr__. sensitive_patterns also see each local's whole rendering before it is cut to exception_max_local_repr; a secret straddling the cut used to escape its pattern half-disclosed.

    # before: "creds": "{'password': 'hunter2'}"
    # after:  "creds": "{'password': [REDACTED]}"
  • Message placeholders are redacted. logger.info("login token={token}", token=...) interpolated the token into the message — and into Sentry — while logger.info("login", token=...) was redacted. A placeholder that names a sensitive key, including attribute and index paths such as {creds[password]}, is now interpolated as [REDACTED], and a mapping, sequence, dataclass, or namespace a placeholder interpolates renders with its sensitive values redacted. The check adds roughly 0.1 µs to a formatted call; calls without arguments skip it.

  • sensitive_patterns cover mapping keys and integers. Patterns ran over string values only, so an e-mail address used as a key, or a card number passed as an int, was written verbatim. They now match every key at every depth and each integer's decimal digits; a matching integer is emitted as the redacted string, and keys that become identical are written once.

  • ASGI credential headers are redacted and bounded. extract_headers bound header names verbatim, so x-api-key, x-auth-token, proxy-authorization, and set-cookie were logged in clear while authorization and cookie were redacted, and a header of any size was copied into every log line of the request. Credential headers are now redacted, values must be printable UTF-8, and they are truncated at 256 characters.

  • Celery headers no longer carry unredacted context. Every bound value — a password included — was copied into the message headers in clear. Header values are now redacted with the runtime's keys, converted to transport-safe types, and bounded (see Integrations for the publish failure this also fixes).

Reliability

  • A forked child no longer hangs with target="stdout". The stdout sink wrote through Rust's std::io::stdout(), whose process-wide lock the parent's writer thread could be holding at fork(); the child's respawned writer then blocked on it forever, and with it flush(), interpreter exit, and — once the queue filled — every logging call. A child forked while the parent was blocked writing to a slowly drained pipe hung 5 times out of 5, and 2 times out of 60 with stdout redirected to a file; both now complete every time, on macOS and on glibc.

  • Callable sinks are called by one thread, in log order. logger.add(), configure(), and update() started a new delivery generation before the old one drained, so a backlogged sink was called from two threads at once and saw records out of order — 53 inversions in 260 records in one run. A new generation now accepts records at once but delivers only after its predecessors finish, and logger.add() no longer replaces the generation or waits for its backlog, which took 3.5 s for 300 queued 10 ms deliveries and never returned behind a stuck sink.

  • Records accepted just before a reconfiguration reach callable sinks and Sentry. Delivery was decided by whether the runtime still used the same writer rather than by the enqueue result. With four threads logging and another looping update(), 15 to 20 of 80,000 records reached the file but never the logger.add() sink, and nothing counted them.

  • A slow sink that keeps delivering is no longer abandoned. The stall check counted only released producers, which settle a whole 64-record batch at a time, so with the default 10-second limit any sink slower than about 156 ms per record was declared stuck and disabled, with 1,045 records counted as lost. Completed deliveries now count as progress. The same limit now also bounds configure(), update(), and logger.remove(), which could wait forever behind a stuck sink, and concurrent flushes report an abandoned generation once.

  • The rotating file sink resolves a relative file_path once, so a later os.chdir() no longer splits the log; retries a lock-file wait interrupted by a signal instead of dropping the record; reopens a log that logrotate renamed or deleted even with rotation disabled; and no longer writes records it counted as failed from a retained buffer once the disk recovers.

  • A forked child whose writer cannot restart disables logging with one line on stderr, instead of keeping the dead writer, dropping records uncounted, and hanging in flush().

  • Logging never raises for its own diagnostics or arguments. Under -W error the queue-full, sink-error, abandonment, and formatting warnings raised out of the logging call; they now go to sys.unraisablehook. A message whose str() raises, or an argument whose __format__ raises, falls back to a placeholder or the raw template.

Correctness

  • Leaving contextualize() in a different context — a generator iterated from a thread pool, an async generator finalized by the garbage collector — raised ValueError. It now restores the entry context when the current context still holds the block's own mapping, and leaves any other context, such as another request...
Read more

v1.3.1

Choose a tag to compare

@kidoz kidoz released this 17 Sep 17:53

structguru v1.3.1

A performance release for structured exceptions. One defect is fixed: with structured_exceptions=True, every logger.exception() call read and dedented source for the whole traceback before discarding all but the last exception_max_frames frames, so a deep traceback such as a RecursionError cost 1.7 ms to report twenty frames; and frames captured with exception_include_locals=True reported "line": null where every other frame carried its source line. No breaking changes; upgrade in place from any v1.x.

Fixes

  • Structured exceptions extract source only for the frames they keep. The frame builder ran traceback.extract_tb() over the entire traceback — linecache reads and dedents the source line for each frame it visits — and then sliced the result to the last exception_max_frames. The cost therefore scaled with the depth of the traceback rather than with the number of frames reported: a RecursionError a thousand frames deep paid for a thousand source lookups to produce twenty. The traceback is now walked to the first kept frame before extraction runs, so the same call costs 64 µs instead of 1.7 ms, and a shallow traceback is unaffected.

    Frames captured with exception_include_locals=True now carry the line field like every other frame. That path walked the raw traceback itself to reach f_locals and never looked up source, leaving line as null; locals now come from the same selected frames that source is extracted for, so the two modes produce identical frames apart from locals.

    configure(structured_exceptions=True, exception_include_locals=True)
    # before: {"filename": "app.py", "lineno": 42, "name": "handle", "line": null, "locals": {"n": "0"}}
    # after:  {"filename": "app.py", "lineno": 42, "name": "handle", "line": "raise ValueError(\"boom\")", "locals": {"n": "0"}}

    Filling in line makes the locals path pay for source lookup too — about 50 µs instead of 10 µs for twenty frames — which is the price of a consistent schema. Formatted tracebacks, exception_max_frames, and the locals redaction and repr limits are unchanged.

Test coverage

  • 2 new Python test cases pin the fix: extract_tb() is called on the first kept frame only, and locals and the source line are captured together on the same frame — 732 Python tests and 79 Rust tests in total, with line coverage at 94%. The in-process writer fork test now filters CPython's multi-threaded fork() deprecation warning, which it triggers on purpose, so the suite runs warning-free on 3.12+.

v1.3.0

Choose a tag to compare

@kidoz kidoz released this 17 Sep 17:08

structguru v1.3.0

A performance and resilience release. The whole logging path — merging bound fields, call keywords and contextvars, redacting, rendering, and enqueuing — now runs in one native call driven by a RuntimeConfig compiled at configuration time, and callable-sink dispatch moved into the extension as well; measured against v1.2.3 on the same machine, a simple logger.info() is about 20% faster, bind() and brace formatting about a third, and redaction about 38%. Settings makes configuration reusable and defines exactly how the environment layers into configure(). Four resilience defects found by fault injection are fixed, including an os.fork() that stalled for tens of seconds in a prefork server under logging load. Nothing was removed from the public API, but three behavior changes are worth reading before you upgrade.

Behavior changes

  • configure() now layers the environment. Defaults are applied first, then the supported STRUCTGURU_* variables, then your explicit keywords. Previously explicit keywords and environment values did not compose in a defined order. Pass configure(Settings(...)) to ignore the environment entirely. configure() still replaces prior settings; the new update() keeps the options you omit.

  • Invalid levels and invalid environment values now fail loudly. Native configuration and set_level() reject unknown level names, booleans, and negative integers with ValueError instead of coercing them, and non-negative integer thresholds are accepted. An invalid value in a selected STRUCTGURU_* variable fails at import rather than starting with logging silently misconfigured, unless autoconfiguration is disabled.

  • Importing without the native extension fails at import. The Rust renderer has been the only logging path since v1.0, and the level table now comes from the core as well, so a missing structguru._rust raises the existing "requires its native extension" RuntimeError immediately instead of at first use. is_available() is kept for compatibility and is always True once the package has imported.

Performance

  • One native call per record. The writer merges bound fields, call keywords, and contextvars, redacts, renders, and enqueues in a single call, using a configuration compiled when you call configure() — no per-record configuration is marshalled across the boundary, and the console format is fused exactly like JSON. Output is byte-for-byte unchanged.

    benchmark v1.2.3 v1.3.0
    logger.info("Hello world") 2.75 µs 2.19 µs
    brace formatting 2.79 µs 1.80 µs
    bind() 2.64 µs 1.80 µs
    redaction 3.97 µs 2.47 µs
    contextvars merge 3.46 µs 2.31 µs
    blocking enqueue 0.18 µs 0.10 µs

    Three runs of the full suite per version on one machine, garbage collection disabled, deltas beyond run-to-run noise. Two isolated conversion micro-benchmarks moved the other way — _conversion_stats and _render_json_debug are 17% to 27% slower — which is the cost of snapshotting dictionary entries before a converter can mutate them, redacting logger and service metadata, and replacing unpaired surrogates. The end-to-end paths above already include it.

  • Callable-sink dispatch runs in the extension. The bounded queue, its worker thread, sink selection, delivery accounting, and deferred handler closes all moved to Rust, and the dispatcher is skipped entirely while no sink is registered. Caller cost with a sink configured drops by a third to a half. Lifecycle, backpressure, fork, and callback-reentrancy semantics are unchanged.

  • The rotating file sink stats instead of reopening. It no longer closes, reopens, and stats the log file before every record to notice a rotation by another process; it stats the path once and compares file identity with the handle it holds. With rotation enabled the writer thread handles 256 records in 0.63 ms instead of 5.08 ms, and a caller throttled by a full queue waits 2.6 µs per record instead of 21 µs.

  • The blocking enqueue only releases the GIL when the queue is actually full. Releasing it around every push forced a hand-off per record whenever another thread was runnable: four threads logging concurrently now spend 3.0 ms instead of 5.6 ms per 1,000 records, eight threads 6.0 ms instead of 10.5 ms.

  • configure(exception_carets=False) omits the PEP 657 position markers — the ~~~^^^ lines under each frame — from formatted tracebacks. CPython computes them per frame while formatting, and on 3.11+ they are most of the cost of logger.exception(): a 22-frame traceback formats in 92 µs instead of 475 µs. The output is exactly what CPython prints under PYTHONNODEBUGRANGES=1, including chained exceptions and exception groups. The default is unchanged.

Features

  • Settings is a frozen, validated configuration object with mapping and environment constructors, readable back through get_config() and adjustable through update().

    from structguru import Settings, configure, update
    
    base = Settings(service="checkout", level="INFO", format="json")
    configure(base)                      # ignores the environment
    configure(Settings.from_env())       # explicit environment layering
    update(level="DEBUG")                # keeps every other option, and in-flight delivery

    configure() keeps accepting its existing keywords through a typed wrapper, so no call site has to change. A level-only update() preserves the writer, buffered records, and rate-limit state rather than rebuilding them.

  • STRUCTGURU_LEVEL, STRUCTGURU_TARGET, STRUCTGURU_FORMAT, STRUCTGURU_SAMPLE_RATE, STRUCTGURU_RATE_LIMIT, and STRUCTGURU_AUTOCONFIGURE are read by autoconfiguration and by Settings.from_env(). The existing environment names remain supported.

  • lifecycle_metrics() reports the cumulative count of native deliveries rejected because their writer was closed. It survives reconfiguration and shutdown, starts fresh in a forked child, and is distinct from queue-full drops, sink errors, and filtered records.

Fixes

Four of these came from fault-injection experiments run against this release candidate; each one is now pinned by a regression test that fails against the previous build.

  • os.fork() no longer stalls while other threads are logging. The pre-fork drain calls flush(), and the native writer waited for its queue to be empty — a moment that never arrives when producers refill it faster than the sink drains it. In a prefork server with four logging threads and a file sink, fork() took 44 to 110 seconds in our measurements. A flush now waits only for the records enqueued before it was called, so its cost is bounded by the queue depth; the same experiment now returns in 0.29 s.

  • A callable sink that never returns no longer blocks shutdown() and interpreter exit forever. flush() and shutdown() give a stalled dispatch worker 10 seconds to make progress, then abandon the generation, warn with the number of undelivered records, and disable callable delivery until the next configure(). Records already handed to the stuck callback cannot be recovered, and the sink's own thread is left detached rather than killed — a timeout cannot safely terminate user code. If that callback ever does return, the abandoned worker winds down without starting another delivery or re-entering the interpreter.

  • The rate limiter's key table is capped. It keys on the formatted message, so a message that embeds an attempt number or an ID creates a new key per record and is never actually limited. One million such messages within a period grew the process by roughly 320 MB; the table now holds at most 8192 keys, sweeping stale buckets and then evicting the least recently seen, and the same workload costs 5.5 MB. Keys that are actively being limited are kept. The message-keyed behavior itself is unchanged and is now documented in configure().

  • A synchronous stream_sink whose write() raises is no longer silent. Failures are counted in writer_metrics()["stream_errors"] and warned about at a bounded rate. The failure is still contained and the record still reaches the native destinations, but lost stream copies are now visible.

  • Closing, reconfiguring, or shutting down in a forked child on Linux no longer panics with failed to join thread: Invalid argument. glibc reuses the parent's dead worker thread identity for the child's respawned worker, and releasing the abandoned parent writer detached that live thread; the stale handle is now leaked instead.

  • The stdlib bridge no longer deadlocks under callable-sink backpressure. It held its handler lock while waiting for queue space, so a sink callback that logged through logging could deadlock both sides. Handler filters, including replacement records on Python 3.12+, still apply.

  • A sink removed from inside a sink callback closes its handler only after the deliveries already queued for it have run. Closing at once left those deliveries writing to a closed handler: a FileHandler silently reopened its file and leaked the stream. The close is deferred to the worker that finishes the last delivery.

  • Queue overflow and closed-writer rejections are distinguished at enqueue time, so racing reconfiguration or shutdown cannot hide a rejection or mislabel it as a full queue. Shutdown drains callable deliveries before closing the writer so their nested logs finish, and interpreter exit clears the active runtime, so a later atexit handler sees disabled logging rather than a closed writer.

  • Redaction and text handling. Stack text is escaped and every continuation line indented for console output, with redaction still running first. Sensitive-key matching handles Unicode case variants, including nested fields. URL sanitization rejects URLs whose authority failed to parse — `https:/user:pw@host/...

Read more

v1.2.3

Choose a tag to compare

@kidoz kidoz released this 05 Sep 22:59

structguru v1.2.3

A correctness release. Three defects are fixed: a field value the native renderer could not represent failed the whole record — raising TypeError from a native logging call, and turning a bridged stdlib record such as Django's HTTP error log into a --- Logging error --- diagnostic on stderr; type detection probed the instance, so a lazily evaluated object was evaluated as a side effect of being logged; and the published wheels reported any value reachable twice as a cycle. No breaking changes; upgrade in place from any v1.x.

Fixes

  • A field value the renderer cannot represent no longer loses the record. The Rust converter raised TypeError for any leaf it did not recognize — a Path, a Decimal, bytes, a request object — and that exception took the whole event with it. From a native call it surfaced to the caller, so logger.info("upload", path=Path("/tmp/x")) raised inside application code. From the stdlib bridge it went to Handler.handleError(), which prints a traceback to stderr instead of emitting the JSON event. Django hits this on every HTTP error: django.utils.log.log_response() attaches the raw WSGIRequest as extra["request"], so the one record that mattered — the 500, with its traceback — was the one that never rendered.

    install_stdlib_bridge(level="INFO")
    log_response("%s: %s", "Internal Server Error", request.path,
                 response=HttpResponse(status=500), request=request, exception=exc)
    # before: --- Logging error --- ... TypeError: Object of type WSGIRequest is not serializable
    # after:  {"request":"<unsupported: WSGIRequest>","status_code":500,"exception":"Traceback (most recent call last): ...",
    #          "logger":"django.request","level":"ERROR",...,"message":"Internal Server Error: /repro"}

    Conversion is now total at the rendering boundary, so native bind() fields and bridged extra= fields follow one policy. An unsupported object becomes <unsupported: WSGIRequest> — the type name only. str() and repr() are never called on an arbitrary object, so headers, cookies, and bodies cannot leak into a log line. A container that refers back to itself becomes <cycle: dict>, and nesting beyond 64 levels becomes <max depth exceeded>. The markers apply at any depth inside mappings and sequences, are redacted by key and by pattern like any other string, and the record and its extra objects are left untouched for other handlers. A conversion that raises — a duck-typed isoformat() that fails, a dataclass field whose access raises — falls back the same way; only KeyboardInterrupt and SystemExit still propagate.

    Three adjacent rejections became representations rather than markers: integers outside the 64-bit range are written as JSON numbers, non-string mapping keys are rendered as strings ({200: 3} becomes {"200": 3}), and text with unpaired surrogates is written with U+FFFD. Code that caught the TypeError from a logging call no longer sees one.

  • Logging an object never evaluates it. Enum, datetime, UUID, and dataclass detection probed the instance with hasattr(), which runs the object's __getattr__. For Django's LazyObject — request.user — that is _setup() and a database query, triggered by a log call that was about to reject the object anyway. Detection now inspects the class; an instance attribute is read only after its type has matched.

  • Release builds no longer report repeated references as cycles. The cycle tracker popped a container off its path stack inside a debug_assert_eq!, which release builds compile out entirely, so the pop never ran in the published wheels. Any value reachable twice — the same dict in two list slots, one settings object referenced from two fields — was rejected as a cycle, and the record was lost.

    shared = {"x": 1}
    logger.info("shape", items=[shared, shared])
    # before: ValueError: cycle detected while converting Python value
    # after:  {"items":[{"x":1},{"x":1}],...,"message":"shape"}

    The pop now runs unconditionally and the assertion only checks its result. A genuine cycle is still detected on the current path and rendered as <cycle: list>.

Test coverage

  • 20 new Python test cases pin the fixed behavior: markers for unsupported objects at the top level and inside containers, cycles, the depth limit, shared references, big integers, non-string keys, surrogates, failing conversions, no instance probing, and KeyboardInterrupt propagation; all four renderer branches — JSON and console, with and without a compiled redaction configuration — with redaction around the markers; the bridge under either logging.raiseExceptions value, with nested and cyclic extras, and with an untouched record; and the Django log_response() reproduction end to end — 508 Python tests and 67 Rust tests in total.

v1.2.2

Choose a tag to compare

@kidoz kidoz released this 05 Sep 21:05

structguru v1.2.2

A dependency release. The native sampling filter now draws its per-record random number from fastrand instead of rand, which removes seven transitive crates from the wheel build — among them the yanked chacha20 release that blocked the first 1.2.1 publish. Sampling behavior, the public API, and the wheel matrix are unchanged. No breaking changes; upgrade in place from any v1.x.

Dependencies

  • The sampling filter uses fastrand instead of rand. The filter needs one uniform f64 in [0, 1) per record, drawn on the calling thread. It got that from rand::random, which routes through rand's ChaCha-based thread RNG — a cryptographically secure generator, with chacha20, rand_core, getrandom, cpufeatures, cfg-if, and r-efi behind it, for a sampling decision that never needed unpredictability. fastrand::f64() gives the same half-open draw from a per-thread WyRand generator seeded from the clock and thread id, and the crate has no dependencies on any wheel target.

    # before: rand 0.10  (+ rand_core, chacha20, getrandom, cpufeatures, cfg-if, r-efi)
    # after:  fastrand 2.5

    Cargo.lock shrinks from 44 packages to 38, and the release audit no longer depends on the RustSec status of a cryptographic crate the filter never used. The sampling tests — rate zero drops everything, rate one keeps everything, rate one half stays within its statistical bound — pass unchanged against the new generator. sample_rate semantics, the drop counters, and the sample_max_level gate are untouched.

Context

  • Why a point release for a dependency swap. The first 1.2.1 tag failed the release audit because chacha20 0.10.1 had been yanked after the lockfile was generated; the tag was moved to a commit with the updated lock before anything reached PyPI. Removing the crate entirely, rather than tracking its releases, closes that class of interruption for the sampling path. fastrand is maintained under the smol-rs organization, forbids unsafe code, is dual-licensed MIT/Apache-2.0, and is the generator tempfile itself relies on.

v1.2.1

Choose a tag to compare

@kidoz kidoz released this 05 Sep 20:04

structguru v1.2.1

A correctness release. Four defects are fixed: stack traces attached with stack_info= were written to the output without redaction, callable-sink lifecycle calls could deadlock or return before a sink callback had finished, the Celery hook let a parent task's identity overwrite the running task's, and the Sentry processor forwarded the raw exception object on paths that bypass redaction. The Rust workspace now requires pyo3 0.29.2, and the development toolchain is upgraded, including a mkdocs-material advisory fix for the documentation site. No breaking changes; upgrade in place from any v1.x.

Fixes

  • stack_info text is redacted in both output formats. The renderer applied sensitive_keys and sensitive_patterns to the message and to every structured field, but the stack value produced by stack_info= was written verbatim to JSON and console output. Anything sensitive that a formatted frame carried left the process unredacted.

    configure(sensitive_patterns=[r"(token=)\w+"], pattern_replacement="$1[MASKED]")
    logger.info("trace", stack_info="frame\ntoken=hunter2")
    # before: ... "stack": "frame\ntoken=hunter2", "message": "trace"}
    # after:  ... "stack": "frame\ntoken=[MASKED]", "message": "trace"}

    Pattern redaction now runs over the stack text exactly as it does over the message, and listing stack in sensitive_keys replaces the whole value with [REDACTED]. Console output redacts the appended stack the same way.

  • Callable-sink lifecycle operations neither deadlock nor return early. The callable dispatcher stopped a queue generation with a stop token under a re-entrant transition lock, which produced four related defects. A sink callback that called shutdown() or disable() while another thread was inside configure(), logger.remove(), stop(), or shutdown() deadlocked both threads: the outer call held the lock while waiting for the queue to drain, and the callback needed that lock to finish its record. A flush() or disable() racing a callback that had just disabled dispatch returned immediately, because the retired generation was no longer referenced anywhere. logger.remove(token) returned while a record captured in a generation retired by an earlier configure() was still being delivered to the removed sink. And closing a generation twice left a stop token behind that made the next flush() hang.

    Generations are now retired rather than stop-tokened: the worker exits once its generation no longer accepts producers and the queue is empty, retired generations stay tracked until they are actually drained, all waiting happens outside the state lock, and a callback never waits on its own worker thread. Every lifecycle operation still drains every record accepted before it began.

  • A Celery task launched from inside another task keeps its own identity. setup_celery_logging() bound the running task's task_id and task_name first and then applied the structguru_context header. When the producer was itself a task, that header carried the parent's task_id and task_name, so every record the child logged was attributed to the parent.

    # parent task (task_id="parent-id") calls child.delay(); the header carries the parent's context
    # before: {'request_id': 'req-999', 'task_id': 'parent-id', 'task_name': 'parent-task'}
    # after:  {'request_id': 'req-999', 'task_id': 'child-id', 'task_name': 'child-task'}

    The running task's identity is now bound after the propagated context, so correlation fields such as request_id still flow through while the task fields always describe the task that is actually executing.

  • SentryProcessor never forwards the raw exception object outside capture_exception. The event dictionary's exc_info value — the live exception — was copied into breadcrumb data, into the structlog_event extra, and, if listed in tag_keys, into a tag. The Sentry SDK serializes arbitrary objects, so an exception message containing a secret reached Sentry through those paths even though the rendered exception field had been redacted. exc_info is now stripped from all three; the raw exception is reserved for capture_exception, the one path meant to receive it.

Tooling

  • pyo3 0.29.2 is the workspace minimum. The lockfile already resolved to it; the floor now says so.

  • mkdocs-material 9.7.7 closes GHSA-xvg9-69gf-fjrf, a DOM XSS in the documentation site's search suggestions via the query parameter. This affects only the built documentation site, never the package.

  • The development toolchain is upgraded — ruff 0.16, mypy 2.3, pytest-benchmark 5.3 among others — and the dependency-group floors in pyproject.toml now match the locked versions. Ruff 0.16 formats Python code blocks inside Markdown, so the README and docs examples were reformatted to match.

Test coverage

  • 18 new Python test cases and 1 new Rust test pin the fixed behavior: stack redaction in both formats, by key and by pattern, on the linear and backtracking engines; the dispatcher's retired-generation drain, the callback-disables-dispatch race, idempotent close, and — in a bounded subprocess — the callback-shutdown deadlock against each lifecycle operation; the Celery child-task identity; and the Sentry exc_info paths — 488 Python tests and 67 Rust tests in total.

v1.2.0

Choose a tag to compare

@kidoz kidoz released this 27 Aug 17:43

structguru v1.2.0

A correctness and documentation release. Two defects are fixed: the gRPC interceptor leaked request context out of every streaming handler, and the documented rewrite for look-behind redaction patterns re-emitted the secret it was supposed to remove. get_contextvars() joins the public context API so custom adapters can implement the snapshot/restore sequence that fix required, and the generated API reference gains real redaction documentation plus a context-propagation guide for integration authors. No breaking changes; upgrade in place from any v1.x.

Fixes

  • Streaming gRPC handlers no longer leak request context. A streaming handler returns a lazy iterator — calling it runs none of the generator body — so the interceptor's grpc_method and request_id stayed bound to the thread for the whole interval between the handler returning and gRPC beginning to consume the response, and indefinitely if the client cancelled and the iterator was never consumed. Any unrelated record logged on that thread in the meantime carried the wrong request's identity.

    result = interceptor.intercept_service(continuation, details)
    stream = result.unary_stream(request, context)  # iterator created, not yet consumed
    # before: {'grpc_method': '/svc/Method', 'request_id': 'leaky'}
    # after:  {}

    The handler's context is now snapshotted and cleared when the iterator is returned, then restored when iteration actually begins. The snapshot matters for handlers that are not generators: one that returns a list runs eagerly during the call, so context it bound there is captured and restored rather than discarded by the clear. Unary handlers are unchanged.

  • The documented look-behind rewrite no longer leaks the secret. Rust's linear-time regex engine rejects look-around, and the README, the configure() docstring, and the ValueError raised on an unsupported pattern all recommended rewriting (?<=password=)\S+ as password=(\S+) with pattern_replacement="$1[REDACTED]". That group captures the secret, so $1 re-emitted it verbatim:

    configure(
        sensitive_patterns=[r"(password=)\S+"],  # group wraps the prefix, not the secret
        pattern_replacement="$1[REDACTED]",
    )
    # before (as documented):  "login password=hunter2 ok" → "login hunter2[REDACTED] ok"
    # after:                   "login password=hunter2 ok" → "login password=[REDACTED] ok"

    Redaction behavior itself was always correct — only the guidance was wrong — so key-based redaction and patterns without a capture group were never affected. If you copied the documented pattern into a configure() call, update it: the capture group belongs around the prefix you want to keep, never around the secret.

Features

  • get_contextvars() is exported from the package root, completing the set alongside bind_contextvars, bound_contextvars, and clear_contextvars. It returns a shallow snapshot of the bound context: mutating the returned dictionary does not change the context, and nested values remain shared.

    from structguru import bind_contextvars, clear_contextvars, get_contextvars
    
    bind_contextvars(request_id="req-1")
    snapshot = get_contextvars()   # {'request_id': 'req-1'}
    clear_contextvars()
    bind_contextvars(**snapshot)   # restore it later, on the same thread or another

    Adapters that must capture context and restore it across a lazy boundary no longer have to import it from the private structguru._contextvars.

Docs

  • A new Context Propagation guide for anyone writing or reviewing an integration adapter. It documents the copy-on-write model and how asyncio tasks (inherit a copy) differ from new threads (start empty), the three rules every adapter must hold, the lazy-result hazard behind the gRPC fix with the snapshot/clear/restore pattern, and the testing rules that catch this class of bug — including closing suspended generators so their cleanup does not run at an arbitrary GC point in an unrelated test.

  • The redaction API reference is real documentation rather than a one-line stub: what key- and pattern-based redaction each cover, the capture-group rule with correct and leaking examples side by side, the fail-closed backtrack limit under allow_backtracking_patterns, and the Sentry marker guard. It also documents that sensitive_keys replaces the built-in set rather than extending it — configure(sensitive_keys=["custom"]) leaves password unredacted — and shows the [*DEFAULT_SENSITIVE_KEYS, ...] spread.

  • Integration guide corrections. The Sentry section previously implied severity alone controls captured events; by default only events carrying exc_info become Sentry events, and a plain logger.error() stays a breadcrumb unless capture_messages=True. The HTTPX section now separates HTTP error responses from transport-level failures (connection refused, DNS, timeouts), which raise before the response hook runs and are not logged — and notes that logger.catch() defaults to reraise=False, so it would swallow them. The ASGI request_id_header is documented as case-insensitive, which is what the middleware has always done.

  • README accuracy. The console-format example showed a stale 2025-… [info ] line; the renderer emits an RFC 3339 timestamp and an uppercase padded [INFO ]. The abbreviated JSON example now matches the renderer's actual key order.

  • The public logging methods are documented. debug, info, warning, error, critical and the loguru-compatible aliases had no docstrings, and since the API reference is generated entirely from them, they appeared as bare signatures. trace → DEBUG and success → INFO are now stated explicitly.

Tooling

  • Ruff enforces docstrings on the published API surface. docs/api/ is generated wholly from docstrings, so a missing one is an invisible hole in the published reference rather than a visible gap. The D ruleset now runs over src/ with the NumPy convention; tests, examples, and scripts are exempt.

  • mkdocstrings parses the NumPy docstring style the codebase actually uses. It was configured for Google style while every module writes NumPy sections, so Parameters rendered as a raw heading instead of a parameter table. Parameters now render with name, type, description, and default, and the docs build runs clean under --strict.

Test coverage

  • 4 new tests pin the fixed behavior: the gRPC pre-iteration window (context must be clean before the first next()) and eager-handler restore, plus the get_contextvars root export and its shallow-snapshot contract — 467 Python tests and 66 Rust tests in total.

v1.1.0

Choose a tag to compare

@kidoz kidoz released this 25 Aug 19:02

structguru v1.1.0

A stdlib-bridge lifecycle release. install_stdlib_bridge() gains a replace option that releases the previous managed bridge before installing the new one — last call wins — so applications whose logging setup legitimately runs more than once per process (multiple entrypoints, repeated setup in test suites) no longer need a hand-rolled scan-and-uninstall loop. The fail-fast default is unchanged; upgrade in place from any v1.0.x.

Features

  • replace=True on install_stdlib_bridge(). With an active bridge, the old one is released with full uninstall_stdlib_bridge() semantics — detached from the root logger, closed, its existing-loggers snapshot restored — and then the new bridge installs. With no active bridge it behaves exactly like a plain install; with the default replace=False, a second install still raises RuntimeError.

    bridge = install_stdlib_bridge(
        level="INFO",
        suppress_loggers=("urllib3", "botocore"),
        replace=True,  # last call wins instead of raising "already installed"
    )
  • STRUCTGURU_STDLIB_REPLACE brings the same capability to install_stdlib_bridge_from_env(), parsed with the same strict boolean conventions as the other STRUCTGURU_STDLIB_* variables (1/0, true/false, yes/no, on/off; empty means unset; an invalid value raises ValueError before any logging state is changed). Default false.

  • Contractual swap semantics. The release-and-install sequence runs in one critical section. A record logged by another thread during the swap is delivered at most once — rendered by the outgoing or incoming bridge, raw, or not at all — never twice, and never by raising. The replacing install re-resolves disable_existing_loggers (explicit value → environment → preserve) and restores the old snapshot before applying the new policy. uninstall_stdlib_bridge() on a replaced handler is a documented no-op, and suppression levels applied by the earlier install are kept.

  • The runnable example grew a replace scenario. examples/stdlib_existing_loggers/main.py now replaces the bridge from code and from the environment, showing that a logger created between the two installs counts as existing for the replacing install's policy — and it still runs as part of the test suite so it cannot rot.

Docs

  • The README and the integrations guide document the replace option, the concurrent-swap delivery guarantees, the cumulative-suppression rule, and the new environment variable in the configuration table.

Test coverage

  • 12 new tests pin the replace semantics: swap with exactly-once delivery, the unchanged fail-fast message, no-op and detached-bridge paths, both disable_existing_loggers transitions asserting restore-then-apply ordering, environment re-resolution on replace, cumulative suppression, stale-handler uninstall safety, STRUCTGURU_STDLIB_REPLACE set/unset/invalid, and a concurrency smoke test with a live logger.add() sink — 463 Python tests in total; the Rust core is untouched.

v1.0.6

Choose a tag to compare

@kidoz kidoz released this 25 Aug 17:47

structguru v1.0.6

A stdlib-bridge control release. The bridge and Django's config builder gain a disable_existing_loggers policy with dictConfig semantics — but reversible — every bridge option can now be configured from environment variables at an explicit call site, and the managed installer refuses to stack a second bridge instead of silently duplicating every record. No changes to correct existing usage; upgrade in place from any v1.0.x.

Features

  • disable_existing_loggers on install_stdlib_bridge(). True disables named loggers that already exist at installation time, False re-enables them — the same policy logging.config.dictConfig applies, except reversible: uninstall_stdlib_bridge() restores the states the bridge changed, unless application code changed them again in the meantime. The root logger, and loggers created after installation, are never touched.

    bridge = install_stdlib_bridge(level="INFO", disable_existing_loggers=True)
    # ... existing named loggers are silenced while the bridge runs ...
    uninstall_stdlib_bridge(bridge)  # their previous states come back

    When the option is omitted, the installer reads STRUCTGURU_STDLIB_DISABLE_EXISTING_LOGGERS; when that is unset too, existing states are preserved. An explicit Python value always wins over the environment.

  • install_stdlib_bridge_from_env() — configure every bridge option from STRUCTGURU_STDLIB_* variables at a controlled point in application startup. Importing structguru still never touches the root logger; only the explicit call reads the environment.

    Variable Default
    STRUCTGURU_STDLIB_LEVEL LOG_LEVEL, then INFO
    STRUCTGURU_STDLIB_CLEAR_HANDLERS true
    STRUCTGURU_STDLIB_DISABLE_EXISTING_LOGGERS preserve existing states
    STRUCTGURU_STDLIB_SUPPRESS_LOGGERS empty comma-separated list
    STRUCTGURU_STDLIB_SUPPRESS_LEVEL WARNING

    Booleans accept 1/0, true/false, yes/no, and on/off; an invalid value raises ValueError before any logging state is changed.

  • Django's build_logging_config() accepts disable_existing_loggers. When omitted it reads the same environment variable and falls back to the previous hard-coded False, so existing settings modules keep their behavior.

  • A runnable example. examples/stdlib_existing_loggers/main.py demonstrates the policy from code and from the environment, and runs as part of the test suite so it cannot rot.

Changed

  • Only one managed bridge can be active. A second install_stdlib_bridge() call now raises RuntimeError instead of stacking a second handler that delivered every third-party record twice. A bridge whose handler was detached by hand is cleaned up — its logger-state snapshot restored — before a replacement installs.

Packaging & CI

  • Cargo dependencies refreshed: fancy-regex 0.18 → 0.19, pyo3 0.29.2, plus compatible serde, serde_json, regex, and time updates.
  • The internal path dependency is exempted from the cargo-deny wildcard ban, so cargo deny enforces the ban for real dependencies without failing on the workspace-internal crate link.

Docs

  • The README and the integrations guide document the existing-logger policy, the full environment-variable table, and the precedence rules (explicit value → environment → preserve).

Test coverage

  • 40 new tests pin the policy semantics: preserve/disable/re-enable and restore-on-uninstall (including the "application changed it since" case), placeholder and root/later-logger exclusions, environment precedence and validation-before-mutation, duplicate-install rejection, and the example smoke paths — 451 Python tests and 66 Rust tests in total.