Repository navigation
v3.0.0
Recursive header and metadata redaction, an atomic block-policy buffer, and a maintainability split (v3.0.0)
Breaking Changes
BufferProtocol.requeue_events_in_memoryandrequeue_metrics_in_memoryare now async. A failed send's requeue now shares the same lock as every other buffer mutation, so both methods on the protocol (and onEventBuffer) must be awaited. A customBufferProtocolimplementation needs to change its signatures toasync def;GuardAgentHandler's own buffer andFlushMixinalready call them withawait.sanitize_headersnow redacts what it cannot safely classify instead of letting it through. A JSON-looking string over 8192 characters, a value of a type it does not recognize (a generator, an arbitrary object), and a bytes header key that fails to decode as UTF-8 previously passed through unchanged or matched nothing; all three now become[REDACTED]. A payload that relied on that pass-through will see[REDACTED]there instead.
Added
sanitize_headersrecurses into every container and into JSON-looking strings, including a double-encoded JSON string. Dicts, lists, tuples, sets, frozensets and JSON string bodies are all walked (up to a bounded depth), not just the top-level mapping; a JSON string whose decoded value is itself a JSON string is unwrapped and sanitized too, then re-encoded the same way.datetime,date,time,Decimal,UUIDandEnumvalues pass through unchanged. They are legitimate scalar metadata, not unrecognized objects to redact.- A dataclass instance or a pydantic model passed as metadata is walked field by field, like a mapping, via
dataclasses.asdict/model_dump, instead of being collapsed to[REDACTED]wholesale. - The default
sensitive_headersset now matches guard-core's:proxy-authorizationjoinsauthorization,cookieandx-api-key. KNOWN_EVENT_TYPESgainedip_ban_failed,rate_limit_script_reloaded,pattern_anomaly_timeout,pattern_anomaly_statistical_anomalyandroute_unresolved, aligning with guard-core 4.0.0's event catalogue. A version-gated test skips until the installedguard_corereaches 4.0.0 and fails on any other drift once it does.tests/test_maintainability_rank.pyenforces radon MI rank A on every module inguard_agent.on_erroralso fires at the flush boundary (stage='flush_events'/'flush_metrics'), once per failed batch, with the exception and{"batch_size": N}, through the samefire_error_hookhelperHTTPTransportuses. Previously it only fired inside the transport's own retry paths.
Fixed
- A bytes header key no longer bypasses the sensitive-header match.
b"authorization"is decoded as UTF-8 before the case-insensitive comparison instead of being stringified to the literal"b'authorization'", which never matched anything. - The event and metric buffers each serialize their overflow-check-and-append, flush, requeue and clear through one
asyncio.Lock. Two concurrentadd_event(oradd_metric) calls at capacity could previously both evict the same slot, appending past it and orphaning the other call's Redis record; the check and the append are now one atomic step. - The block overflow policy waits on an
asyncio.Conditionbound to that same lock and re-checks at least once every 0.5 seconds. A requeue after a failed send always keeps its slot (durability over a new writer), and a blocked writer no longer depends solely on an explicit signal that a send-failure backoff (up to 300 seconds) can delay past. clear_buffernow clears the Redis-key maps along with the buffers. Previously only the deques were cleared, so a later event or metric object could inherit a stale or foreign key once CPython reused itsid().- A Redis record is deleted on a requeue eviction and on an overflow drop without racing the buffer it was tracked against.
- The Makefile's
semgreptarget scansguard_agentwith--no-git-ignore, so a new, not-yet-tracked file is scanned deterministically instead of depending on git state. - A
BaseExceptionfrom the transport (anasyncio.CancelledError, not anExceptionsubclass) no longer skips the batch's requeue and failure bookkeeping._flush_events/_flush_metricscatchBaseException, requeue the popped batch and advance the failure streak and backoff before re-raising, so a cancellation still propagates but never silently drops the batch. - Loading persisted events/metrics from Redis at startup no longer races live
add_event/add_metrictraffic. Each load now runs under its buffer's own condition lock, and forgets the oldest tracked key before its own append would otherwise evict an item past capacity, so a startup load can never orphan a Redis record or corrupt the key map racing a concurrent write.
Internal (v3.0.0)
buffer.py,client.pyandtransport.pywere split into focused mixins (_buffer_lifecycle,_buffer_overflow,_buffer_queue,_buffer_redis;_client_flush,_client_ingest,_client_loops,_client_status;_transport_dispatch,_transport_lifecycle,_transport_send), each independently testable and each rank A.anyio<4.15andsemgrepwere added to thedevextra.
What's Changed
- release(v3.0.0): redaction at ingest and egress, lossless atomic buffer, rank-A split, event-type parity by @rennf93 in #52
Full Changelog: 2.10.0...3.0.0