v3.15.0

Post-3.14.0 follow-ups: redis_fail_open honored by the rate limiter, a depth-bounded JSON body walk and per-request scan-char budget, corrected X-Forwarded-For depth over-counting and oversized-body scanning, no more double-logging into host handlers, and a rank-A internal module split (v3.15.0)

Special thanks to @RomanPszonka for the reports and advisories behind this release and the support throughout. Thank you for making the Guard Core ecosystem better!

Highlights

• Breaking: the rate limiter now honors redis_fail_open on a Redis failure instead of always falling back to the in-memory window. redis_fail_open=True still falls back to the in-memory window but now logs a WARNING once per process instead of an ERROR on every request; redis_fail_open=False (the default) now raises GuardRedisError, letting the pipeline apply fail_secure exactly as for any other check's Redis failure instead of silently degrading a shared rate limit to a per-process one.
• Bounded JSON body walk and per-request scan-char budget. A new SecurityConfig.detection_max_json_depth (default 32) rewrites the JSON body walk as a depth-bounded, recursion-free iteration instead of one Python call per nesting level, and a subtree reached at that depth is serialized back to text and scanned as one value (GHSA-f6cf-jjhc-qp85); a new SecurityConfig.detection_max_scan_chars (default 65536) separately bounds the total characters handed to the pattern engine per request (GHSA-3hfx-8m47-5f9h residual).
• X-Forwarded-For over-count correction. A declared trusted_proxy_depth that over-counts the real proxy hops let a client rotate its resolved identity freely; guard-core now verifies every entry to the right of the depth-selected one is itself a listed trusted proxy and, when it is not, resolves identity by walking the chain right to left instead, warning once per process (GHSA-8xvm-856x-7hwp claim 1 residual).
• Oversized-body scanning. A body whose declared Content-Length exceeds detection_max_body_inspect_bytes is no longer silently skipped when the adapter supports a bounded read; the first detection_max_body_inspect_bytes bytes are read through it and scanned instead, keeping the existing single-memory-bound guarantee (GHSA-3hfx-8m47-5f9h residual).
• Auto-configuring detection singleton. detect_penetration_attempt(request, config) now configures the detection singleton from config itself whenever it is unconfigured or was last configured from a different config object, so a direct caller (guard-core-mcp's check_payload, a PoC script, a test written against this function directly) runs the same enhanced detection path an adapter runs instead of the slower legacy fallback.
• No more double-logging into host handlers. setup_custom_logging's console handler now carries a filter that checks the host's root logger handlers at the moment each record is emitted, yielding to them whenever they exist instead of printing every guard security event twice when a host has already configured its own root logging.
• Internal module split, rank A across the board. suspatterns_handler.py, handler_initializer.py, _security_config_validators.py and _redos_structure.py are restructured into cohesive private sibling modules, with every previously importable name re-exported from its original module path; detection results, sync mirrors, and behavior are unchanged.
• Bug-hunt fixes. A signature straddling the oversized-body scan boundary is now caught the same way whether or not the client declared an oversized Content-Length; the sync BehaviorTracker stores are now thread-locked, closing a RuntimeError under a real WSGI thread pool; an X-Forwarded-For entry carrying a port (1.2.3.4:5678, [2001:db8::1]:5678) is now parsed instead of discarding the whole header.

Added

• SecurityConfig now warns at construction when whitelist contains a /0 network. A whitelist entry of 0.0.0.0/0 or ::/0 makes every address whitelisted, so blacklist, blocked_countries and IP bans can never block anyone; this was previously silent. Precedence and every access decision are unchanged, a /0 whitelist still allows everyone, this is a signal only (#79).

Changed

• PerformanceMonitor.record_metric no longer recomputes its running average with statistics.mean's exact-Fraction arithmetic. Profiling the detection_max_scan_chars budget's worst case (28 values x 9342 characters) found this call, made once per pattern check (about 8600 times for that request), accounted for roughly 30% of enhanced-mode detection CPU; stats.avg_execution_time is now math.fsum(stats.recent_times) / len(stats.recent_times), the same approach monitor_anomalies.py already used elsewhere, correct to within ordinary floating-point rounding. First of three constant-factor cuts named in the profile report for GHSA-3hfx-8m47-5f9h; none change any detection decision.
• SemanticAnalyzer.analyze no longer tokenizes its input twice. analyze_attack_probability and the token_count field both called extract_tokens independently on the same content; analyze now extracts tokens once and reuses them for both. Second of the three constant-factor cuts.
• The url-decoded detection view no longer re-runs percent/unicode decoding from scratch. _check_url_decoded_view_patterns called preprocess_url_decoded_newline_preserving, which independently repeated decode_common_encodings on the raw content even when the main preprocessing pass had already decoded it, or found nothing to decode; it now reuses the main pass's decoded intermediate (ContentPreprocessor.preprocess_with_decoded) instead of decoding twice. Third of the three constant-factor cuts.
• Breaking: the rate limiter now honors redis_fail_open on a Redis failure instead of always falling back to the in-memory window. _redis_request_count (check_rate_limit and check_rate_limit_by_ip, guard_core/handlers/ratelimit_handler.py) previously caught every Redis error itself and silently used the in-memory window regardless of redis_fail_open, so a Redis outage silently turned a shared, cross-worker rate limit into a per-process one (N workers effectively multiplying rate_limit by N) with no way to opt out. redis_fail_open=True still falls back to the in-memory window, but now logs a WARNING once per process ("Redis unavailable for rate limiting...") instead of an ERROR on every request. redis_fail_open=False (the default) now raises GuardRedisError instead of silently falling back: check_rate_limit lets SecurityCheckPipeline._handle_check_error apply fail_secure exactly as for any other check's Redis failure (500 under fail_secure=True, pass-through otherwise), and check_rate_limit_by_ip lets the exception reach its own caller. A deployment relying on the previous always-fall-back behavior with the default redis_fail_open=False now needs to set redis_fail_open=True explicitly to keep it. NoScriptError reload handling and the auto-ban feed are unchanged.
• Internal module split, no public name changes. guard_core/handlers/suspatterns_handler.py, guard_core/core/initialization/handler_initializer.py, guard_core/_security_config_validators.py and guard_core/detection_engine/_redos_structure.py were each at maintainability rank below A; every one is restructured into cohesive private sibling modules (guard_core/handlers/_suspatterns_*.py, guard_core/detection_engine/_redos_structure_*.py, and similarly for the other two), with every previously importable name re-exported from its original module path. Detection results, sync mirrors, and behavior are unchanged.

Fixed

• A signature straddling the detection_max_body_inspect_bytes cap boundary was missed when the client declared an oversized Content-Length, but caught when it did not. _read_capped_body_prefix (no declared Content-Length, or one at or under the cap) fetched max_bytes plus a small overlap (the longest compiled pattern length, up to 256 bytes) so a signature split across the cap boundary was still scanned in full; _read_oversized_declared_body (a declared Content-Length over the cap, added in the previous fix wave) read exactly max_bytes with no overlap, so the identical body missed detection on that path alone. Both paths now compute the fetch size through one shared helper, _capped_body_fetch_size. Sync mirror regenerated.
• CloudManager.refresh_async and _refresh_providers_via_redis_handler no longer apply freshly fetched cloud IP ranges to the in-memory cache before the store write that persists them. A store or Redis write failure was caught and logged, but the in-memory ip_ranges had already been updated, so is_cloud_ip reported ranges that were never persisted, a silent divergence between what a single process believes and what other workers see through Redis. The write now happens first; on failure the provider falls through to the existing preserve-or-empty fallback, the same one already used for a fetch failure. Sync mirror regenerated.
• BehaviorTracker.usage_counts/return_patterns outer stores, keyed by endpoint id, are now bounded. Only the inner per-client store was LRU-bounded; the outer dict grew one entry per distinct endpoint id forever. It now evicts the least-recently-touched endpoint once it tracks 10,000, the same _lru_pop_or_create pattern already used for the inner store, dropping that endpoint's entire inner store on eviction.
• The sync BehaviorTracker.usage_counts/return_patterns in-memory stores had no lock, unlike their async counterparts (which never need one). get_recent_event_count iterated usage_counts.values() while track_endpoint_usage/track_return_pattern inserted or LRU-evicted entries from another thread with no synchronisation; under a real WSGI thread pool this raised RuntimeError: dictionary changed size during iteration (measured: 55 times across 6 writer and 2 reader threads in 3 seconds), silently dropped by the enricher's blanket exception handler. BehaviorTracker.__init__ now builds a threading.Lock in the async source (a pattern already used elsewhere, e.g. core/checks/helpers.py's _suspicious_counts_lock) that guards every read and write of both stores; it carries through the sync mirror unchanged since a plain threading.Lock needs no async-to-sync translation, and costs one uncontended acquire per call in the async tree. ratelimit_handler.py's sync-only _redis_fail_open_warned flag now flips under its existing _by_ip_lock too, so the warn-once guarantee holds under threads.
• An X-Forwarded-For entry carrying a port was discarded, falling back to the connecting peer for the whole header. 1.2.3.4:5678 and [2001:db8::1]:5678 failed ip_address() parsing outright, since neither _strip_ip_brackets nor anything downstream stripped the port. Each comma-separated entry now has a trailing port stripped before parsing (_strip_forwarded_entry_port, guard_core/_utils/ip_extraction.py): a bracketed [v6]:port shape or an entry with exactly one colon has the port removed; a bare (unbracketed) IPv6 address, which always carries two or more colons, is never mistaken for host:port and is left unchanged, as is an entry that looks like host:port but has a non-numeric or missing port, which still fails parsing through the existing malformed-entry path. Sync mirror regenerated.
• guard-core no longer double-logs into a host's own root logger handlers. setup_custom_logging unconditionally attached a console StreamHandler to the guard_core logger and left propagate=True; a host that had already configured its own root logger handlers (the common logging.basicConfig() case) printed every guard security event twice, once formatted by guard's own handler and once via propagation through the host's. A setup-time check alone is not enough, since the ordering can go the other way: guard configures itself first while the root logger is still empty, and the host configures its own root handlers afterward, which reintroduces the same duplicate under the opposite ordering. setup_custom_logging now always attaches its console handler, but that handler carries a logging.Filter that checks logging.getLogger().handlers at the moment each record is emitted rather than at setup time: guard's console output yields to the host's root handlers whenever they exist, whichever was configured first, and the event propagates to the host's own handlers exactly once. The custom_log_file handler carries no such filter and is attached in every case, unaffected, and level handling is unchanged. A host that wants guard's own formatted output (text or JSON) while running its own root handlers should set custom_log_file, or attach its own handler directly to the guard_core logger.

Security

• Uncontrolled recursion in the JSON body walk (CWE-674). The structural JSON-body scan recursed one Python call per nesting level with no depth bound; a {"a":{"a":...}} body nested past Python's recursion limit raised RecursionError out of detect_penetration_attempt, and just below that limit the enhanced-detection exception fallback silently swallowed the error and scanned the request as clean instead of failing secure. The walk is now an explicit-stack iteration with no recursion anywhere in guard-core's own JSON traversal. A new SecurityConfig.detection_max_json_depth (default 32, 1 to 1000) bounds the nesting depth walked structurally; a dict or list reached at that depth is serialized back to text and scanned as one value instead, bounded by detection_max_content_length, and a one-time warning names the client IP. _check_value_enhanced no longer treats RecursionError as a fallback case, it now re-raises into the pipeline's fail-secure handling instead of ever returning a silent clean verdict (GHSA-f6cf-jjhc-qp85).
• Uncontrolled recursion in embedded-JSON detection for query, header and form values (CWE-674, same class as GHSA-f6cf-jjhc-qp85, different context). _try_check_json_value (the check that looks for a JSON object embedded in a single query parameter, header, or form-field value) only caught json.JSONDecodeError around json.loads; a value holding around 1000 nested braces raised RecursionError instead, out of detect_penetration_attempt, before the surrounding fallback logic ever ran. RecursionError is now caught in the same place and treated as "not embedded JSON": the value falls through to the ordinary pattern scan of its raw text instead of the structural walk, and the existing detection_max_json_depth warning fires once per request.
• A declared trusted_proxy_depth that over-counts the real proxy hops let a client rotate its resolved identity freely (GHSA-8xvm-856x-7hwp, claim 1 residual). With trusted_proxies non-empty, extract_client_ip selected ips[-trusted_proxy_depth] from X-Forwarded-For with no check that the entries to its right were actually appended by trusted proxies; when the real chain had fewer hops than the declared depth, a client that supplied extra entries controlled which one landed at that position, and the 3.14.0 chain-warnings did not catch it because the selected entry is an ordinary public address, not itself a listed trusted proxy. guard-core now verifies that every entry to the right of the depth-selected one is itself a listed trusted proxy (canonicalised, so an IPv4-mapped or bracketed IPv6 spelling is recognised); if any is not, it logs one warning per process naming trusted_proxy_depth, the count of untrusted entries, and the fix, and resolves the identity by walking the chain right to left, returning the first entry that is not a listed trusted proxy (the address the nearest genuine proxy actually appended). Nothing changes when trusted_proxies is empty, when the chain is shorter than the declared depth, or when every entry to the right of the selection is already a listed trusted proxy.
• A body whose declared Content-Length exceeds detection_max_body_inspect_bytes is no longer silently skipped when the adapter supports a bounded read (GHSA-3hfx-8m47-5f9h residual). _read_capped_body returned None for the whole request body once the declared length exceeded the cap, so an attacker could evade body-based detection entirely just by declaring an oversized Content-Length, even with an attack in the first few bytes. When the request implements the bounded reader (read_body_prefix), it now reads only the first detection_max_body_inspect_bytes bytes through it and scans that prefix, keeping the existing single-memory-bound guarantee (a JSON body that gets cut mid-structure falls back to the existing blob-text scan); when the request does not implement a bounded reader, the body is not read at all, the same skip as before this change, since reading it any other way would mean buffering the whole oversized body first. Either way a one-time warning names the cap, the client, and which of the two happened. A new SecurityConfig.detection_max_scan_chars (default 65536, 1024 to 262144) separately bounds the total characters handed to the pattern engine per request, at the same accounting point detection_max_scan_values uses; a handful of individually large values could previously cost as much CPU as many small ones without ever approaching the value-count cap (28 values x 9342 characters cost 1.81s CPU in enhanced mode). The budget is a leaky bucket, not a per-value truncation: a value already in progress when checked is always scanned in full even if that pushes the running total over the cap, so a single large legitimate value is never silently dropped; only a value that would start after the budget is already spent is skipped, with the same one-time warning. detect_penetration_attempt(request, config) also now configures the detection singleton from config itself whenever it is unconfigured or was last configured from a different config object (compared by identity, so this is a no-op once already configured from the same object), instead of requiring the caller to call sus_patterns_handler.configure(config) first: a direct caller that never did (guard-core-mcp's check_payload, a PoC script, a test written against this function directly) previously ran the legacy per-pattern thread-pool dispatch, an order of magnitude slower than the enhanced path and, on some payloads, producing different detection results.

---

What's Changed

• release(v3.15.0): post-3.14.0 follow-ups by @rennf93 in #83

Full Changelog: 3.14.0...3.15.0