Skip to content

v3.14.0

Choose a tag to compare

@rennf93 rennf93 released this 26 Aug 19:39
· 159 commits to master since this release
500d4ef

Post-3.13.0 hardening: identity and proxy-trust warnings, bounded in-memory stores, resilient Redis/GeoIP/Azure startup, telemetry secret redaction, and detection scan-cap and SSRF hardening (v3.14.0)

Special thanks to @RomanPszonka for the twelve reports and advisories behind this release and the support throughout. Thank you for making the Guard Core ecosystem better!

Highlights

  • Breaking: PerformanceMonitor.get_summary_stats/get_slow_patterns/get_problematic_patterns are now coroutines, and detection_max_scan_values has a floor. All three reporting methods are async def in the async tree and must now be awaited; the guard_core.sync mirror keeps them synchronous. SecurityConfig.detection_max_scan_values now requires at least 2 (each named value costs two scan units, so 1 could never scan a value) (GHSA-3hfx-8m47-5f9h).
  • Missing-client requests rejected; new unix trusted-proxy token. A request with no client address (request.client_host is None) is now rejected instead of skipping the entire security pipeline; a new "unix" token in trusted_proxies resolves X-Forwarded-For behind Unix-socket deployments (GHSA-634g-4wr8-xwxv).
  • X-Forwarded-For chain warnings. guard-core now warns once when a forwarded-for chain cannot satisfy trusted_proxy_depth, and once when the depth-selected entry is itself a listed trusted proxy; resolved identity is unchanged in both cases (GHSA-8xvm-856x-7hwp).
  • Prefix-0 trusted-proxy and empty-detection-category startup warnings. SecurityConfig now warns at construction on a /0 trusted-proxy network (0.0.0.0/0, ::/0) and on an empty enabled_detection_categories with detection enabled, two previously silent misconfigurations (#79).
  • Ban canonicalisation. ban_ip, is_ip_banned, and unban_ip canonicalise the address before storing, querying, or deleting it, closing a silent no-op when the same IP is banned and unbanned under different spellings (#81).
  • Redis-outage startup degrade. A Redis outage at startup no longer crashes the app; redis_fail_open=True degrades to in-memory backends, redis_fail_open=False re-raises so the adapter returns a clean error (#76).
  • GeoIP last-known-good. IPInfoManager keeps the last known good GeoIP database on a failed refresh or download instead of deleting it, and downloads to a temporary file that swaps in atomically (#78).
  • AzureCloud service tag. Azure cloud-IP fetch now loads the AzureCloud service tag by name instead of whichever tag sorts first (#77).
  • Rate-limit stores bounded; Retry-After on 429s. In-memory rate-limit stores are now LRU-bounded at 10,000 IPs; every 429 response now carries Retry-After (GHSA-g53w-gmp9-9ch3, #81).
  • Behavior-tracker stores bounded. BehaviorTracker.usage_counts/return_patterns per-client stores are now LRU-bounded at 10,000 clients per key, the same pattern applied to the rate-limit stores above (GHSA-g53w-gmp9-9ch3).
  • Float anomaly statistics. PerformanceMonitor's statistical-anomaly check replaces exact-Fraction arithmetic with math.fsum-based float mean/variance, removing an unbounded per-value CPU tax (GHSA-grqq-qh92-hw79).
  • Per-request scan cap. SecurityConfig.detection_max_scan_values bounds the number of request values scanned per request, replacing an unbounded scan an attacker could exhaust with padding (GHSA-3hfx-8m47-5f9h).
  • Telemetry secret redaction. The IPInfo token and Redis password no longer reach telemetry: the token moves to an Authorization header, and RedisManager.initialize strips userinfo from redis_url before it reaches agent events (#80).
  • Header-name validation. Custom security-header names, including Redis-loaded ones, are now validated against RFC 9110 token grammar and rejected on CRLF or other non-token characters (#81).
  • SSRF form coverage. ssrf now detects the IPv4-mapped IPv6 loopback bracket form and a trailing-dot localhost (#81).

Added

  • New SecurityConfig.detection_max_scan_values bounds the number of request values scanned per request, including JSON embedded within a single query-parameter, header, or body value. Default 512. Once reached, remaining query-parameter, header, and body values are not scanned, and a one-time logger.warning names the client IP, replacing an unbounded per-request scan an attacker could exhaust with padding (GHSA-3hfx-8m47-5f9h).

Changed

  • PerformanceMonitor.get_summary_stats, get_slow_patterns, and get_problematic_patterns are now coroutines in the async tree. They snapshot recent_metrics/pattern_stats under the monitor's lock before reading, the same fix applied to record_metric's statistical-anomaly check; a direct caller must now await them. SusPatternsManager.get_performance_stats, the only production caller, was already async and its own call signature is unchanged. The guard_core.sync mirror keeps all three as plain methods.

Fixed

  • A request with no client address (request.client_host is None, e.g. behind a Unix domain socket or a misbehaving ASGI adapter) is now rejected instead of skipping the entire security pipeline. fail_secure=True (the default) returns 403 with a one-time warning naming the cause and the fix; fail_secure=False runs the pipeline with identity "unknown", the same fallback already used elsewhere, with the same one-time warning. Excluded paths (health and readiness endpoints) are unaffected: they pass through before identity is resolved, exactly as before, so a Unix-socket deployment upgrading without "unix" in trusted_proxies does not start failing orchestrator probes. A new "unix" token in trusted_proxies marks a peer-less connection as a trusted hop, so X-Forwarded-For still resolves the real client on Unix-socket deployments. Under fail_secure=False, check_ip_access now treats the "unknown" identity as no address available: the request is allowed unless a whitelist or a country allow-list is configured (whitelist or whitelist_countries), and the blacklist, blocked_countries, and cloud-provider checks are skipped since none can match without an address; check_route_ip_access applies the same rule to a decorated route's ip_whitelist/whitelist_countries, so a route with any other restriction (or none at all) no longer 403s an "unknown" identity outright (GHSA-634g-4wr8-xwxv).
  • guard-core now warns once when an X-Forwarded-For chain cannot satisfy trusted_proxy_depth, and once when the depth-selected entry is itself a listed trusted proxy. Both are misconfiguration signals; resolved identity is unchanged in both cases (a chain shorter than the depth still falls back to the connecting peer, exactly as before). trusted_proxy_depth semantics are unchanged: it is not reinterpreted as a ceiling (GHSA-8xvm-856x-7hwp).
  • SecurityConfig now warns at construction on two silent-gap configurations. A trusted_proxies entry that is a /0 network (0.0.0.0/0, ::/0) trusts every peer to set X-Forwarded-For; an empty enabled_detection_categories with enable_penetration_detection=True runs detection that can never match anything. Both previously succeeded with no signal; both now log a WARNING and still succeed (#79).
  • ban_ip, is_ip_banned and unban_ip now canonicalise the address before storing, querying or deleting it. 2001:DB8::1, 2001:db8::1 and the fully expanded form (and IPv4-mapped IPv6 like ::ffff:1.2.3.4 vs 1.2.3.4) were previously three different keys on both the in-memory cache and Redis, so unban_ip called with a different spelling than the one used to ban was a silent no-op. CIDR entries are unaffected (_canonicalize_ip returns non-address strings unchanged) (#81).
  • HandlerInitializer.initialize_redis_handlers no longer lets a Redis outage at startup crash the app. GuardRedisError from RedisManager.initialize() is now caught and logged; redis_fail_open=True degrades to the in-memory backends used when Redis is disabled instead of wiring ip_ban_manager, the rate limiter, and sus_patterns_handler to Redis, and redis_fail_open=False (default) re-raises so the adapter returns a clean error instead of an unhandled exception. sus_patterns_handler.initialize_redis and IPInfoManager.initialize/_download_database now also catch a Redis failure on their own eager cache reads and writes, logging a warning and keeping the handler wired to Redis for later retries instead of letting the exception escape past the fail-open degrade (#76).
  • IPInfoManager keeps the last known good GeoIP database on a failed refresh or download. initialize() and refresh() no longer delete the on-disk .mmdb file or clear reader when a download fails; maxminddb.open_database is now guarded at every call site, and only an unreadable (corrupt) file is removed. refresh() no longer clears reader before attempting the download, and only swaps in the new reader once the fresh database opens successfully (#78).
  • Azure cloud-IP fetch now loads the AzureCloud service tag instead of whichever tag sorts first. The Microsoft ServiceTags document lists many tags per download; fetch_azure_ip_ranges previously took values[0], which is not guaranteed to be AzureCloud. It now selects the entry named AzureCloud and logs an error and returns an empty set if that tag is missing (#77).
  • In-memory rate-limit stores are now bounded, and 429 responses carry Retry-After. _by_ip_request_timestamps, _by_ip_autoban_counts, and RateLimitManager.request_timestamps grew one entry per distinct IP forever; all three now evict the least-recently-touched key once they track 10,000 IPs, the same LRU-bound pattern _increment_suspicious_counts already uses. _handle_rate_limit_exceeded now sets Retry-After to the effective rate-limit window on every 429 it returns, in-memory and Redis-backed alike. Sync mirror hand-updated (GHSA-g53w-gmp9-9ch3, #81).
  • BehaviorTracker.usage_counts/return_patterns per-client in-memory stores are now bounded. Each outer key (endpoint id, or endpoint:pattern for return-pattern rules) now evicts its least-recently-touched client IP once it tracks 10,000 clients, the same LRU-bound pattern applied to the rate-limit stores above. The Redis-backed path and get_recent_event_count are unchanged (GHSA-g53w-gmp9-9ch3).
  • PerformanceMonitor's statistical-anomaly check no longer uses exact-Fraction arithmetic. detect_statistical_anomaly called statistics.mean/statistics.stdev on every record_metric once a pattern's window warmed to min_samples_for_anomaly, an unbounded per-value CPU tax on all enhanced-mode traffic (CWE-400); replaced with math.fsum-based float mean/sample-variance, producing the same anomaly decisions. Sync mirror regenerated.
  • IPInfo token and Redis password no longer reach telemetry. IPInfoManager._download_database sends the IPInfo token as an Authorization: Bearer header instead of a ?token= query-string parameter, and _send_geo_event's reason no longer embeds a raw exception's str() (which could carry the request URL via aiohttp.ClientResponseError/requests.HTTPError); reduced to the exception class name plus HTTP status when available. RedisManager.initialize strips userinfo from redis_url before it reaches the redis_connection/redis_error agent events, keeping only scheme://host[:port]. Async and sync mirrors updated identically (issue #80).
  • Custom security-header names are now validated, including Redis-loaded ones. SecurityHeadersConfigMixin._validate_header_name enforces the RFC 9110 token grammar and rejects a name containing CRLF or any other non-token character, wired into _add_custom_headers. SecurityHeadersCacheMixin._load_cached_config now runs every cached custom_headers name and value through the same two validators and discards the entry (logs a warning, keeps the prior configuration) instead of applying an invalid cached header. Async and sync mirrors updated identically (issue #81).
  • ssrf detects the IPv4-mapped IPv6 loopback bracket form and a trailing-dot localhost. http://[::ffff:127.0.0.1]/ and http://localhost./ now match the ssrf category. http://[::ffff:8.8.8.8]/ (a public address in the same bracket shape) and the existing localhost.example.com/notlocalhost.io lookalikes stay unflagged. Sync mirror updated in lockstep (issue #81).
  • guard_core.handlers.behavior_handler and guard_core.sync.handlers.behavior_handler can now be imported standalone. The behavior-tracker bound-store fix above pulled _lru_pop_or_create in from guard_core.core.checks.helpers, which imports guard_core.core, whose package init reaches back through guard_core.decorators.behavioral to BehaviorRule in the still-initializing behavior_handler module; a fresh interpreter raised ImportError: cannot import name 'BehaviorRule' from partially initialized module 'guard_core.handlers.behavior_handler' (most likely due to a circular import), breaking import guard in fastapi-guard. _lru_pop_or_create now lives in a dependency-free leaf module, guard_core._utils.lru_store; helpers.py re-exports it and behavior_handler.py/ratelimit_handler.py import it directly, so neither pulls in guard_core.core at module load. Sync mirror regenerated; the hand-maintained guard_core/sync/handlers/ratelimit_handler.py updated to match.
  • RedisManager.initialize no longer abandons a live client when re-initialising. Every re-initialised manager (each middleware built in tests, any app that constructs the manager twice) previously overwrote self._redis without closing the client already there, leaking a redis-py connection pool that GC later finalised with ResourceWarning: unclosed Connection; the enable_redis=False branch and the connection-failure branch dropped their client the same way. initialize and close now route through a shared _discard_client that closes the existing client before it is replaced or dropped, so the manager owns at most one live client at a time; a close failure is logged and swallowed rather than blocking re-initialisation. Sync mirror regenerated.
  • PerformanceMonitor.record_metric no longer runs its statistical-anomaly check outside the lock. _check_anomalies iterated a pattern's recent_times deque (math.fsum plus a generator) after the lock guarding appends to that same deque had already been released; a concurrent record_metric call under real OS threads (the sync tree, e.g. gunicorn --threads) could append mid-iteration and raise RuntimeError: deque mutated during iteration, dropping detection to the fallback scan. The statistical-anomaly check now runs inside the same lock as the append that feeds it; anomaly emission (agent send, cooldown) stays outside the lock. Sync mirror regenerated (PR #82).
  • The enhanced-detection exception fallback no longer runs an unbounded regex scan. _fallback_pattern_check called pattern.search() directly on every compiled pattern with no scan-window bound and no timeout, so any exception in the enhanced path (including the PerformanceMonitor race above) silently re-exposed the quadratic built-in patterns GHSA-r7hm-rjvg-xx7j bounded. The fallback now dispatches each pattern through SusPatternsManager._check_regex_pattern, the same bounded matchers (scan-window finders and the executor-timeout path) the enhanced scan uses, so a RecursionError on one pattern is still logged and skipped but every scan stays bounded. Async and sync mirrors updated identically (GHSA-r7hm-rjvg-xx7j).
  • PerformanceMonitor.get_summary_stats/get_slow_patterns/get_problematic_patterns no longer iterate recent_metrics/pattern_stats outside the lock. get_summary_stats iterated the recent_metrics deque three times, and get_slow_patterns/get_problematic_patterns iterated the pattern_stats dict, both unlocked while record_metric appends to recent_metrics and inserts/evicts pattern_stats keys under the lock; reachable through the public SusPatternsManager.get_performance_stats(), this raised the same RuntimeError: deque mutated during iteration class as the record_metric race above (and, for pattern_stats, RuntimeError: dictionary changed size during iteration). All three now snapshot the collection they read under self._lock before handing it to the (unlocked, pure) reporting helpers; get_pattern_report was audited and left unlocked, since it only does a single dict lookup plus scalar reads, never an iteration. Async and sync mirrors updated identically.

What's Changed

  • release(v3.14.0): post-3.13.0 hardening by @rennf93 in #82

Full Changelog: 3.13.0...3.14.0