feat(store): rustls for production PostgreSQL sslmode=require - #100
Merged
seonghobae merged 7 commits intoAug 23, 2026
Merged
Conversation
Issue #80 remainder. sslmode=require/verify-ca/verify-full connect with rustls and Mozilla roots; certificates are always verified. allow/prefer are still rejected so the process cannot silently drop to plaintext. Live test against plaintext CI postgres proves fail-closed. Do not re-implement the postgres gate or the outbox.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Still-valid #100 Devin finding. tokio-postgres 0.7 only parses disable/prefer/require. Map verify-ca/verify-full to require before connect; rustls still verifies certificates. Password query-lookalikes are left untouched.
Comment on lines
+323
to
+328
| fn rustls_connector() -> Result<MakeRustlsConnect, String> { | ||
| rustls::crypto::ring::default_provider() | ||
| .install_default() | ||
| .ok(); | ||
| Ok(MakeRustlsConnect::with_webpki_roots()) | ||
| } |
There was a problem hiding this comment.
🔍 require/verify TLS trusts only Mozilla roots, no custom CA
rustls_connector() hardcodes with_webpki_roots(), so require/verify-ca/verify-full validate the server certificate against the Mozilla root store only, with no sslrootcert or custom-CA option. PostgreSQL servers using a private CA or a cloud-provider CA (Amazon RDS, GCP Cloud SQL) chain to roots outside that set, so sslmode=require fails to connect there. This is the documented intent, but it narrows the production deployments this feature can reach.
Was this helpful? React with 👍 or 👎 to provide feedback.
4 tasks
* feat(store): bound outbox listing and prune processed rows Still-valid #99 finding. GET /api/outbox returns at most EVENT_LIMIT rows (dead letters, then pending, then leased, then processed). Processed outbox_message rows prune to that cap; receipts and dead letters stay. Do not re-implement the outbox, postgres gate, or rustls. * docs: record PR #101 in the product-technical gap baseline * fix(store): prune processed outbox to EVENT_LIMIT on save and ack Still-valid #101 Devin finding. Snapshot save and worker ack used LIST_LIMIT (1000) while append used operator EVENT_LIMIT. Store the configured cap on PostgresPlane so all three paths retain the same processed-row bound. Receipts and dead letters stay.
* feat(store): logical backup and isolated restore drill Issue #80 remainder stacked on #101. GET /api/backup exports a hashed tenant snapshot; POST /api/backup restores after schema and payload-hash checks; POST /api/backup/drill restores into an isolated tenant, compares unmasked invariants, and drops the drill rows. Declared RPO is last successful export; declared RTO is 60s. File/memory adapters report backup=disabled. Do not re-implement rustls, outbox, or retention. * docs: record PR #102 in the product-technical gap baseline
Comment on lines
+279
to
+327
| fn ssl_mode(raw: &str) -> Result<SslMode, String> { | ||
| let lower = raw.to_ascii_lowercase(); | ||
| let Some((_, query)) = lower.split_once('?') else { | ||
| return Ok(SslMode::Disable); | ||
| }; | ||
| for part in query.split('&').flat_map(|chunk| chunk.split('#')) { | ||
| let Some((key, value)) = part.split_once('=') else { | ||
| continue; | ||
| }; | ||
| if key != "sslmode" { | ||
| continue; | ||
| } | ||
| return match value { | ||
| "disable" => Ok(SslMode::Disable), | ||
| "require" | "verify-ca" | "verify-full" => Ok(SslMode::Require), | ||
| other => Err(format!( | ||
| "unsupported sslmode {other}; use disable or require/verify-full" | ||
| )), | ||
| }; | ||
| } | ||
| Ok(SslMode::Disable) | ||
| } | ||
|
|
||
| /// tokio-postgres 0.7 only parses `disable` / `prefer` / `require`. Map the | ||
| /// libpq verification modes we already treat as `Require` so rustls can | ||
| /// still verify certificates. | ||
| fn rewrite_sslmode_for_tokio(raw: &str) -> String { | ||
| let Some((head, query)) = raw.split_once('?') else { | ||
| return raw.to_string(); | ||
| }; | ||
| let rewritten = query | ||
| .split('&') | ||
| .map(|part| { | ||
| let Some((key, value)) = part.split_once('=') else { | ||
| return part.to_string(); | ||
| }; | ||
| if key.eq_ignore_ascii_case("sslmode") | ||
| && (value.eq_ignore_ascii_case("verify-ca") | ||
| || value.eq_ignore_ascii_case("verify-full")) | ||
| { | ||
| format!("{key}=require") | ||
| } else { | ||
| part.to_string() | ||
| } | ||
| }) | ||
| .collect::<Vec<_>>() | ||
| .join("&"); | ||
| format!("{head}?{rewritten}") | ||
| } |
There was a problem hiding this comment.
📝 Info: Hand-rolled sslmode URL parsing
ssl_mode and rewrite_sslmode_for_tokio in control_plane.rs scan the URL with split_once('?')/split('&') rather than a URL parser. A raw ? in a password or a # fragment can hide the sslmode key (defaulting to plaintext Disable) or leave verify-full unrewritten. Standard URLs percent-encode these characters, so impact is limited, but the two functions also disagree on fragment handling.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+499
to
+530
| pub async fn restore_drill(&self) -> Result<BackupDrillReport, String> { | ||
| let started = Instant::now(); | ||
| let backup = self.logical_backup().await?; | ||
| let isolated = format!("restore-drill-{}-{}", std::process::id(), unix_now_i64()); | ||
| let mut client = self.client.lock().await; | ||
| restore_backup(&mut client, &isolated, &backup).await?; | ||
| let restored = export_backup(&mut client, &isolated).await?; | ||
| drop_tenant(&mut client, &isolated).await?; | ||
| let source_hash = backup.semantic_hash()?; | ||
| let restored_hash = restored.semantic_hash()?; | ||
| let passed = source_hash == restored_hash | ||
| && restored.snapshot.routes == backup.snapshot.routes | ||
| && restored.snapshot.events == backup.snapshot.events | ||
| && restored.snapshot.threats == backup.snapshot.threats | ||
| && restored.snapshot.dnsbl == backup.snapshot.dnsbl | ||
| && restored.outbox.len() == backup.outbox.len() | ||
| && restored.receipts.len() == backup.receipts.len(); | ||
| let duration_ms = u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX); | ||
| Ok(BackupDrillReport { | ||
| passed, | ||
| duration_ms, | ||
| rpo: BACKUP_RPO.to_string(), | ||
| rto_budget_ms: BACKUP_RTO_BUDGET_MS, | ||
| source_hash, | ||
| restored_hash, | ||
| route_count: backup.snapshot.routes.len(), | ||
| event_count: backup.snapshot.events.len(), | ||
| outbox_count: backup.outbox.len(), | ||
| receipt_count: backup.receipts.len(), | ||
| isolated_tenant_id: isolated, | ||
| }) | ||
| } |
There was a problem hiding this comment.
📝 Info: Backup and drill block all other DB work
PostgresPlane serializes every operation on one Client behind a Mutex. restore_drill (control_plane.rs) holds that lock across restore, re-export, and drop, and logical_backup holds it across a full snapshot read. On a large tenant this blocks request-path event recording for the whole operation. Bounded today by the small default EVENT_LIMIT.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+1678
to
+1694
| async fn export_backup(client: &mut Client, tenant_id: &str) -> Result<ControlPlaneBackup, String> { | ||
| let snapshot = load_snapshot(client, tenant_id) | ||
| .await? | ||
| .ok_or_else(|| format!("tenant {tenant_id} has no snapshot to back up"))?; | ||
| let outbox = list_outbox(client, tenant_id, i64::MAX).await?; | ||
| let receipts = list_receipts(client, tenant_id).await?; | ||
| ControlPlaneBackup { | ||
| schema_version: MIGRATION_VERSION, | ||
| tenant_id: tenant_id.to_string(), | ||
| created_unix: unix_now_i64(), | ||
| snapshot, | ||
| outbox, | ||
| receipts, | ||
| payload_hash: String::new(), | ||
| } | ||
| .seal() | ||
| } |
There was a problem hiding this comment.
📝 Info: export_backup spans three transactions
export_backup at control_plane.rs reads snapshot, outbox, and receipts in three separate transactions, so a concurrent worker ack between them can yield a slightly inconsistent artifact. Acceptable for an on-demand logical backup; the drill compares hashes of one exported artifact, so its pass/fail logic is unaffected.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+1280
to
+1300
| let actor = audit_actor(&state, &headers); | ||
| match state | ||
| .mutate_and_persist(|data| { | ||
| record_successful_audit_log( | ||
| data, | ||
| actor, | ||
| "restore_backup", | ||
| "control_plane_backup", | ||
| backup.payload_hash.clone(), | ||
| ); | ||
| }) | ||
| .await | ||
| { | ||
| Ok(_) => Json(serde_json::json!({ | ||
| "status": "restored", | ||
| "schema_version": backup.schema_version, | ||
| "payload_hash": backup.payload_hash, | ||
| })) | ||
| .into_response(), | ||
| Err(message) => error(StatusCode::INTERNAL_SERVER_ERROR, message), | ||
| } |
There was a problem hiding this comment.
📝 Info: Restore performs a second full snapshot write
In restore_backup (lib.rs), after the restore commits and reloads, mutate_and_persist runs plane.save() to record the audit entry, rewriting every snapshot table and enqueuing another policy.snapshot_replaced outbox row. Restored outbox/receipt rows survive, so behavior is correct, but restore does the full snapshot write twice.
Was this helpful? React with 👍 or 👎 to provide feedback.
This was referenced Aug 23, 2026
* feat(store): non-owner PostgreSQL runtime role after migrate Still-valid #98 finding. CI connects as a superuser, which bypasses FORCE RLS. Migrations stay on the login role, then SET ROLE wardnet_runtime (NOSUPERUSER, NOBYPASSRLS, not table owner). Missing tenant GUC yields no rows; DROP TABLE and DISABLE RLS are denied. Do not re-implement rustls, outbox, retention, or backup/restore. * docs: record PR #103 in the product-technical gap baseline * fix(store): restore logical backups across role-only schema versions v3 only provisions wardnet_runtime and does not change table shape. verify() accepts schema 2 through the current migration version so a role-only upgrade cannot void the last pre-upgrade logical backup.
* feat(store): HASH-partition security_event by tenant Convert unpartitioned security_event to PARTITION BY HASH (tenant_id) with eight children under pg_advisory_lock. Rows keep unmasked client IPs and paths. /healthz.event_partitions reports the child count. Logical restore still accepts schema 2 through the current version. * docs: record PR #104 in the product-technical gap baseline
seonghobae
merged commit Aug 23, 2026
02ca9b9
into
feat/issue-81-outbox-workers
6 of 7 checks passed
Contributor
Author
|
Stack update this hour: #103 (runtime role + restorable schema 2..=current) and #104 (HASH-partition |
Comment on lines
+1271
to
+1285
| if let Err(message) = plane.restore_logical_backup(&backup).await { | ||
| return error(StatusCode::BAD_REQUEST, message); | ||
| } | ||
| match plane.load().await { | ||
| Ok(Some(loaded)) => { | ||
| *state.inner.write().await = loaded; | ||
| } | ||
| Ok(None) => { | ||
| return error( | ||
| StatusCode::INTERNAL_SERVER_ERROR, | ||
| "restore committed but tenant snapshot is empty", | ||
| ); | ||
| } | ||
| Err(message) => return error(StatusCode::INTERNAL_SERVER_ERROR, message), | ||
| } |
There was a problem hiding this comment.
🟡 Restore swaps state without the persistence lock
The restore path rewrites PostgreSQL and replaces the in-memory snapshot (state.inner.write()) without holding persist_lock, the lock that serializes every other mutation. A gateway event recorded at the same instant can overwrite the restored data or be dropped, leaving memory and the database out of sync.
Prompt for agents
The restore_backup handler in src/lib.rs performs two unsynchronized steps: it calls plane.restore_logical_backup(&backup) to rewrite the database, then does *state.inner.write().await = loaded to replace the in-memory AppData. Neither step holds state.persist_lock, which every other mutation path (record_event, mutate_and_persist) acquires to keep the in-memory snapshot and the persisted store consistent. Under concurrent gateway traffic a record_event can interleave with the restore and either clobber the restored rows (its append writes a fresh event and bumps event_sequence) or have its in-memory event dropped by the restore's snapshot swap, leaving memory and PostgreSQL divergent. Consider performing the database restore, the plane.load(), and the in-memory replacement while holding persist_lock (or routing the whole restore through a mechanism analogous to mutate_and_persist) so the restore is atomic with respect to concurrent writes.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+384
to
+386
| if let Ok(count) = plane.event_partition_count().await { | ||
| health.event_partitions = count; | ||
| } |
There was a problem hiding this comment.
📝 Info: Partition-count failure reported as zero
health_status_live only overwrites event_partitions on success (src/lib.rs:384-386), leaving the default 0 on query failure. Unlike the outbox branch which surfaces error, a failing partition-count query on PostgreSQL makes /healthz report event_partitions: 0, indistinguishable from file/memory mode. Minor observability gap.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
1917
to
+1926
| FROM outbox_message WHERE tenant_id = $1 | ||
| ORDER BY created_unix, aggregate_id, aggregate_version", | ||
| &[&tenant_id], | ||
| ORDER BY CASE message_status | ||
| WHEN 'dead_letter' THEN 0 | ||
| WHEN 'pending' THEN 1 | ||
| WHEN 'leased' THEN 2 | ||
| ELSE 3 | ||
| END, | ||
| created_unix DESC, message_id DESC | ||
| LIMIT $2", | ||
| &[&tenant_id, &limit], |
There was a problem hiding this comment.
📝 Info: Bounded outbox list can omit dead letters
list_outbox applies LIMIT=EVENT_LIMIT ordered dead_letter-first. Dead letters are never pruned and can exceed EVENT_LIMIT, so GET /api/outbox can omit some when many accumulate. The health endpoint still reports the full dead_letter count and the docs state the list is intentionally bounded, so impact is limited to the listing view.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+479
to
+531
| async fn assume_runtime_role(client: &Client) -> Result<(), String> { | ||
| let current: String = client | ||
| .query_one("SELECT current_user", &[]) | ||
| .await | ||
| .map_err(|error| format!("control plane current_user failed: {error}"))? | ||
| .get(0); | ||
| if current != RUNTIME_ROLE { | ||
| client | ||
| .batch_execute( | ||
| "DO $$ | ||
| BEGIN | ||
| IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'wardnet_runtime') THEN | ||
| CREATE ROLE wardnet_runtime | ||
| NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT NOLOGIN | ||
| NOBYPASSRLS NOREPLICATION; | ||
| END IF; | ||
| END | ||
| $$; | ||
| GRANT wardnet_runtime TO CURRENT_USER; | ||
| GRANT USAGE ON SCHEMA public TO wardnet_runtime; | ||
| REVOKE CREATE ON SCHEMA public FROM wardnet_runtime; | ||
| GRANT SELECT, INSERT, UPDATE, DELETE ON | ||
| tenant_account, tenant_profile, route_config, threat_indicator, | ||
| dnsbl_entry, security_event, audit_record, threat_feed, | ||
| outbox_message, outbox_receipt TO wardnet_runtime; | ||
| GRANT SELECT ON schema_migration TO wardnet_runtime; | ||
| SET ROLE wardnet_runtime;", | ||
| ) | ||
| .await | ||
| .map_err(|error| { | ||
| format!( | ||
| "control plane runtime role {RUNTIME_ROLE} failed (provision NOSUPERUSER NOBYPASSRLS and GRANT it to the login role): {error}" | ||
| ) | ||
| })?; | ||
| } | ||
| let row = client | ||
| .query_one("SELECT current_user, current_setting('is_superuser')", &[]) | ||
| .await | ||
| .map_err(|error| format!("control plane runtime identity failed: {error}"))?; | ||
| let user: String = row.get(0); | ||
| let superuser: String = row.get(1); | ||
| if user != RUNTIME_ROLE { | ||
| return Err(format!( | ||
| "control plane must run as {RUNTIME_ROLE}, current_user is {user}" | ||
| )); | ||
| } | ||
| if superuser == "on" { | ||
| return Err(format!( | ||
| "control plane role {RUNTIME_ROLE} must not be a superuser" | ||
| )); | ||
| } | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
📝 Info: Runtime role switch needs CREATE ROLE / GRANT on the login user
assume_runtime_role provisions wardnet_runtime, GRANTs it to the login user, and SET ROLEs to it, verifying non-superuser afterward. This requires the URL login user to hold CREATE ROLE / GRANT on first connect; otherwise startup fails with a descriptive error. Operators must pre-provision the role and grant it when the login user cannot, as documented.
Was this helpful? React with 👍 or 👎 to provide feedback.
Comment on lines
+1229
to
+1252
| async fn get_backup(State(state): State<AppState>, headers: HeaderMap) -> Response { | ||
| if !admin_authenticated(&state, &headers) { | ||
| return error(StatusCode::UNAUTHORIZED, "missing or invalid X-Admin-Token"); | ||
| } | ||
| let Some(plane) = &state.control_plane else { | ||
| return Json(BackupView { | ||
| status: "disabled".to_string(), | ||
| rpo: control_plane::BACKUP_RPO.to_string(), | ||
| rto_budget_ms: control_plane::BACKUP_RTO_BUDGET_MS, | ||
| artifact: None, | ||
| }) | ||
| .into_response(); | ||
| }; | ||
| match plane.logical_backup().await { | ||
| Ok(artifact) => Json(BackupView { | ||
| status: "ready".to_string(), | ||
| rpo: control_plane::BACKUP_RPO.to_string(), | ||
| rto_budget_ms: control_plane::BACKUP_RTO_BUDGET_MS, | ||
| artifact: Some(artifact), | ||
| }) | ||
| .into_response(), | ||
| Err(message) => error(StatusCode::INTERNAL_SERVER_ERROR, message), | ||
| } | ||
| } |
There was a problem hiding this comment.
🟨 Read-only tokens can export the full backup
The new backup export authorizes with admin_authenticated, which accepts read-only tokens documented as audit-log-read-only. It returns the whole tenant: routes, threats, DNSBL, license metadata, every security event, and outbox payloads. A read-only principal can exfiltrate the entire control-plane state. The artifact holds no admin tokens or database URL.
Was this helpful? React with 👍 or 👎 to provide feedback.
seonghobae
added a commit
that referenced
this pull request
Aug 26, 2026
* feat(store): require PostgreSQL as the production control plane Non-loopback binds fail closed without CONTROL_PLANE_DATABASE_URL. Migrations create 3NF two-word tables with default-deny RLS. Snapshot persist commits policy rows and audit records in one transaction. Loopback still uses the JSON file or memory adapter. * feat(store): transactional outbox and leased workers Issue #81 first slice on the PostgreSQL control plane. Security events append with an outbox row in one transaction instead of rewriting the snapshot. Workers claim with SKIP LOCKED, retry, dead-letter, and record unique receipts. Stdout SIEM is at-least-once; the receipt is the exactly-once ack. Also deterministic ORDER BY on postgres loads (still-valid #98 finding). Do not re-implement the postgres gate. * feat(store): rustls for production PostgreSQL sslmode=require (#100) * feat(store): rustls for production PostgreSQL sslmode=require Issue #80 remainder. sslmode=require/verify-ca/verify-full connect with rustls and Mozilla roots; certificates are always verified. allow/prefer are still rejected so the process cannot silently drop to plaintext. Live test against plaintext CI postgres proves fail-closed. Do not re-implement the postgres gate or the outbox. * docs: record PR #100 in the product-technical gap baseline * fix(store): rewrite verify-full sslmode for tokio-postgres 0.7 Still-valid #100 Devin finding. tokio-postgres 0.7 only parses disable/prefer/require. Map verify-ca/verify-full to require before connect; rustls still verifies certificates. Password query-lookalikes are left untouched. * feat(store): bound outbox listing and prune processed rows (#101) * feat(store): bound outbox listing and prune processed rows Still-valid #99 finding. GET /api/outbox returns at most EVENT_LIMIT rows (dead letters, then pending, then leased, then processed). Processed outbox_message rows prune to that cap; receipts and dead letters stay. Do not re-implement the outbox, postgres gate, or rustls. * docs: record PR #101 in the product-technical gap baseline * fix(store): prune processed outbox to EVENT_LIMIT on save and ack Still-valid #101 Devin finding. Snapshot save and worker ack used LIST_LIMIT (1000) while append used operator EVENT_LIMIT. Store the configured cap on PostgresPlane so all three paths retain the same processed-row bound. Receipts and dead letters stay. * feat(store): logical backup and isolated restore drill (#102) * feat(store): logical backup and isolated restore drill Issue #80 remainder stacked on #101. GET /api/backup exports a hashed tenant snapshot; POST /api/backup restores after schema and payload-hash checks; POST /api/backup/drill restores into an isolated tenant, compares unmasked invariants, and drops the drill rows. Declared RPO is last successful export; declared RTO is 60s. File/memory adapters report backup=disabled. Do not re-implement rustls, outbox, or retention. * docs: record PR #102 in the product-technical gap baseline * feat(store): non-owner PostgreSQL runtime role after migrate (#103) * feat(store): non-owner PostgreSQL runtime role after migrate Still-valid #98 finding. CI connects as a superuser, which bypasses FORCE RLS. Migrations stay on the login role, then SET ROLE wardnet_runtime (NOSUPERUSER, NOBYPASSRLS, not table owner). Missing tenant GUC yields no rows; DROP TABLE and DISABLE RLS are denied. Do not re-implement rustls, outbox, retention, or backup/restore. * docs: record PR #103 in the product-technical gap baseline * fix(store): restore logical backups across role-only schema versions v3 only provisions wardnet_runtime and does not change table shape. verify() accepts schema 2 through the current migration version so a role-only upgrade cannot void the last pre-upgrade logical backup. * feat(store): HASH-partition security_event by tenant (#104) * feat(store): HASH-partition security_event by tenant Convert unpartitioned security_event to PARTITION BY HASH (tenant_id) with eight children under pg_advisory_lock. Rows keep unmasked client IPs and paths. /healthz.event_partitions reports the child count. Logical restore still accepts schema 2 through the current version. * docs: record PR #104 in the product-technical gap baseline
seonghobae
added a commit
that referenced
this pull request
Aug 26, 2026
* feat(security): fail-closed destination policy for outbound HTTP One DestinationPolicy mediates gateway upstreams, threat-intel fetches, Clearfolio, SOC LLM, and the Coraza sidecar URL. Private, loopback, link-local, CGNAT, and metadata classes are denied unless DESTINATION_ALLOWLIST (or loopback development) permits them; DESTINATION_DENYLIST wins. Clients ignore ambient HTTP proxies and do not follow redirects. Refs #79. * docs: record PR #96 in the product-technical gap baseline * feat(security): harden destination policy per-IP CIDR and readiness order CIDR allowlist matches apply per resolved address, authorize non-default ports, and reject prefixes outside the address-family width. IPv6 site-local is a denied class. Hostnames that merely contain 0x are not hex IP literals. AppState constructors default to production policy; seeded fixtures opt into development. Blocking DNS runs on spawn_blocking with a timeout. Persistence and destination-list validation complete before the readiness line. * feat(security): pin outbound HTTP to evaluated destination addresses After destination policy allows a host, the reqwest client resolves only those IPs so a rebinding answer cannot reach loopback, private, or metadata classes. Host and SNI stay on the original name. Unpinned hostnames fail closed instead of falling back to OS DNS. * feat(waf): evaluate live gateway transactions with in-process libcoraza Issue #86 remainder: dlopen operator-supplied libcoraza and drive the C ABI on each /gateway request. Missing library or empty ruleset fail closed before bind. CI stays hermetic with a fixture cdylib that exports the same symbols. * docs: record PR #97 in the product-technical gap baseline * feat(waf): forward bounded client headers into in-process libcoraza In-process transactions now receive the same forwarded-header allowlist as the sidecar path (host, user-agent, accept, content-type, referer, origin, x-requested-with, x-forwarded-for, x-real-ip, cookie — never Authorization; 32 headers / 8 KiB caps enforced by proven_engine::engine_forwarded_headers). Each header crosses the C ABI via coraza_add_request_header before process_request_headers, so CRS rules that inspect headers evaluate real client input instead of a synthetic Host only. Brings in the PR #95 sidecar hardening via merge so both engines share one allowlist implementation and one status/bound contract. Behavioral header-battery evidence lands with the issue-11 battery fixture (PR #110); this slice ships the plumbing and keeps the stub contract unchanged. * feat(store): require PostgreSQL as the production control plane (#98) * feat(store): require PostgreSQL as the production control plane Non-loopback binds fail closed without CONTROL_PLANE_DATABASE_URL. Migrations create 3NF two-word tables with default-deny RLS. Snapshot persist commits policy rows and audit records in one transaction. Loopback still uses the JSON file or memory adapter. * feat(store): transactional outbox and leased workers Issue #81 first slice on the PostgreSQL control plane. Security events append with an outbox row in one transaction instead of rewriting the snapshot. Workers claim with SKIP LOCKED, retry, dead-letter, and record unique receipts. Stdout SIEM is at-least-once; the receipt is the exactly-once ack. Also deterministic ORDER BY on postgres loads (still-valid #98 finding). Do not re-implement the postgres gate. * feat(store): rustls for production PostgreSQL sslmode=require (#100) * feat(store): rustls for production PostgreSQL sslmode=require Issue #80 remainder. sslmode=require/verify-ca/verify-full connect with rustls and Mozilla roots; certificates are always verified. allow/prefer are still rejected so the process cannot silently drop to plaintext. Live test against plaintext CI postgres proves fail-closed. Do not re-implement the postgres gate or the outbox. * docs: record PR #100 in the product-technical gap baseline * fix(store): rewrite verify-full sslmode for tokio-postgres 0.7 Still-valid #100 Devin finding. tokio-postgres 0.7 only parses disable/prefer/require. Map verify-ca/verify-full to require before connect; rustls still verifies certificates. Password query-lookalikes are left untouched. * feat(store): bound outbox listing and prune processed rows (#101) * feat(store): bound outbox listing and prune processed rows Still-valid #99 finding. GET /api/outbox returns at most EVENT_LIMIT rows (dead letters, then pending, then leased, then processed). Processed outbox_message rows prune to that cap; receipts and dead letters stay. Do not re-implement the outbox, postgres gate, or rustls. * docs: record PR #101 in the product-technical gap baseline * fix(store): prune processed outbox to EVENT_LIMIT on save and ack Still-valid #101 Devin finding. Snapshot save and worker ack used LIST_LIMIT (1000) while append used operator EVENT_LIMIT. Store the configured cap on PostgresPlane so all three paths retain the same processed-row bound. Receipts and dead letters stay. * feat(store): logical backup and isolated restore drill (#102) * feat(store): logical backup and isolated restore drill Issue #80 remainder stacked on #101. GET /api/backup exports a hashed tenant snapshot; POST /api/backup restores after schema and payload-hash checks; POST /api/backup/drill restores into an isolated tenant, compares unmasked invariants, and drops the drill rows. Declared RPO is last successful export; declared RTO is 60s. File/memory adapters report backup=disabled. Do not re-implement rustls, outbox, or retention. * docs: record PR #102 in the product-technical gap baseline * feat(store): non-owner PostgreSQL runtime role after migrate (#103) * feat(store): non-owner PostgreSQL runtime role after migrate Still-valid #98 finding. CI connects as a superuser, which bypasses FORCE RLS. Migrations stay on the login role, then SET ROLE wardnet_runtime (NOSUPERUSER, NOBYPASSRLS, not table owner). Missing tenant GUC yields no rows; DROP TABLE and DISABLE RLS are denied. Do not re-implement rustls, outbox, retention, or backup/restore. * docs: record PR #103 in the product-technical gap baseline * fix(store): restore logical backups across role-only schema versions v3 only provisions wardnet_runtime and does not change table shape. verify() accepts schema 2 through the current migration version so a role-only upgrade cannot void the last pre-upgrade logical backup. * feat(store): HASH-partition security_event by tenant (#104) * feat(store): HASH-partition security_event by tenant Convert unpartitioned security_event to PARTITION BY HASH (tenant_id) with eight children under pg_advisory_lock. Rows keep unmasked client IPs and paths. /healthz.event_partitions reports the child count. Logical restore still accepts schema 2 through the current version. * docs: record PR #104 in the product-technical gap baseline * feat(security): add bounded outbound fetch API (#113) * feat(security): add bounded outbound fetch API * fix(security): isolate fetch DNS pins per request * feat(security): route browser DNS and HTTPS through Wardnet (#116) * feat(security): add bounded outbound fetch API * feat(security): route browser DNS and HTTPS through Wardnet * fix(security): isolate fetch DNS pins per request * fix(dns): bound concurrent UDP query handling * fix(security): close destination policy review gaps * fix(runtime): make worker shutdown durable
seonghobae
added a commit
that referenced
this pull request
Aug 26, 2026
* feat(security): fail-closed destination policy for outbound HTTP One DestinationPolicy mediates gateway upstreams, threat-intel fetches, Clearfolio, SOC LLM, and the Coraza sidecar URL. Private, loopback, link-local, CGNAT, and metadata classes are denied unless DESTINATION_ALLOWLIST (or loopback development) permits them; DESTINATION_DENYLIST wins. Clients ignore ambient HTTP proxies and do not follow redirects. Refs #79. * docs: record PR #96 in the product-technical gap baseline * feat(security): harden destination policy per-IP CIDR and readiness order CIDR allowlist matches apply per resolved address, authorize non-default ports, and reject prefixes outside the address-family width. IPv6 site-local is a denied class. Hostnames that merely contain 0x are not hex IP literals. AppState constructors default to production policy; seeded fixtures opt into development. Blocking DNS runs on spawn_blocking with a timeout. Persistence and destination-list validation complete before the readiness line. * feat(security): pin outbound HTTP to evaluated destination addresses After destination policy allows a host, the reqwest client resolves only those IPs so a rebinding answer cannot reach loopback, private, or metadata classes. Host and SNI stay on the original name. Unpinned hostnames fail closed instead of falling back to OS DNS. * feat(waf): evaluate live gateway transactions with in-process libcoraza Issue #86 remainder: dlopen operator-supplied libcoraza and drive the C ABI on each /gateway request. Missing library or empty ruleset fail closed before bind. CI stays hermetic with a fixture cdylib that exports the same symbols. * docs: record PR #97 in the product-technical gap baseline * feat(store): require PostgreSQL as the production control plane Non-loopback binds fail closed without CONTROL_PLANE_DATABASE_URL. Migrations create 3NF two-word tables with default-deny RLS. Snapshot persist commits policy rows and audit records in one transaction. Loopback still uses the JSON file or memory adapter. * feat(store): transactional outbox and leased workers Issue #81 first slice on the PostgreSQL control plane. Security events append with an outbox row in one transaction instead of rewriting the snapshot. Workers claim with SKIP LOCKED, retry, dead-letter, and record unique receipts. Stdout SIEM is at-least-once; the receipt is the exactly-once ack. Also deterministic ORDER BY on postgres loads (still-valid #98 finding). Do not re-implement the postgres gate. * feat(store): rustls for production PostgreSQL sslmode=require (#100) * feat(store): rustls for production PostgreSQL sslmode=require Issue #80 remainder. sslmode=require/verify-ca/verify-full connect with rustls and Mozilla roots; certificates are always verified. allow/prefer are still rejected so the process cannot silently drop to plaintext. Live test against plaintext CI postgres proves fail-closed. Do not re-implement the postgres gate or the outbox. * docs: record PR #100 in the product-technical gap baseline * fix(store): rewrite verify-full sslmode for tokio-postgres 0.7 Still-valid #100 Devin finding. tokio-postgres 0.7 only parses disable/prefer/require. Map verify-ca/verify-full to require before connect; rustls still verifies certificates. Password query-lookalikes are left untouched. * feat(store): bound outbox listing and prune processed rows (#101) * feat(store): bound outbox listing and prune processed rows Still-valid #99 finding. GET /api/outbox returns at most EVENT_LIMIT rows (dead letters, then pending, then leased, then processed). Processed outbox_message rows prune to that cap; receipts and dead letters stay. Do not re-implement the outbox, postgres gate, or rustls. * docs: record PR #101 in the product-technical gap baseline * fix(store): prune processed outbox to EVENT_LIMIT on save and ack Still-valid #101 Devin finding. Snapshot save and worker ack used LIST_LIMIT (1000) while append used operator EVENT_LIMIT. Store the configured cap on PostgresPlane so all three paths retain the same processed-row bound. Receipts and dead letters stay. * feat(store): logical backup and isolated restore drill (#102) * feat(store): logical backup and isolated restore drill Issue #80 remainder stacked on #101. GET /api/backup exports a hashed tenant snapshot; POST /api/backup restores after schema and payload-hash checks; POST /api/backup/drill restores into an isolated tenant, compares unmasked invariants, and drops the drill rows. Declared RPO is last successful export; declared RTO is 60s. File/memory adapters report backup=disabled. Do not re-implement rustls, outbox, or retention. * docs: record PR #102 in the product-technical gap baseline * feat(store): non-owner PostgreSQL runtime role after migrate (#103) * feat(store): non-owner PostgreSQL runtime role after migrate Still-valid #98 finding. CI connects as a superuser, which bypasses FORCE RLS. Migrations stay on the login role, then SET ROLE wardnet_runtime (NOSUPERUSER, NOBYPASSRLS, not table owner). Missing tenant GUC yields no rows; DROP TABLE and DISABLE RLS are denied. Do not re-implement rustls, outbox, retention, or backup/restore. * docs: record PR #103 in the product-technical gap baseline * fix(store): restore logical backups across role-only schema versions v3 only provisions wardnet_runtime and does not change table shape. verify() accepts schema 2 through the current migration version so a role-only upgrade cannot void the last pre-upgrade logical backup. * feat(store): HASH-partition security_event by tenant (#104) * feat(store): HASH-partition security_event by tenant Convert unpartitioned security_event to PARTITION BY HASH (tenant_id) with eight children under pg_advisory_lock. Rows keep unmasked client IPs and paths. /healthz.event_partitions reports the child count. Logical restore still accepts schema 2 through the current version. * docs: record PR #104 in the product-technical gap baseline * feat(store): optimistic concurrency on postgres snapshots Issue #80 last remainder. tenant_account.snapshot_version must match the loaded token or persist fails closed (HTTP 409). Restores overwrite. File/memory stay single-writer. Do not re-implement rustls, outbox, runtime role, HASH, or backup/restore. * docs: record PR #105 in the product-technical gap baseline * fix(store): keep postgres snapshot_version aligned after startup save load_postgres was saving with OCC and leaving the in-memory token one behind the database, so every later management write returned HTTP 409. Advance the loaded snapshot_version to the value save() wrote. * feat(store): outbox consumers for TAXII, Clearfolio, and orchestrator Enqueue operator-triggered TAXII polls, Clearfolio submits, and contextual-orchestrator SOC analysis on the PostgreSQL leased outbox. Request path returns 202; GET /api/outbox/{id} exposes receipt evidence. Secrets stay in the credential registry. Startup postgres save advances snapshot_version so the first management write cannot false-conflict. * docs: record PR #106 in the product-technical gap baseline * feat(release): tagged GitHub Release with SHA-256 and immutable GHCR (#107) * feat(release): tagged GitHub Release with SHA-256 and immutable GHCR Issue #84 first slice. A vX.Y.Z tag builds a locked binary, checksums, a GitHub Release, and ghcr.io/contextualwisdomlab/waf-ids-ai-soc:vX.Y.Z with no moving latest tag. Promotion and rollback are tag-for-tag. Do not re-implement store slices or OCC. * docs: record PR #107 in the product-technical gap baseline * fix(release): basename checksums and serialize postgres GRANTs SHA256SUMS recorded dist/ prefixes so sha256sum -c failed next to the downloaded binary. Emit basenames. Parallel PostgresPlane connects raced HASH convert GRANT with SET ROLE GRANT (tuple concurrently updated); hold the advisory lock across both. Do not re-implement HASH layout. * feat(release): keyless cosign, SPDX SBOM, and SLSA on the same tag Issue #84 remainder. GitHub OIDC signs the binary, checksums, SBOMs, and the GHCR image by digest. Release is created only after signatures. Syft SPDX fails closed without syft or non-SPDX JSON. NIST SP 800-218 is attached. Do not re-implement checksums or store slices. * feat(release): refuse lightweight tags and pin k8s by digest Issue #84 remainder. Annotated vX.Y.Z tags only; lightweight tags fail closed before the release job builds. Kubernetes pin is the GHCR content digest; tag aliases are refused. Do not re-implement checksums or cosign/SBOM. * docs: record PR #109 in the product-technical gap baseline * feat(waf): detect OWASP CRS attack battery on the live binary Issue #11 first slice. The build-script libcoraza ABI stub gains a deterministic battery covering SQLi (942100), XSS (941100), path traversal (930100), Unix RCE (932100, with first-match ordering so '; cat /etc/passwd' attributes to RCE over traversal), and Log4j JNDI (944120) in raw and percent-encoded forms across URI and POST-body phases. tests/binary.rs now starts the real gateway with the stub engine, creates a block route through the admin API, fires nine cases over HTTP, and asserts each is 403-blocked citing the expected CRS rule id while a benign request still forwards; /api/events must record one event per attempt with the forwarded client IP kept unmasked. Doctoring: docs/doctoring/ci-attack-evidence-battery.md grounds the split between detection-path evidence (CI) and detection efficacy (operator-supplied libcoraza + Core Rule Set), APA 7th. * fix(control-plane): close OCC and credential race gaps * fix(release): capture pushed image digest
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Issue #80 remainder. Does not re-implement #78, the #86 sidecar/libcoraza slices, the #79 TCP-peer pin, the #80 production postgres gate, or the #81 outbox.
CONTROL_PLANE_DATABASE_URLwithsslmode=require/verify-ca/verify-fullnow connects with rustls and Mozilla roots. Certificates are always verified (stricter than libpqrequire, which skips verification).sslmode=allow/preferstay rejected so the process cannot silently drop to plaintext. Omitted /disablestays plaintext for loopback CI postgres.Stacked on #99 (
feat/issue-81-outbox-workers). Merge order: #95, then #96, then #97, then #98, then #99, then this PR. Org ruleset 18156473 still requires two independent approvals; do not--adminmerge.Operator-visible
sslmode=requireno longer fail-closes as unwired TLSsslmode=prefer/allowstill fail startup before bind/healthz.persistenceis unchanged (postgres|file|memory)Tests
cargo fmt --checkcargo test --locked --workspace(includes liverequire_tls_fails_closed_against_plaintext_postgresagainst postgres:16)cargo clippy --locked --workspace --all-targets -- -D warningsscripts/smoke.shruns (/healthz+/admin, 2B KRW readiness)Doctoring:
docs/doctoring/postgres-control-plane.md(APA 7th).Remaining on #80: non-owner runtime role, backup/restore drill, HASH partitioning, optimistic concurrency. Remaining on #81: bounded
list_outbox/retention (still-valid #99 Devin thread), additional consumers (TAXII poll, Clearfolio, contextual-orchestrator).