feat(audit): JSONB path filters and NDJSON export (#8)

[cjimti]

First slice of the v1.1.1 follow-up under #8. Lands the query-side improvements that operators wanted before the portal UI rewrite: JSONB-aware filters that hit the existing GIN indexes, and a streaming NDJSON export. Replay, SSE live tail, the inspection drawer, and comparison page land in subsequent PRs on top of this.

What's in this PR

Filter surface (/audit/events, /audit/export)

Operators can now narrow the audit query by values inside the audit_payloads sibling row, not just the indexed summary columns:

?param.<dotted.path>=<value>     -> request_params @> {...}
?response.<dotted.path>=<value>  -> response_result @> {...}
?header.<name>=<value>           -> request_headers @> {name:[value]}
?has=<column>                    -> column IS NOT NULL

• Each filter compiles to an EXISTS (SELECT 1 FROM audit_payloads p WHERE p.event_id = audit_events.id AND p.<col> @> $N::jsonb) subquery, so it hits the existing jsonb_path_ops GIN indexes on request_params and response_result. request_headers is unindexed today; pair header.* with a time range.
• Closed allow-lists (audit.AllowedHasKeys, audit.AllowedJSONSources) gate column and source names. The HTTP layer drops anything outside the lists silently rather than 400-ing, matching how unknown plain query params are handled.
• Values are type-detected before the containment doc is built: true / false become JSON booleans, integer / float strings become numbers, quoted forms force string. So ?response.isError=true matches the JSON literal true, and ?param.code="200" forces the string "200".
• Filters are AND-combined with each other and with the existing indexed-column filters (tool, user, success, from, to, q, etc.).

NDJSON export (GET /api/v1/portal/audit/export?format=jsonl)

• Same filter surface as /audit/events; format=jsonl is the default and only supported format today (csv etc. reserved, returns 400).
• Streams via the new audit.StreamingLogger capability. The Postgres store paginates buildSelect at 1000 rows/page so memory stays bounded; the MemoryLogger iterates a Query result (cheap, but matches the contract); NoopLogger is a no-op.
• Hard cap at 100,000 events per request to bound query cost and response time. Operators wanting more should narrow the filter window or page through /events.
• http.Flusher flushes every 100 lines so a long export shows progress to the consumer immediately.
• Summary rows only; the captured audit_payloads envelope is omitted from each line. Pull full payloads via /audit/events/{id} after filtering.
• Sets Content-Type: application/x-ndjson and Content-Disposition: attachment; filename="audit.jsonl" for browser-driven downloads; curl | jq consumers ignore the disposition.

New audit package surface

• audit.QueryFilter gains JSONFilters []JSONPathFilter and HasKeys []string.
• audit.JSONPathFilter is {Source, Path, Value} where Source is one of "param" / "response" / "header".
• audit.AllowedHasKeys and audit.AllowedJSONSources expose the closed sets so the HTTP layer can validate.
• pkg/audit/jsonfilter.go is the shared compiler used by both backends: ParseJSONFilterValue (type detection), JSONFilterToContainment (dotted path to nested map), MatchJSONPath (memory-side traversal with JSON-semantic equality), SplitJSONPath, IsAllowedHasKey, IsAllowedJSONSource.
• audit.StreamingLogger: optional capability, type-asserted by the export endpoint, with a graceful fallback through AsyncLogger.Stream to a bounded Query.

Drive-by

tests/integration_test.go::TestIntegration_WhoamiOverHTTP was a pre-existing timing flake under -race: it queried Postgres for the audit row before AsyncLogger's background drain had flushed. Surfaced consistently while running the integration suite for this branch. Adds a 2-second poll loop. No related code change; just makes the assertion patient enough for the race-detector slowdown.

Tests

File	Coverage
pkg/audit/jsonfilter_test.go	ParseJSONFilterValue type detection (bool, int, float, forced-string, plain), JSONFilterToContainment (empty path, single, nested), MatchJSONPath (numeric widening, missing path, scalar dead-end), SplitJSONPath, IsAllowedHasKey allow/reject
pkg/audit/memory_test.go	JSONFilters + HasKeys matching against Event.Payload (param path, response bool, header value, has-shortcut, AND combination); Stream returns Query order
pkg/audit/postgres/store_test.go (testcontainer-tagged)	every JSONFilter / HasKey case from the memory test repeated against real Postgres so the GIN-index + EXISTS-subquery path is exercised end-to-end; new 1100-event Stream test that proves the page-loop iterates past the 1000-row internal page boundary
pkg/httpsrv/portal_api_export_test.go	/audit/export?format=jsonl returns one line per event with payload omitted; unknown format gets 400; missing format defaults to jsonl; parseQueryFilter parses the four new syntaxes (param.*, response.*, header.*, has=) and rejects unknown sources / has-keys

Docs

• docs/reference/http-api.md: new rows for /audit/events/{id} and /audit/export. New "JSONB path filters" section: syntax table, type-detection rule, index notes (which filters are GIN-backed, which scan).
• docs/operations/audit.md: new "Two-table layout" section explaining the audit_events / audit_payloads split that landed in v1.1.0; updated REST endpoints table; new "JSONB filters" subsection with curl examples for the four common cases (param path, response shape, header match, has-key); new "NDJSON export" subsection with a curl-piped-through-jq example.

Closes (subtasks of #8)

• JSONB path filter compiler. Compiles to safe parameterized SQL using the existing GIN indexes.
• NDJSON export. Filters mirror /events; streams.

Does not close the umbrella issue. Remaining subtasks (replay endpoint, SSE live tail, portal UI inspection drawer, comparison page, docs/operations/inspection.md walkthrough) land in subsequent PRs per the focused-PR plan.

Verification

• make verify green: gofmt, vet, race-tested unit suite, golangci-lint v2.11.4, gosec v2.25.0, govulncheck, semgrep, coverage gate (80.0%, gate is 80%).
• go test -tags integration -count=1 -timeout 300s ./... green: testcontainers Postgres exercises the JSONB filter compiler and the streaming pagination; the full HTTP stack tests pass.

Test plan

• make verify green
• tagged integration tests green
• Manual: bring up make dev, fire a few progress and echo calls, then:
  • curl -H "X-API-Key: $KEY" "$BASE/api/v1/portal/audit/events?param.message=hello" returns only the matching echo calls
  • curl -H "X-API-Key: $KEY" "$BASE/api/v1/portal/audit/events?response.isError=true" returns only error responses
  • curl -H "X-API-Key: $KEY" "$BASE/api/v1/portal/audit/events?has=notifications" returns only progress-tool calls
  • curl -H "X-API-Key: $KEY" "$BASE/api/v1/portal/audit/export?from=$(date -u -v-1H +%FT%TZ)" | wc -l returns the count of last-hour events
  • curl -H "X-API-Key: $KEY" "$BASE/api/v1/portal/audit/export?format=csv" returns 400 with a clear message