Skip to content

Phase 4: Scheduling, distribution & governance (M1–M4)#17

Merged
kosminus merged 1 commit into
mainfrom
feat/phase4-governance
Jun 8, 2026
Merged

Phase 4: Scheduling, distribution & governance (M1–M4)#17
kosminus merged 1 commit into
mainfrom
feat/phase4-governance

Conversation

@kosminus

@kosminus kosminus commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Implements Phase 4 of the platform roadmap full-stack — turning QueryWise into an enterprise-governed self-service platform. Migrations 009012.

Milestones

M1 — Audit log

  • AuditEvent model; fire-and-forget audit_service.record (inline, SAVEPOINT-isolated, never raises) wired at login, connection CRUD/rotation, and query executed/blocked
  • Admin org-scoped read + CSV export API (/api/v1/audit-events); admin-gated AuditPage

M2 — Scheduled reports

  • Schedule model + schedule_service (cron via croniter [scheduling] extra, threshold alerting, run/deliver pipeline)
  • In-process scheduler loop: atomic FOR UPDATE SKIP LOCKED claim → run_schedule job through the Phase 0 queue (safe for inprocess + arq, multi-replica)
  • Pluggable app/notifications/ adapters — email (SMTP, stdlib), Slack (webhook), log fallback. Magic-link delivery now routed through the notifier (closes the Phase 1 deferral)
  • /schedules CRUD + run-now; SchedulesPage with saved-query/dashboard target picker

M3 — Policy engine

  • DataPolicy model + policy_service: most-restrictive resolve_effective, fail-closed enforce_sql via sqlglot (allow/block tables, blocked columns, row-filter injection), post-exec column masking (star-safe), row/runtime caps
  • Enforced in the query pipeline before the connector on both NL and raw-SQL paths, re-applied across error-handler retries; masking applied in place so no PII reaches the interpreter LLM
  • /connections/{id}/policies CRUD; admin-gated PoliciesPage

M4 — Cost & usage analytics

  • CostAttribution model + QueryResult.stats connector seam (BigQuery scanned-bytes/slot-ms populated); cost_service (pluggable pricing, fire-and-forget capture, aggregations) recorded after each execution
  • Admin /analytics/{usage,cost,slowest,tables} API; admin-gated AnalyticsPage

Exit criteria (all met)

A dashboard can email a team every morning · viewers see PII masked · admins see per-team cost · every important mutation writes an audit event · a blocked query explains why.

Testing

  • +50 backend unit tests (audit, notifications, schedules, policy, cost); 220 pass
  • ruff clean modulo the standard FastAPI B008; frontend tsc + ESLint + build clean
  • CI now installs the [scheduling] extra
  • ⚠️ Migrations 009012 apply on docker compose up (no local Postgres in this run)

Deferred / follow-ups

  • Human-approval gate + certified-metrics-only mode (M3 extras)
  • Role-aware result cache — the saved-query cache isn't yet keyed by effective policy, so masking/row-filters could cross roles via shared cache entries; key by policy before relying on masking there

🤖 Generated with Claude Code

Implements Phase 4 of the platform roadmap full-stack: audit logging,
scheduled reports + notification adapters, a data-policy enforcement
engine, and cost/usage analytics. Migrations 009–012.

M1 — Audit log
- AuditEvent model; fire-and-forget audit_service.record (inline,
  SAVEPOINT-isolated, never raises) wired at login, connection
  CRUD/rotation, and query executed/blocked
- Admin org-scoped read + CSV export API; admin-gated AuditPage

M2 — Scheduled reports
- Schedule model + schedule_service (cron via croniter [scheduling]
  extra, threshold alerting, run/deliver pipeline)
- In-process scheduler loop: atomic FOR UPDATE SKIP LOCKED claim →
  run_schedule job via the Phase 0 queue (works for inprocess + arq)
- Pluggable app/notifications adapters (email SMTP / Slack webhook /
  log fallback); magic-link delivery now routed through the notifier
  (closes the Phase 1 deferral)
- /schedules CRUD + run-now; SchedulesPage with target picker

M3 — Policy engine
- DataPolicy model + policy_service: most-restrictive resolve_effective,
  fail-closed enforce_sql (sqlglot: allow/block tables, blocked columns,
  row-filter injection), post-exec column masking, row/runtime caps
- Enforced in the query pipeline before the connector on both NL and
  raw-SQL paths, re-applied across error-handler retries; masking applied
  in place so no PII reaches the interpreter LLM
- /connections/{id}/policies CRUD; admin-gated PoliciesPage

M4 — Cost & usage analytics
- CostAttribution model + QueryResult.stats connector seam (BigQuery
  scanned-bytes/slot-ms populated); cost_service (pluggable pricing,
  fire-and-forget capture, aggregations) recorded after each execution
- Admin /analytics/{usage,cost,slowest,tables} API; admin-gated
  AnalyticsPage

Tests: +50 unit tests (audit, notifications, schedules, policy, cost);
220 backend tests pass. ruff clean (modulo the standard FastAPI B008).
Frontend tsc + eslint + build clean. CI installs the [scheduling] extra.

Deferred: human-approval gate + certified-metrics-only mode.
Known caveat: the saved-query result cache is not yet role-aware — key
it by effective policy before relying on masking/row-filters there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kosminus kosminus merged commit 981cb30 into main Jun 8, 2026
2 checks passed
@kosminus kosminus deleted the feat/phase4-governance branch June 8, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant