Skip to content

Phase 0 follow-up: integration + Alembic migration tests for the CI baseline #8

@kosminus

Description

@kosminus

Context

The Phase 0 plan calls for a CI baseline of integration tests covering the pipeline, connectors, semantic context builder, SQL sanitizer, and Alembic up/down — before the refactors land.

PR #7 delivered DB/LLM-free unit tests (secrets, jobs/registry/arq, rate limiter, telemetry, tracing, SQL sanitizer, health-probe shape) and a CI workflow that gates on pytest. Of the integration targets, only the SQL sanitizer is covered. This issue tracks closing the rest so the stated baseline is fully met.

Scope

  • Alembic round-tripupgrade head then downgrade base (and back) against a disposable Postgres+pgvector, asserting no errors and a clean schema diff.
  • Query pipelineexecute_nl_query end-to-end with a fake/stub LLM provider and a real (or sqlite/pg) target DB: context build → compose → validate → execute → interpret, plus the retry path.
  • ConnectorsPostgreSQLConnector against a real Postgres (read-only enforcement, timeout, max-rows truncation); BigQuery/Databricks behind recorded/mocked clients.
  • Semantic context builderbuild_context with seeded glossary/metrics/dictionary/knowledge, asserting prompt assembly and relevance selection.

Infra needed

These require services CI doesn't yet provide:

  • A postgres:16 + pgvector service container in the workflow.
  • A deterministic fake LLM provider (registered via provider_registry) to avoid network/key dependencies and keep tests hermetic.
  • Test fixtures/factories for connections + metadata (the existing tests/fixtures/sample_seed.sql is a starting point).

Notes

  • Keep the existing unit tests as the fast, no-dependency gate; add integration tests as a separate (possibly slower) CI job.
  • Once green, consider flipping the advisory ruff/mypy CI steps to gating after the pre-existing lint debt is addressed.

Surfaced during review of #7.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions