Skip to content

Add Redshift warehouse adapter #59

Description

@carolsimone

Add a Redshift warehouse engine: one WarehouseAdapter implementation, published as continuo-redshift-adapter, with a continuo-python-runtime-redshift base image.

Part of the warehouse-engines rollout tracked in carolsimone/continuo#517. Sibling engines: BigQuery (#57), Snowflake (#58).

Scope

Mirror adapters/postgres/ and adapters/trino/. A new engine is one WarehouseAdapter subclass serving both roles (data plane + validation) and one image that runs both.

  • New uv workspace member adapters/redshift/: continuo_redshift_adapter/ (the adapter class), pyproject.toml, tests/, README.md. Add it to [tool.uv.workspace] in the root pyproject.toml.
  • Implement the data-plane methods: fetch, ensure_table, load.
  • Implement the validation methods: ensure_schema, drop_schema, build_empty_from_sql, build_empty_from_columns, clone_empty_from_prod, check_binds.
  • Register the adapter under the continuo_engine.adapters entry-point group (so the runtime discovers it at run time).
  • Publish continuo-redshift-adapter on the shared vX.Y.Z tag (wire into publish-pypi.yml).
  • Add Dockerfile.redshift + image-requirements-redshift.txt; wire into images.yml → base image continuo-python-runtime-redshift (installs the pinned continuo-redshift-adapter==X.Y.Z).
  • Tests: adapter unit tests + the shared adapter conformance suite.

Engine notes

  • Wire protocol is postgres-compatible (psycopg / redshift-connector), so adapters/postgres/ is the closest starting point — but the type system and DDL differ, so it is a distinct adapter, not a postgres subclass by default.
  • clone_empty_from_prod: CREATE TABLE ... (LIKE prod) is supported.
  • Watch for Redshift-specific differences: no true SERIAL/identity parity, COPY-based bulk load, distribution/sort keys, and reduced type coverage vs. Postgres.

Do not touch

The result-block wire contract is frozen (docs/boundary-contract.md, continuo_engine_contract.result). The Go side (pkg/validationresult in carolsimone/continuo) parses it byte-for-byte. Reuse it as-is.

Reference

  • adapters/postgres/ (closest — shared wire protocol) and adapters/trino/.
  • docs/boundary-contract.md — the external boundary contract.
  • continuo-engine-contract (contract/) — the WarehouseAdapter port and the shared SQL/type/config guards.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions