engines: buffer, s3, distributed

[phiSgr]

Note

Adds end-to-end support for ClickHouse S3, Buffer, and Distributed engines (CLI, TS/Py SDKs, codegen), with runtime env credential resolution, docs, and tests (incl. e2e).

• Infrastructure (Rust CLI):
  • Add S3, Buffer, Distributed engine configs, parsing/serialization, non-alterable hash, and DDL generation in clickhouse/queries.rs.
  • Enforce clause support: MergeTree-only ORDER BY/PRIMARY KEY/SAMPLE BY; allow PARTITION BY for S3.
  • Factor resolve_s3_credentials and apply to S3/S3Queue in partial_infrastructure_map.rs.
• Codegen:
  • TypeScript/Python generators emit engine-specific fields for S3/Buffer/Distributed and honor clause constraints/settings.
• SDKs:
  • TS: extend ClickHouseEngines and OlapConfig unions; map engine configs in dmv2/internal.ts.
  • Python: add S3Engine, BufferEngine, DistributedEngine; update internal serialization and validation to restrict unsupported clauses.
• Docs:
  • Add sections for S3 (direct access), Buffer, and Distributed engines (TS & Py), with config examples and caveats; clarify S3 vs S3Queue.
• Tests:
  • New e2e S3 engine env-var resolution tests (TS/Py) and Buffer table verification in template tests.
  • Unit tests updated to validate new engines and clause restrictions; templates export new S3/Buffer examples.
• Misc:
  • Minor test comment tweaks to unify S3/S3Queue env var naming.

^{Written by Cursor Bugbot for commit eb4f916. This will update automatically on new commits. Configure here.}

[linear]

ENG-986 Support engines other than those in the MergeTree family

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
framework-docs	Ready	Preview	Comment	Oct 28, 2025 10:46pm

[callicles]

Small comment about interop with another feature

[callicles]

Missing docs !