Skip to content

Ask for the ClickHouse HTTPS port instead of inferring it from the native port #8150

Description

@gagantrivedi

A ClickHouse connection stores a single port, documented and defaulted as the native protocol port (CLICKHOUSE_DEFAULTS["port"] = 9440). Verification connects over the native protocol on that port, but delivery (#8107) needs the HTTP interface and derives its port from a fixed map (9440 -> 8443, 9000 -> 8123). Asking for one protocol's port and using another's causes several problems.

A non-standard port verifies healthy and can never deliver

validate_clickhouse_config accepts any port in 1-65535, so a server behind a load balancer on, say, 19000 verifies successfully and the UI shows connected. The first delivery run then raises DeliveryConfigError("No HTTP port is known for ClickHouse port 19000"), marks the connection errored, and repeats every ten minutes. Events accumulate under events/ until the bucket's 30-day expire-objects rule removes them, with no recovery short of moving the server to a default port.

Verification does not test the path delivery uses

A green "Test connection" only proves the native port is reachable. The HTTP port may be firewalled, and a missing events table is not discovered until the first delivery run ten minutes later.

Two libraries, two error taxonomies

Verification uses clickhouse-driver and delivery clickhouse-connect, so there are two near-duplicate error mappers (_describe_verification_error and describe_delivery_error) to keep in step.

Proposal

Ask only for the HTTPS port and use it everywhere.

  • Replace port with the HTTP(S) port in ClickHouseConfig, defaulting to 8443, and relabel the field in the connection form.
  • Move verify_clickhouse_connection onto clickhouse-connect, so verification exercises the exact interface, port and client that delivery uses.
  • Extend verification to check the events table exists with compatible columns, so a schema problem surfaces at setup rather than in a delivery run.
  • Retire NATIVE_TO_HTTP_PORT and collapse the two error mappers into one.
  • Migrate stored configs, or require reconnection for any that exist.

clickhouse-driver stays in the tree either way: it is the DBAPI for django-clickhouse-backend, which backs the segment_membership ClickHouse path. This only removes the direct dependency from experimentation.

Note that clickhouse-connect ignores the clickhouse:// scheme in a DSN and derives the interface from secure/port, so EXPERIMENTATION_CLICKHOUSE_URL needs updating in every deployed environment as part of this. Worth doing alongside #8033.

Metadata

Metadata

Assignees

Labels

apiIssue related to the REST API

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions