Skip to content

Vouch request: Leafgard (fix for #4343) #4487

Description

@Leafgard

Why do you want to contribute?

I'd like to fix #4343 — self-hosted ClickHouse on the hosting/docker layout burning CPU merging its own system telemetry. We run a self-hosted Trigger.dev v4 stack in production and hit this failure mode three times since early July; I posted the full diagnosis and data points on the issue: #4343 (comment)

The planned PR (against current main, post-#4249 layout) ports the dev stack's docker/config/clickhouse-disable-system-logs.xml to hosting/docker, extended with the newer log tables, keeping query_log/error_log but bounded with a config-level TTL, and moves the low-memory profile settings from config.d (where they are silently ignored) to a users.d mount so they actually take effect. The fix is currently soaking in our production; I'll include before/after numbers in the PR.

Prior contributions or relevant experience

I work on infrastructure at Actemium Schweiz AG, where we operate a self-hosted Trigger.dev v4 deployment (webapp + worker VMs) orchestrating our database/storage backup jobs. Relevant background for this PR: three documented production incidents of this ClickHouse failure class diagnosed down to the root cause (merge-retry loop under the memory cap, no backoff for failed non-replicated merges, ClickHouse/ClickHouse#89811), and we independently performed the Bitnami → official ClickHouse image migration on our deployment two weeks before #4249 landed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions