Self hosted: Clickhouse Migration

[birdiemayank]

Describe your question

Can anyone please help me with Clickhouse migration.
Currently we are running self hosted langfuse setup with all the services on a single server. We have created clickhouse container on another server to isolate database services. Please help me with documentation or steps to backup and migrate all the data to another clickhouse hosted on another server and use the same for future traces in my langfuse.

Langfuse Cloud or Self-Hosted?

Langfuse Cloud

If Self-Hosted

3.84.0

If Langfuse Cloud

No response

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

Migrating ClickHouse data to a new server involves a few key steps. While there isn't a dedicated Langfuse migration tool for this specific scenario, here's the approach:

1. Configure the New ClickHouse Server

Ensure your new ClickHouse instance has the required environment variables configured for both Langfuse Web and Worker containers:
(1)

CLICKHOUSE_URL=http://<new-hostname>:8123
CLICKHOUSE_MIGRATION_URL=clickhouse://<new-hostname>:9000
CLICKHOUSE_USER=<username>
CLICKHOUSE_PASSWORD=<password>
CLICKHOUSE_CLUSTER_ENABLED=false

(1)

Set CLICKHOUSE_CLUSTER_ENABLED=false for single-container setups.
(1)

2. User Permissions

The ClickHouse user must have these grants:
(1)

GRANT INSERT ON default.* TO 'user';
GRANT SELECT ON default.* TO 'user';
GRANT ALTER UPDATE, ALTER DELETE ON default.* TO 'user';
GRANT ALTER DROP INDEX ON default.* TO 'user';
GRANT CREATE ON default.* TO 'user';
GRANT DROP TABLE ON default.* TO 'user';

(1)

3. Data Migration

For the actual data migration between ClickHouse instances, you'll need to use ClickHouse's native backup and restore capabilities. The Langfuse documentation doesn't provide specific steps for this, but there's a relevant GitHub discussion on recovering from ClickHouse backups: Documentation / help for recovering Langfuse from a Clickhouse DB backup.
(2)

4. Update Langfuse Configuration

After migrating the data, update your Langfuse Web and Worker containers to point to the new ClickHouse server by updating the CLICKHOUSE_URL and CLICKHOUSE_MIGRATION_URL environment variables.
(1)

Important Notes

• Langfuse supports ClickHouse versions >= 24.3.
  (1)
• Langfuse expects infrastructure components to default to UTC timezone.
  (1)

I'd recommend checking the GitHub discussion linked above and ClickHouse's official documentation for backup/restore procedures, as the specific data migration steps fall outside Langfuse's documented scope.

📚 Sources:

1. ClickHouse
2. Troubleshooting & FAQ

---

Have another question? Just tag @inkeep.