Skip to content

chore: bump clickhouse-sync#3957

Merged
vpol merged 3 commits into
mainfrom
chore/bump_clickhouse_sync
Jun 22, 2026
Merged

chore: bump clickhouse-sync#3957
vpol merged 3 commits into
mainfrom
chore/bump_clickhouse_sync

Conversation

@vpol

@vpol vpol commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

No description provided.

vpol and others added 3 commits June 22, 2026 09:54
Signed-off-by: Viktor Poluksht <viktor@daily.dev>
Signed-off-by: Viktor Poluksht <viktor@daily.dev>
@pulumi

pulumi Bot commented Jun 22, 2026

Copy link
Copy Markdown

🍹 The Update (preview) for dailydotdev/api/prod (at 5804a03) was successful.

✨ Neo Code Review

Routine application image rollout (`4aee402a` → `5804a034`) with targeted ClickHouse sync connector changes: a snapshot mode rename and a new sink queue size cap. The connector will restart with a brief CDC replication gap (no data loss due to the replication slot), but the new `sink.connector.max.queue.size: 1000` should be validated against production throughput. 🟡 Moderate Risk

This PR deploys a new application image (5804a034 → replaces 4aee402a across all vpc-native deployments and cronjobs) alongside targeted changes to the ClickHouse Debezium sync connector.

The two ClickHouse-specific config changes are worth understanding:

snapshot.mode: "never""no_data": Both modes skip snapshotting existing table data, but no_data is the newer Debezium terminology that also captures schema structure without data. This is a behavioral change in how the connector initializes — it should be functionally equivalent for an already-running CDC pipeline, but the connector restart (caused by the replacement) means it will re-read from the existing replication slot position.

sink.connector.max.queue.size: 1000: New backpressure limit on the sink side. This caps how many records are buffered before the sink connector applies backpressure upstream. At 1,000 vs max.queue.size: 128,000 on the source side, this is a significant constraint — verify this is intentional and won't cause the connector to stall under normal load.

The Secret replacement (clickhouse-sync-clickhouse-sync-config) triggers the Deployment replacement. The Deployment uses strategy: Recreate, meaning the old pod is terminated before the new one starts — there will be a brief gap in CDC replication during the rollout. Events during this window will be buffered in the Postgres replication slot (clickhouse_sync), so no data is lost, but replication lag will increase temporarily.

🟡 Warning — The sink.connector.max.queue.size: 1000 is orders of magnitude smaller than the source queue size. If this value is too low for production throughput, the connector could bottleneck or exhibit unexpected behavior after the forced restart.

Resource Changes

    Name                                                       Type                                      Operation
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment             update
~   clickhouse-sync                                            dailydotdev:pulumi-common:ClickHouseSync  update
~   vpc-native-expire-super-agent-trial-cron                   kubernetes:batch/v1:CronJob               update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob               update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob               update
-   vpc-native-api-clickhouse-migration-3571d872               kubernetes:batch/v1:Job                   delete
+-  clickhouse-sync-clickhouse-sync-config                     kubernetes:core/v1:Secret                 create-replacement
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob               update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob               update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob               update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob               update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob               update
~   vpc-native-post-lifecycle-state-clickhouse-cron            kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob               update
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment             update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob               update
~   vpc-native-post-analytics-achievements-cron                kubernetes:batch/v1:CronJob               update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob               update
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment             update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob               update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob               update
~   vpc-native-daily-digest-cron                               kubernetes:batch/v1:CronJob               update
~   vpc-native-bg-deployment                                   kubernetes:apps/v1:Deployment             update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment             update
~   vpc-native-materialize-monthly-best-post-archives-cron     kubernetes:batch/v1:CronJob               update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob               update
~   vpc-native-squad-posts-analytics-refresh-cron              kubernetes:batch/v1:CronJob               update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob               update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob               update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob               update
+-  clickhouse-sync-clickhouse-sync                            kubernetes:apps/v1:Deployment             create-replacement
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob               update
~   vpc-native-materialize-yearly-best-post-archives-cron      kubernetes:batch/v1:CronJob               update
+   vpc-native-api-db-migration-15e1774e                       kubernetes:batch/v1:Job                   create
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob               update
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob               update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob               update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment             update
~   vpc-native-channel-highlights-cron                         kubernetes:batch/v1:CronJob               update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob               update
... and 17 other changes

@vpol vpol merged commit 8d699b7 into main Jun 22, 2026
9 checks passed
@vpol vpol deleted the chore/bump_clickhouse_sync branch June 22, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant