feat: publish canonical highlight events

[vpol]

Summary

Publishes canonical highlight CDC changes on the existing api.v1.post-highlighted topic using the new HighlightsCanonicalPublishedMessage payload.

Changes included:

• adds public.highlights_canonical to the Debezium table include list
• publishes an event when a canonical highlight is created or gains new channels
• includes both full channels state and published_channels delta in the payload
• registers a real-time worker subscription on the existing api.v1.post-highlighted topic
• broadcasts canonical highlight updates to the existing Redis highlight channel per target channel
• bumps @dailydotdev/schema to 0.3.11

Related PRs

• Schema message: https://github.com/dailydotdev/schema/pull/181
• Consumer first: https://github.com/dailydotdev/social-firehose/pull/5

Dependency

@dailydotdev/schema@0.3.11 is published and includes HighlightsCanonicalPublishedMessage.

Because the package was published today, this PR temporarily adds @dailydotdev/schema to minimumReleaseAgeExclude in pnpm-workspace.yaml, following the repo README guidance for urgent dependency updates.

Validation

• pnpm exec prettier --write ... on touched TS files
• pnpm run build

[pulumi]

🍹 The Update (preview) for dailydotdev/api/prod (at 7863b38) was successful.

✨ Neo Code Review

This change wires up a new real-time highlights pipeline: Debezium now captures `highlights_canonical` table changes and routes them through a new Pub/Sub topic/subscription to a Redis broadcast worker. The Debezium Secret replacement causes a brief CDC pause (Recreate strategy), but no event data loss is expected due to WAL replication. ✅ Low Risk with a minor transient ingestion gap during Debezium restart.

This PR introduces real-time highlight broadcasting: when a highlights_canonical row is created or updated with new channels, the CDC worker now emits an api.v1.post-highlighted Pub/Sub event, which the new newHighlightRealTime worker consumes and fans out to a Redis pub/sub channel per published channel.

The infrastructure changes reflect three distinct concerns:

Routine: All deployments and cron jobs are rolling to the new image digest (7863b38 → replacing f1668d7), and DB/Clickhouse migration jobs are being cycled to the new commit hash. This is standard per-deploy behavior.

New Pub/Sub subscription (api.new-highlight-real-time) is being created to wire the new worker to the api.v1.post-highlighted topic declared in common.ts.

🟡 Warning — vpc-native-debezium-props (Kubernetes Secret) is being replaced due to the change in application.properties that adds public.highlights_canonical to Debezium's table.include.list. Replacing this Secret forces a replacement of the vpc-native-debezium-deployment as well (it mounts the Secret). The Debezium deployment uses a Recreate strategy, meaning the old pod is terminated before the new one starts — there will be a brief CDC ingestion gap during the restart. Any DB changes to tracked tables during this window will still be captured via WAL replication slots once Debezium reconnects, so no events should be permanently lost, but real-time delivery will pause momentarily.

🔵 Info — The @dailydotdev/schema package is pinned to 0.3.11 (from 0.3.10) and exempted from minimumReleaseAge in pnpm-workspace.yaml, meaning this schema version bypasses the 7-day delay policy. This is intentional to unblock the new HighlightsCanonicalPublishedMessage type needed by this feature.

Resource Changes

    Name                                                       Type                                  Operation
+   vpc-native-api-db-migration-a0bce9e0                       kubernetes:batch/v1:Job               create
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment         update
~   vpc-native-post-analytics-history-day-clickhouse-cron      kubernetes:batch/v1:CronJob           update
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob           update
~   vpc-native-post-analytics-achievements-cron                kubernetes:batch/v1:CronJob           update
+   vpc-native-api-clickhouse-migration-a0bce9e0               kubernetes:batch/v1:Job               create
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob           update
~   vpc-native-update-views-cron                               kubernetes:batch/v1:CronJob           update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment         update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob           update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob           update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob           update
+   api-sub-api.new-highlight-real-time                        gcp:pubsub/subscription:Subscription  create
+-  vpc-native-debezium-deployment                             kubernetes:apps/v1:Deployment         create-replacement
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob           update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob           update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob           update
~   vpc-native-update-tags-str-cron                            kubernetes:batch/v1:CronJob           update
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment         update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob           update
+-  vpc-native-debezium-props                                  kubernetes:core/v1:Secret             create-replacement
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob           update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob           update
~   vpc-native-deployment                                      kubernetes:apps/v1:Deployment         update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob           update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob           update
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob           update
~   vpc-native-daily-digest-cron                               kubernetes:batch/v1:CronJob           update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob           update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob           update
~   vpc-native-expire-super-agent-trial-cron                   kubernetes:batch/v1:CronJob           update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-gifted-plus-cron                          kubernetes:batch/v1:CronJob           update
-   vpc-native-api-clickhouse-migration-00a7c1bc               kubernetes:batch/v1:Job               delete
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob           update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob           update
~   vpc-native-materialize-yearly-best-post-archives-cron      kubernetes:batch/v1:CronJob           update
... and 17 other changes