feat(seo): add sources sitemap

[idoshamun]

Summary

• add a new /api/sitemaps/sources.xml endpoint for qualified public machine sources
• include sources.xml in the sitemap index and cover it with integration tests
• simplify the qualification query to a grouped join over public posts

Key decisions

• require active, public, machine sources with at least 10 public posts and recent activity in the last 12 months
• count only public posts for qualification (visible, not deleted, not private, not banned)
• keep sitemap output ordered by source creation time with a handle tie-breaker

Closes ENG-1183

---

Created by Huginn 🐦‍⬛

[pulumi]

🍹 The Update (preview) for dailydotdev/api/prod (at 30e02a3) was successful.

✨ Neo Explanation

This is a full production rollout of a new application version, updating all 7 services and 38 cron jobs to the latest build, and replacing the previous database and ClickHouse migration jobs with new ones for this release.

Root Cause Analysis

A new version of the API application has been built and is being deployed to production. Every Kubernetes resource is being updated to roll out the new container image, moving from the previous build to the latest one.

Dependency Chain

The new application version triggers a broad rollout across the entire platform:

• 7 Deployments (main API, background jobs, WebSocket server, private API, Temporal worker, personalized digest, and worker jobs) will perform rolling updates to the new image — existing pods stay running until new ones are healthy.
• 38 CronJobs are updated so that their next scheduled runs will use the new image.
• 2 one-time migration Jobs (database and ClickHouse) are being replaced: the old migration jobs from the previous version are deleted and new ones are created to run the migrations bundled with this release.

Risk analysis

The migration jobs (both DB and ClickHouse) represent the most sensitive part of this deployment — they will run schema/data migrations against the production databases. If the migrations contain destructive or irreversible changes, there is a potential for data issues. The application deployments themselves use rolling updates, so there is no expected downtime. The GCP credentials warning is cosmetic and does not block the deployment.

Resource Changes

    Name                                                       Type                           Operation
-   vpc-native-api-clickhouse-migration-a156bfdc               kubernetes:batch/v1:Job        delete
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-gifted-plus-cron                          kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                      kubernetes:apps/v1:Deployment  update
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment  update
~   vpc-native-update-views-cron                               kubernetes:batch/v1:CronJob    update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-f7b49cfe                       kubernetes:batch/v1:Job        create
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-history-day-clickhouse-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-update-tag-recommendations-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob    update
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-squad-posts-analytics-refresh-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
-   vpc-native-api-db-migration-a156bfdc                       kubernetes:batch/v1:Job        delete
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment  update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-tag-view-cron                     kubernetes:batch/v1:CronJob    update
... and 10 other changes