Scheduled background jobs use per-pod setInterval with no leader election — redundant, non-deterministic execution

[Wilfred007]

What

Replace the per-pod setInterval scheduling in backend/src/jobs/part49Jobs.ts with a leader-elected or externally-triggered scheduling mechanism so daily jobs run exactly once, at a predictable time.

Why

The docstring in part49Jobs.ts claims the jobs are "Designed to run once per day... at midnight UTC," but the actual implementation (L13-28) is a plain setInterval(fn, 24h) started from each pod's boot time, called unconditionally in every replica via index.ts. With k8s/base/backend-hpa.yaml setting minReplicas: 2, both the daily usage snapshot and nightly integrity check run redundantly on every pod, at whatever time each pod happened to boot — not midnight, and drifting further with every deploy/restart/scale event.

Scope

In scope:

• Either: (a) add a leader-election mechanism (e.g. a Postgres advisory lock claimed once per day) so only one pod executes the job, or (b) move these jobs to a proper scheduled trigger (k8s CronJob) external to the app pods
• Ensure the job actually runs at a fixed, predictable time (e.g. midnight UTC) rather than pod-boot-relative
• Update the docstring to match actual behavior, or make behavior match the docstring — whichever approach is chosen

Out of scope:

• The payroll scheduler cron (scheduleExecutor.ts), covered by a separate issue
• The content/logic of the daily usage snapshot or integrity check themselves

Acceptance Criteria

• With 2+ backend replicas running, the daily job executes exactly once per day, not once per pod
• Execution time is deterministic (e.g. actually midnight UTC, not boot-time-relative)
• Behavior matches documentation
• Test or manual verification with multiple simulated instances

Technical Context

• backend/src/jobs/part49Jobs.ts — docstring L10-11, setInterval implementation L13-28
• backend/src/index.ts — unconditional job init in every replica
• k8s/base/backend-hpa.yaml — minReplicas: 2

[MarcusDavidG]

I can handle this issue and will like to work on it, Please assign thank you.

[Akatenvictor]

I’d like to take on this issue. I understand the core problem with the current setInterval implementation: every backend pod schedules and executes the daily jobs independently, causing duplicate runs and unpredictable execution times.

I can implement a reliable solution using PostgreSQL advisory locking or a Kubernetes CronJob, ensuring the jobs run exactly once per day at a fixed time (midnight UTC) regardless of pod restarts, deployments, or scaling.

I’ll also update the relevant documentation and add appropriate tests to verify single execution across multiple replicas, while keeping the scope limited to the scheduling mechanism.

I’m confident I can deliver a clean, reliable, and well-tested solution for this issue.

[grantfox-oss]

🦊 GrantFox — @Akatenvictor has been assigned to this issue as part of the Third Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #450)
2. Your PR will be reviewed by the Protocol-Guild maintainers

Good luck! Track your progress on GrantFox.

[grantfox-oss]

🎉 This issue has been marked as completed on GrantFox as part of the Third Campaign campaign!

@Akatenvictor's PR #565 was approved and merged by @Wilfred007.

🏆 @Akatenvictor: You earned 35 FoxPoints for this contribution! Your current tier: Explorer (309 total points). Track your full progress on GrantFox.

👏 Great work, @Akatenvictor! Keep contributing to Protocol-Guild.