Proposal: deterministic schedule jitter so daily DAGs don't all fire at the same time

[Pebble32]

Hi all,

Not sure what the formatting here is and if there is a template for me to use to apologies if the setup is unclear. I tried to include all relevant information needed.

The problem

In bigger deployments a lot of DAGs end up on the same cron time. @daily always resolves to 0 0 * * *, so every daily DAG fires at midnight at the exact same moment. On a production MWAA environment I work on we had around 26 daily DAGs all firing at 00:00 and it caused task failures from the contention at that boundary.

Right now there is no native way to spread them out. You either:

• hand pick a unique minute for every DAG, which drifts and collides as the number of DAGs grows, or
• hash the dag id into a literal cron string yourself, which works but throws away the @daily intent and is not reusable.

Neither feels like something every team should be reinventing on their own.

What I would like to propose

A way to add deterministic jitter to a schedule. Something like a JitteredCronTimetable, or a jitter option on the existing cron timetables, that offsets each DAG by a stable function of its dag id inside a configurable window (say up to 60 minutes), while keeping the data_interval and logical_date semantics exactly the same.

The important part is that it stays deterministic rather than random, so a given DAG always lands in the same slot and runs stay stable and predictable across scheduler restarts and timetable serialization.

Prior art

I looked and could not find anything that already does this. No native option, and no community plugin that I could find. Other schedulers tend to offer something in this space (Kubernetes CronJobs for example).

Questions before I build anything

1. Is this something you would want in core, or is it better as a community plugin?
2. If core, what shape do you prefer, a new Timetable or a jitter option on the existing cron timetables?
3. Does this need an AIP, or is it small enough to go straight to a PR?

I already have a working product at my current workplace in production and I am happy to bring tests and docs.