Allow custom RevisionMap subclass for pluggable revision ordering

[jasonwbarnett]

Problem

We run a high-velocity monorepo where multiple teams land migrations concurrently. In our merge queue, when two PRs each add a migration, the second one to merge discovers there are now two Alembic heads. This kicks the PR out of the merge queue and requires the author to manually rebase and update their migration's down_revision to point at the new head, re-push, and re-enter the queue. With many developers contributing migrations, this compounds — the more PRs in flight, the more frequently authors get ejected and have to rebase, creating a frustrating cycle of churn.

We want to solve this by using git history to automatically determine migration ordering — when concurrent migrations create multiple heads, order them by when they were first introduced in git, so that the merge queue can resolve the ordering deterministically without human intervention. This requires being able to swap in a custom RevisionMap subclass that overrides _topological_sort().

Today there is no way to customize the ordering behavior without monkey-patching internals. The RevisionMap class is instantiated directly inside ScriptDirectory.__init__() with no way to substitute a subclass.

Proposed Solution

Add a revision_map_class configuration option to ScriptDirectory that accepts a dotted Python path to a RevisionMap subclass. This would allow external packages (e.g. a hypothetical alembic-git-ordering) to provide custom ordering logic by overriding _topological_sort() or other methods, without any changes to Alembic core.

This is a minimal, non-breaking change: a single new keyword argument with a None default, and a config option that is entirely opt-in.

Use Cases

• Merge queue compatibility: In CI merge queues (GitHub, Mergify, etc.), concurrent migrations cause repeated head conflicts that eject PRs. Authors must rebase and update down_revision each time, which compounds with team size. A git-history-aware RevisionMap could order unrelated migrations deterministically, eliminating this manual churn.
• Git-history-based ordering: Order concurrent migrations by their first commit date, giving a stable timeline-based ordering across branches.
• Custom merge strategies: Organizations with many parallel branches could implement project-specific rules for how unrelated migrations should be sequenced.

Current Workarounds

The only current options are:

1. Monkey-patching RevisionMap or ScriptDirectory in env.py — fragile and breaks across Alembic upgrades
2. Forking Alembic — heavy-handed for a small customization point
3. Manually rebasing and updating down_revision every time a PR gets ejected — the status quo we're trying to escape

We have a PR ready at #1805 that implements this.

[CaselIT]

Hi,

I'm not sure you use case can work by just changing the topological sort since there are other instances what depend on the revision.down_revision attribute, but we can look at the pull request.

Note that your proposed strategy is not safe in general, since you generally cannot automatically order your migrations using their git commit data, since you may miss migrations if a revision A took one day to merge while another created before A took a week to merge.
That said it would not be an alembic problem if this happened while you used a custom revision map.

@zzzeek what do you think?

[jasonwbarnett]

Thanks for the thoughtful response, @CaselIT.

You raise a good point about down_revision. Let me explain the full picture of what we're trying to achieve, since the revision_map_class is just the first piece.

Our current problem:

1. Two developers independently create migrations on separate branches. Neither migration depends on the other.
2. Both PRs pass CI pre-merge (including a check that there's only one Alembic head).
3. Both PRs enter the merge queue.
4. The first PR merges. Its migration's down_revision correctly points at what was head on main.
5. The second PR fails CI in the merge queue because there are now two heads. The just-merged migration and this PR's migration both point at the old head. The PR gets ejected, and the author has to manually rebase, update down_revision, and re-enter the queue.

What we want to build (the full solution, not just this PR):

The revision_map_class hook would be one piece of a larger solution. The idea is:

1. Migrations would be authored with down_revision = None. Developers would no longer manually maintain the revision chain.
2. A custom RevisionMap subclass would build the ordering entirely from git history, specifically the date each migration file was first committed. If two migrations are introduced in the same commit, the filename/timestamp in the migration acts as a tiebreaker.
3. This means when the merge queue processes the second PR, instead of seeing "two heads, error", the custom RevisionMap would construct a linear chain from git history: the migration committed first goes first, the other follows after it. There are never multiple heads because the ordering is always derived dynamically.

On your concern about ordering safety:

You're right that git commit date ordering is not universally safe. A migration that took a week to merge could end up ordered after one that was created later but merged sooner. However, in our case these are independent, non-conflicting migrations. If migration A adds a widgets table and migration B adds a gadgets column to an unrelated table, it doesn't matter which runs first. The key constraint is that we need a deterministic ordering so CI passes consistently, not that the ordering reflects some semantic dependency.

The ordering would always be fully dynamic. Developers would never set down_revision manually. If two migrations truly do have a dependency between them, the developers would need to coordinate their merge order explicitly, the same way they'd handle any other cross-PR dependency.

CI as a safety net:

We also run the full migration suite in CI both in pre-merge pipelines and in the merge queue. So in the rare case where PR 1's migration actually breaks PR 2's migration when run in sequence, PR 2 would get kicked from the merge queue with a meaningful migration error, the same way any other real conflict surfaces. The difference from today is that PRs only get ejected when there's an actual problem, not just because two independent migrations happened to land concurrently.

Does that clarify the use case? We fully understand the risks and accept that the ordering strategy is our responsibility. We just need the hook to plug it in.

[CaselIT]

I think you misunderstood what I was saying.

The ordering is not deterministic using git commit date:

State 0: B
Merge PR 1 with revision X committed april 9
State 1: B -> X
Merge PR 2 with revision Y committed april 7
State 2: B -> Y -> X

If you deployed to a db state 1, then you go to update to state 2 alembic would see no change is need, but migration Y wan never applied.

That said, if you are fine with it that's ok ok my part.
Note that it would not be considered an issue in alembic the above behavior

[jasonwbarnett]

I think I understand your concern now, and it's a valid one if the ordering were based on the original commit date from the developer's branch. Let me clarify what we'd actually use.

We use merge commits on GitHub. When a PR merges (or enters the merge queue), the merge commit on main gets a new timestamp reflecting when it was actually merged, not when the developer originally authored the migration. So the ordering would be based on when the migration file first appears in main's history via its merge commit.

In your example:

• Developer A commits migration X locally on April 2.
• Developer B commits migration Y locally on March 15.
• PR 1 (migration X) merges into main on April 9 10:00. X gets merge timestamp April 9 10:00.
• Deploy. DB is at state B -> X.
• PR 2 (migration Y) merges into main on April 10 09:00. Y gets merge timestamp April 10 09:00.
• Ordering is B -> X -> Y (not B -> Y -> X), because Y's merge commit on main is newer than X's, regardless of when Developer B originally committed Y locally.
• Alembic sees DB is at X, applies Y. Correct.

The merge commit timestamps on main are always sequential, so the ordering always reflects the actual merge order. A migration can never retroactively slot in before one that was already merged and deployed.

I should have been more precise in my earlier comment. When I said "the date each migration file was first committed" I meant the date it first appears in main's history (via the merge commit), not the original authoring date on the feature branch. Am I missing something obvious here that you still see as a problem?

[CaselIT]

if there are no force pushes in main I guess it should be ok

[jasonwbarnett]

Yeah, we don't allow any force pushes, period. Everything is governed via GitHub rulesets and even "forces" must go through GitHub which still re-stamps via squash merge commit.

[CaselIT]

though of a drawback: you really just have the final ordering when you merge in main, if you ever need a feature branch then it doesn't work

[jasonwbarnett]

100% agree but that's even true with typical alembic migrations, ie if a feature branch is out of date it has the latest migrations from when it was first branched plus the newest migrations. It's pretty similar with commit based ordering. There may be some nuance or details I'm missing but I agree that the edge case exists. Our goal is to use git commit history as the source of truth and that is per branch. For example if we were to create a long lived branch and deploy off of it that would have a distinct and separate order from any other branch (from the time of the branch that is).

[zzzeek]

i have a lot of thoughts on this and the first one is that it seems like you are proposing a system where your code would not be able to run without being present inside of a git checkout. isnt that a serious issue?

beyond that this seems to be more appropriately addressed using merge hooks to automate the renumbering of alembic revision files to be based on the apparent git hashes in the final source tree. just because developers are doing this manually now doesnt mean it cant be automated and this automation doesnt seem any more complex than doing it at runtime based on what's present in .git, and in fact it seems vastly safer to ensure the source tree has a fixed ordering rather than one that can change dynamically if any kind of manipulations to the .git history occur.

[zzzeek]

ive made this general suggestion to Claude (opus) and it proposes three approaches based on github actions so far. I think an approach that creates a safe, static migration directory based on GH actions adapted to your workflow would be the way to go here. see claude output below

Issue #1804 — Automated Migration Ordering in Merge Queues

The Problem

In a high-velocity monorepo with a GitHub merge queue, two PRs that each
add a migration both point down_revision at the same current head. When
the first PR merges, the second gets ejected from the merge queue because
there are now two heads. The author must manually rebase and update
down_revision — this cascades with team size.

Why a Custom RevisionMap Subclass Is Problematic

The proposal in the issue (a custom RevisionMap subclass that derives
ordering from git history at runtime) has fundamental issues:

1. Requires .git at runtime — migrations can't run from a built
   artifact, Docker image without .git, or any deployment that strips
   source control metadata.

2. Feature branches have incomplete/different ordering — as CaselIT
   noted, the "real" ordering is only known on main after merge.

3. down_revision = None on every migration breaks the core contract
   — every migration becomes a root, and the entire revision graph depends
   on a side-channel (git) that may not be present.

4. Other parts of Alembic inspect down_revision directly — not just
   topological sort. This would be a leaky abstraction that breaks in
   unexpected ways.

Proposed Solutions (No Alembic Changes Needed)

All of these produce a committed source tree with correct, explicit
down_revision values that work anywhere.

Solution 1: GitHub Action That Fixes down_revision in the Merge Queue (Recommended)

A two-part automation:

Part A — Merge queue CI fixup (working tree only, no commit):

Before running tests in the merge queue, a CI step detects if the PR's
migration creates multiple heads, and if so, rewrites down_revision in
the working tree to chain off the existing head from main. Tests then run
against the corrected tree.

# .github/workflows/ci.yml
on:
  merge_group:
  pull_request:

jobs:
  test:
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Fix migration ordering
        run: python scripts/fix_migration_head.py

      - name: Run tests
        run: pytest

Part B — Post-merge fixup (commits the correction):

After a push to main, a separate action checks for multiple heads and
commits a fix if needed.

# .github/workflows/fix-migration-heads.yml
on:
  push:
    branches: [main]
    paths: ['alembic/versions/*.py']

jobs:
  fix-heads:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Fix multiple heads
        run: python scripts/fix_migration_head.py --commit

      - name: Push fix
        run: |
          git add alembic/versions/
          git diff --cached --quiet || \
            (git commit -m "auto-fix migration down_revision" && git push)

The fix_migration_head.py script would be straightforward — parse
revision and down_revision from migration files using regex, build
the graph, find heads, and rewrite the down_revision line in the newest
migration file. This is roughly 50–80 lines of Python.

Why this works: The merge queue CI passes because the working tree is
fixed before tests run. The post-merge action ensures the committed code
on main always has correct down_revision. The source tree is
self-contained — no .git needed at runtime.

Solution 2: Accept Multiple Heads + Automated Merge Migration

Simpler, but produces extra migration files:

1. Remove the single-head CI enforcement.
2. Add a GitHub Action on push to main that runs alembic merge heads
   when multiple heads are detected:

on:
  push:
    branches: [main]
    paths: ['alembic/versions/*.py']

jobs:
  merge-heads:
    steps:
      - uses: actions/checkout@v4

      - run: |
          HEADS=$(alembic heads | grep -c "^")
          if [ "$HEADS" -gt 1 ]; then
            alembic merge heads -m "auto-merge concurrent migrations"
            git add alembic/versions/
            git commit -m "auto-merge alembic heads"
            git push
          fi

This uses Alembic's native multi-head support. The downside is
accumulating merge migration files, but it is the simplest approach and
is fully supported by Alembic's existing design.

Solution 3: PR Bot That Pre-Rebases Migration down_revision

A GitHub Action on pull_request that watches for PRs touching migration
files, compares the PR's migration down_revision against main's current
head, and pushes a fixup commit to the PR branch if stale:

on:
  pull_request:
    paths: ['alembic/versions/*.py']

jobs:
  update-migration:
    steps:
      - uses: actions/checkout@v4
        with:
          ref: ${{ github.head_ref }}
          fetch-depth: 0

      - name: Update down_revision to current main head
        run: python scripts/rebase_migration.py

      - name: Push if changed
        run: |
          git add alembic/versions/
          git diff --cached --quiet || \
            (git commit -m "update migration down_revision" && git push)

This is proactive — PRs always enter the queue with the right
down_revision. The downside is it triggers another CI run on the PR
after the push.

Recommendation

Solution 1 is the most robust. It handles the merge queue correctly
(Part A fixes the working tree so CI passes) and ensures main always has
correct committed state (Part B). The migration files work anywhere — no
.git required, no Alembic patching, standard down_revision chain.

The core script needed is a small amount of Python that parses revision
files with regex and rewrites one down_revision line. That is a much
simpler and more maintainable artifact than a custom RevisionMap
subclass with git subprocess calls at runtime.

[jasonwbarnett]

Thanks for the detailed response @zzzeek. I want to address a few things.

On the .git runtime dependency:

Totally fair point. Our plan was never to require .git at runtime in production. The idea is to compute the ordering from git history at container build time and bake the fixed down_revision values into the artifact. The dynamic ordering from git would only be used during development and CI. The deployed artifact would have a static, fully resolved migration chain with no .git dependency.

On the GitHub Actions solutions:

I appreciate the suggestions, but unfortunately I don't think any of them actually solve the problem in a merge queue context. I think the LLM you used was being too agreeable here and not pressure-testing these against the specific constraints of a high-velocity merge queue. Let me walk through why:

Solution 1 Part A (fix working tree in merge queue CI): The merge queue creates a temporary merge commit, runs CI on it, and then merges the original PR branch into main. Modifications to the working tree during CI are thrown away. The merged code on main still has the wrong down_revision.

Solution 1 Part B (post-merge fixup commit): This pushes directly to main via git push, which bypasses our merge queue and branch rulesets. We require everything to go through the merge queue. Even if we relaxed that, there's a race window between the merge and the fixup where main has multiple heads. At high velocity, multiple PRs can merge before the fixup runs, compounding the problem. Two fixup actions running concurrently can also conflict with each other.

Solution 2 (auto-merge heads post-merge): Same problems as Solution 1 Part B. Direct push to main, race windows, concurrent conflicts.

Solution 3 (PR bot that pre-rebases): This doesn't help with the merge queue. Two PRs can both be updated to point at the same head on main, both enter the merge queue, and the second one still creates two heads. This is exactly the problem we already have today.

The core issue:

All three solutions try to maintain a static down_revision chain through an inherently concurrent process. They either fix things too late (post-merge, with race windows), fix things in a context that gets thrown away (merge queue working tree), or fix things too early (before the merge queue, when the final order isn't known). None of them actually solve the problem, they just move it around.

Dynamic ordering at the RevisionMap level sidesteps this entirely because there's nothing to fix. The ordering is computed from the actual state of the repository. During CI and development, that comes from git. In deployed artifacts, it comes from pre-computed values baked in at build time.

I understand if this isn't something you want to support in Alembic core. But I want to make sure the problem is clearly understood, because the suggested alternatives don't work for our use case.