Kubernetes Operator for Alembic

[bdrakester]

Hi everyone,

My team has been using Alembic for database migrations for several years.

Our deployment model follows a GitOps approach using Argo CD. One area that remained outside that workflow was Alembic migrations, which we were executing manually. While Argo CD Sync Hooks are a common solution, they did not fit well with our operational requirements.

To address this, we built a Kubernetes Operator for Alembic. The operator introduces an AlembicMigration Custom Resource that specifies a target Git branch or tag and a destination database. These resources are managed through Argo CD, allowing database migrations to be reconciled as part of the GitOps workflow.

Current capabilities include:
• Alembic upgrades and downgrades
• Git revision drift detection
• Database revision drift detection
• Automatic reconciliation to the desired migration state

For example, if a migration is applied manually and the database state diverges from the declared configuration, the operator detects the drift and reconciles the database back to the intended revision.

We've found this approach valuable in production and believe it may be useful to other teams adopting GitOps workflows.

Given the operator's general-purpose design and our appreciation for both SQLAlchemy and Alembic, we would like to contribute it to the open-source community and explore whether it could have a place within the Alembic/SQLAlchemy ecosystem. The project is written entirely in Python using the Kubernetes Operator Framework (Kopf). We would also welcome feedback and contributions from the Alembic community to help improve the project.

Company policy requires that I contribute to open-source projects using my personal GitHub account, but I do have the company's support in exploring this contribution.

I'd appreciate feedback from the maintainers and community on whether the Kubernetes Operator for Alembic could be a good fit for the Alembic/SQLAlchemy ecosystem, and what the best path forward for collaboration might be.

Thank you for your time and consideration.

[zzzeek]

hi there -

I work at Red Hat so we actually work with kubernetes operators all day, I'm with the openstack project and we write all our operators custom in go.

So on the one hand I have a lot of experience with custom operators, but it's limited to exactly this one job, so on the other i have none - the use case of "using third party, off the shelf custom operators" is mostly unknown to me. From my narrow experience, if someone is using k8s (or openshift in my case) they are already a large code-producing shop with highly specific needs, and generic "off the shelf" operators might be good for ideas but they otherwise are going to roll their own. However, I have no idea. I did find a bunch of third-party, ready-to-go custom operators when I first started with the k8s stuff here but they're all way too generic and simple for how our deployments work.

What I have seen is like, in the dashboard for linode or OVH, is a whole "kubernetes" system where you can build out your own k8s stack. maybe that's where people use "off the shelf" custom operators?

all of that said, I would encourage you to open source your custom operator, as this could serve as a great example for how to do this sort of thing for people who are interested.

as far as where it would live on github, that would be out of our domain (the sqlalchemy project) - we just maintain the core Python libraries here and we eschew being responsible for as much "downstream" as possible.

[bdrakester]

Thanks for the thoughtful response.

Your comments on downstream ownership make a lot of sense, and I completely understand the desire to keep the SQLAlchemy project focused on the core libraries rather than ecosystem tooling.

I also appreciate your encouragement to open source the operator. Even if it ultimately lives outside the SQLAlchemy organization, I agree that it could be valuable as a reference implementation and as a way to connect with others solving similar GitOps and database migration challenges.

Thanks again for taking the time to share your thoughts. It's very helpful as I think through the best path forward for the project.