Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-43727: Add initial support for Alembic migrations #8

Merged
merged 6 commits into from
May 28, 2024
Merged

Conversation

JeremyMcCormick
Copy link
Contributor

The URL for the live database should be specified using the CONSDB_URL, and the path to the sdm_schemas Git repository should be set using the SDM_SCHEMAS_DIR environment variable.

alembic/env.py Outdated
sdm_schemas_dir = os.getenv("SDM_SCHEMAS_DIR")
if sdm_schemas_dir is None:
raise ValueError("SDM_SCHEMAS_DIR not found in environment")
schema_path = f"{sdm_schemas_dir}/yml/cdb_latiss.yaml"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll eventually need to have one of these per instrument.

How do we make sure the alembic_version table goes in the schema, not outside it? Right now, it seems to be going in the default public schema.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it looks like alembic is taking responsibility for every schema in the database, which would cause it to drop all the Butler tables. Not so good...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll eventually need to have one of these per instrument.

If they all need separate migration histories then we will likely have to give them their own environments with different env.py and alembic.ini files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like alembic is taking responsibility for every schema in the database

We can have it ignore certain schemas if their names are known:

https://stackoverflow.com/questions/35342367/how-can-i-ignore-certain-schemas-with-alembic-autogenerate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we make sure the alembic_version table goes in the schema, not outside it? Right now, it seems to be going in the default public schema.

This is easily fixed in the configuration:

version_table_schema='your_custom_schema'

Though you might actually want that in a separate schema than the one being managed because the migrations themselves may drop/create it.

JeremyMcCormick and others added 4 commits May 20, 2024 01:25
The URL for the live database should be specified using the CONSDB_URL,
and the path to the sdm_schemas Git repository should be set using the
SDM_SCHEMAS_DIR environment variable.
@ktlim
Copy link
Contributor

ktlim commented May 20, 2024

I moved the versions tables to a common cdb schema. I separated the instruments using the alembic --name functionality using sections in alembic.ini. I added a custom config item for the schema name and filtered out all other schemas.

@ktlim ktlim merged commit 0fb466a into main May 28, 2024
4 checks passed
@ktlim ktlim deleted the tickets/DM-43727 branch May 28, 2024 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants