Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement database migrations for Postgres backend #1

Merged
merged 2 commits into from
Jul 19, 2023

Conversation

metalgrid
Copy link

Description
This prevents attempts to (re-)create the postgres schema with every instance of the alerta application. When running with multiple workers, e.g. via gunicorn, each worker attempts to run sql/schema.sql. This sometimes causes deadlocks, and if running with raise_on_error=False, it ends up not being able to register the History type (due to missing conn.rollback()).
After this moment, you have a worker that is broken and all requests routed to said worker end up returning a psycopg2.ProgrammingError: can't adapt type 'History' exception.

With the introduction of migrations, we can reliably track the state of the database schema and ensure that no unnecessary database calls are made and reduce the amount of queries made on worker startup.

Changes
Include a brief summary of changes...

  • Adds private methods to the postgresql backend for a migrations table and for applying migrations
  • Changes the naming convention of schema files in sql/ directory to follow a <version>_<name>.sql for ordering the migrations
  • Allows modifying the schema in a step-by-step fashion, simplifying future changes

Checklist

  • Pull request is limited to a single purpose
  • Code style/formatting is consistent
  • All existing tests are passing
  • Added new tests related to change
  • No unnecessary whitespace changes

conn.commit()
except Exception as e:
conn.rollback()
current_app.logger.error('Failed to apply migration %s: %s', migration, e)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preferably remove logging from here and leave the info in the exception so the create_engine method can handle any errors

@metalgrid metalgrid merged commit 60f1ed8 into master Jul 19, 2023
@ManiacMaxo ManiacMaxo deleted the database-migrations-impl branch August 25, 2023 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants