Skip to content

Commit

Permalink
alembic: addition of documentation
Browse files Browse the repository at this point in the history
Signed-off-by: Dinos Kousidis <konstantinos.kousidis@cern.ch>
Co-authored-by: Nicolas Harraudeau <nicolas.harraudeau@cern.ch>
  • Loading branch information
Dinos Kousidis and Nicolas Harraudeau committed Jun 22, 2017
1 parent 4df7ce1 commit df0f229
Showing 1 changed file with 118 additions and 0 deletions.
118 changes: 118 additions & 0 deletions alembic.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
Alembic
=======
Alembic is a database migration library used with SQLAlchemy ORM. Invenio works with the Flask-Alembic library, its documentation can be found here: http://flask-alembic.readthedocs.io/en/latest/
Invenio-DB fully supports Alembic and each Invenio module having a database model is also expected to provide the corresponding alembic scripts.
Alembic migrations do not work with SQLite.


Adding alembic support to existing modules
------------------------------------------

We can create the alembic directory with:

.. code-block:: console
$ invenio alembic mkdir
and then add the invenio_db.alembic entrypoint in setup.py:

.. code-block:: python
setup(
...
entry_points={
...
'invenio_db.alembic': [
'invenio_unicorn = invenio_unicorn:alembic',
]
})
This will register the invenio_unicorn.alembic directory in alembic's version_locations
Each module creates a branch for its revisions. To create a new branch:

.. code-block:: console
$ invenio alembic revision "Create unicorn branch." -b invenio_unicorn -p parent_revision -d dependencies --empty
| -b sets the branch label
| -p sets the parent revision, as default all branches root from 'dbdbc1b19cf2' in invenio-db
| -d sets the dependencies if they exist. For example when there is a foreign key pointing to the table of another invenio module, we need to make sure that table exists before applying this revision so we add it as a dependency.
Creating a new revision
-----------------------

The following procedure adds an alembic revision to invenio-unicorn for demonstration purposes.

After making changes to the models of a module, we need to create a new alembic revision to apply these changes in the DB with a migration.
After making sure that the module creates its own branch and the alembic entrypoint is present as described above, we can create the new revision.

First to make sure that the DB is up to date we apply any pending revisions by

.. code-block:: console
$ invenio alembic upgrade heads
and now we can create the revision by:

.. code-block:: console
$ invenio alembic revision "Revision message." --path alembic_directory_path
A short message describing the changes is required and the path parameter should point to the alembic directory of the module. If the path is not given the new revision will be placed in the invenio_db/alembic directory and should be moved.

The directories accepted are registered by the invenio_db.alembic entrypoints.

Show current state
------------------

To see the list of revisions in the order they will run:

.. code-block:: console
$ invenio alembic log
The list of heads for all branches is given by:

.. code-block:: console
$ invenio alembic heads
in this list revisions will be shown as (head) or (effective head), the difference being that effective heads are not shown in the alembic_version table as they are dependencies of other branches and they are overwritten.

The list of the revisions that have been applied is given by:

.. code-block:: console
$ invenio alembic current
Enabling alembic migrations in existing invenio instances
---------------------------------------------------------

When there is a DB in place but there is no alembic_version table, meaning that the DB wasn't created through alembic upgrade, in order to start using alembic the table has to be inserted and stamped with the revisions matching the current state of the DB:

.. code-block:: console
$ invenio alembic stamp
Given that there have been no changes in the DB, and the models match the alembic revisions, alembic upgrades and downgrades will be working now.
A thing to note is that the any constraints that were created without names, will have the default names from the DB which can be different from the ones in the alembic revisions, leading to errors in the future.

Naming Constraints
------------------

In http://alembic.zzzcomputing.com/en/latest/naming.html, the need for naming constraints in the models is explained. In invenio-db the '35c1075e6360' revision applies the naming convention. If models contain constraints that are unnamed an "InvalidRequestError: Naming convention including %(constraint_name)s token requires that constraint is explicitly named."" will be raised.

The naming convention rules are:

| index 'ix_%(column_0_label)s'
| unique 'uq_%(table_name)s_%(column_0_name)s'
| check 'ck_%(table_name)s_%(constraint_name)s'
| foreign key 'fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s'
| primary key 'pk_%(table_name)s'
In the alembic revisions the constraints that will produce a name longer that 64 characters have to be named explicitly to a truncated form.

Testing revisions
-----------------
The test for the created revisions starts from an empty DB, upgrades to the last branch revision and then downgrades to the base. We can check that there are no discrepancies in the state of the DB between the revisions and the models, by asserting that alembic.compare_metadata() returns an empty list.

0 comments on commit df0f229

Please sign in to comment.