Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try out Alembic #82

Draft
wants to merge 13 commits into
base: dev
Choose a base branch
from
Draft

Commits on Jan 7, 2021

  1. Add initial Alembic structure

    Except for some style fixes, these files where generated by running
    `alembic init alembic` inside "src/egon/data".
    gnn committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    d4d24c4 View commit details
    Browse the repository at this point in the history
  2. Add "alembic.ini"

    This is also the result of running `alembic init alembic`, but I forgot
    to add the file in the previous commit, because I only committed the
    contents of the alembic directory, whereas the "alembic.ini" file got
    placed directly outside of it.
    gnn committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    c2250be View commit details
    Browse the repository at this point in the history
  3. Use egon_data engine creator for Alembic

    Guido Pleßmann committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    f60f7cf View commit details
    Browse the repository at this point in the history
  4. Add ORM for OpenStreeMap tables

    This includes only ORMs for target tables in schema openstreetmap
    Guido Pleßmann committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    83f334c View commit details
    Browse the repository at this point in the history
  5. Add ORM for VG250 tables in boundaries schema

    Guido Pleßmann committed Jan 7, 2021
    Configuration menu
    Copy the full SHA
    d34ef5d View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2021

  1. Configure revision script filenames

    Just using the revision id doesn't convey too much information about the
    script when looking at the filename. It especially doesn't convey any
    information with respect to the relative ordering among scripts. Adding
    the time stamp to the script's filename at least gives an idea about
    this and it also gives a hint on how recent the change is. One even
    might use this information to figure how whether old migrations should
    be pruned in the future. When one talks about timestamps, it's of course
    good to always have them in UTC. That way, they are comparable even
    without specifying time zone information, which would just add
    unnecessary noise here.
    I also changed the character separating the different parts of the
    filename from underscores to periods, because those are already used to
    separate the suffix from the rest of the filename and using a smaller
    number if different characters is easier on the eyes, IMHO.
    Last but not least, I relaxed the length at which the revision message
    to 53 characters. This is the maximum length of the first line of a Git
    commit message, i.e. 50 characters, with a little bit of leeway.
    gnn committed Jan 8, 2021
    Configuration menu
    Copy the full SHA
    c4504d7 View commit details
    Browse the repository at this point in the history
  2. Reformat newly generated migration scripts

    Since we use `black` and `isort` to ensure a consistent style in our
    Python code, running newly generated migration scripts through them
    ensures that not code violating our style guide gets automatically
    generated.
    Note that using Alembic's `console_scripts` entry point type
    unfortunately only works with Python console scripts installed in the
    same virtual environment as the `alembic` in use. If that starts to
    become a problem, we have to implement a custom write hook in "env.py"
    which calls arbitrary external programs.
    gnn committed Jan 8, 2021
    Configuration menu
    Copy the full SHA
    97980b9 View commit details
    Browse the repository at this point in the history
  3. Prefix migration scripts with a sequential number

    Since this moves the migration script, this hook has to be run after
    every other hook. Everything else should be explained in the comments
    and docstrings.
    We also have to set `revision_environment` to `true` because the
    "env.py" file contains our hook definition which always has to be loaded
    in order to move a generated migration script.
    gnn committed Jan 8, 2021
    Configuration menu
    Copy the full SHA
    8e93c7b View commit details
    Browse the repository at this point in the history
  4. Add the base migration

    The file was generated using the command
    
    ```
    alembic revision -m "Do nothing for the base migration"
    ```
    
    inside the "src/egon/data/alembic" directory. This is important, as this
    directory contains the "alembic.ini" file which Alembic needs to
    operate.
    
    As the docstring states, this migration is initially empty in order to
    provide a blank slate to build upon and to go back to.
    gnn committed Jan 8, 2021
    Configuration menu
    Copy the full SHA
    db0f9b4 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    df82f29 View commit details
    Browse the repository at this point in the history
  6. Add a migration creating database extensions

    The extensions in question are hstore and postgis. These are currently
    added via a Docker entry point. Since this already created problems and
    since it also wouldn't work when not using Docker, moving it into a
    migration is both, a good proof of concept and a fix for an open issue.
    gnn committed Jan 8, 2021
    Configuration menu
    Copy the full SHA
    00f4af3 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2021

  1. Run all migrations during initdb

    Since the migrations currently also contain one that creates the
    database extensions we need, this also fixes #76 and the Docker entry
    point is removed with this commit.
    gnn committed Jan 10, 2021
    Configuration menu
    Copy the full SHA
    c30bc1b View commit details
    Browse the repository at this point in the history
  2. Add the OpenStreetMap migration script

    The script was automatically generated by setting `target_metadata` in
    "src/egon/data/alembic/env.py" to `egon.data.orm.openstreetmap.metadata`
    and running
    
    ```
    alembic revision \
      --autogenerate \
      -m "Create the OpenStreetMap database structure"
    ```
    
    The script needs some adjustments though, which I'm putting into a
    separate commit for documentation purposes.
    gnn committed Jan 10, 2021
    Configuration menu
    Copy the full SHA
    853bc1c View commit details
    Browse the repository at this point in the history