Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add functionality for writing migrations. #30

Merged
merged 68 commits into from
Oct 11, 2019

Conversation

moigagoo
Copy link
Owner

@moigagoo moigagoo commented Sep 16, 2019

UPD: working on this PR allowed me to extend my knowledge on Nim metaprogramming and rewrite the internals of Norm. The code is now simpler and more powerful. Most importantly, this refactoring allowed me to implement proper migration primitives with proper SQLite and PostrgeSQL queries. So, the information under the horizontal line is no longer valid.

Here are the migration primitives I'm going to ship with this PR:

  • SQLite:

    • addColumn(Type.field) → ALTER TABLE ... ADD COLUMN ..., run after adding field to Type.
    • dropUnusedColumns() → copy currently available columns to a temp table, drop the table, rename the temp table to the original table name. This is done because SQLite can't drop columns. Run after removing a field from a type.
    • renameColumnFrom(Type.field, "oldName") → ALTER TABLE ... RENAME COLUMN ... TO ..., run after renaming a field or changing its dbCol pragma value.
    • renameTableFrom(Type, "oldName") → ALTER TABLE ... RENAME TO ..., run after renaming a type or changing its table pragma value.
  • PostgreSQL:

    • addColumn(Type.field) → ALTER TABLE ... ADD COLUMN ..., run after adding field to Type.
    • dropColumns(["field1", "field2"]) → ALTER TABLE ... DROP COLUMN IF EXISTS ... CASCADE, DROP COLUMN IF EXISTS ... CASCADE, ..., run after removing fields from a type.
    • dropUnusedColumns() → copy currently available columns to a temp table, drop the table, rename the temp table to the original table name. Run after removing a field from a type.
    • renameColumnFrom(Type.field, "oldName") → ALTER TABLE ... RENAME COLUMN ... TO ..., run after renaming a field or changing its dbCol pragma value.
    • renameTableFrom(Type, "oldName") → ALTER TABLE ... RENAME TO ..., run after renaming a type or changing its table pragma value.

Important: all migration primitives are to be used after the model has been changed to mirror the changes onto the database.

Also, I've added createTable and dropTable to create and drop individual tables.


The goal is to add a minimal set of templates to allow writing migrations.

So far, I've come to this list of actions:

  • Create table
  • Drop table
  • Copy all data from one table to another
  • Copy data from one column in one table to another column in another table

With these actions, the following kinds of migrations could be implemented:

  • Add or remove columns, rename table, change column type without data migration:

    • create tmp table from the new model
    • copy data from all columns from the old table to the tmp table
    • drop the old table
    • create a table from the new model
    • copy data from all columns from the tmp table to the new table
    • drop the tmp table
  • Rename column, change column type with data migration:

    • create tmp table from the new model
    • copy data from all columns from the old table to the tmp table
    • copy data from the original column in the old table to the renamed column in the tmp table
    • drop the old table
    • create a table from the new model
    • copy data from all columns from the tmp table to the new table
    • drop the tmp table

Norm will provide only the means to write migrations. Migration generation and application will be implemented in a separate project.

@moigagoo moigagoo changed the title Add minimal functionality for writing basic migrations. Add functionality for writing migrations. Sep 19, 2019
@moigagoo moigagoo changed the title Add functionality for writing migrations. [WIP] Add functionality for writing migrations. Sep 19, 2019
@moigagoo moigagoo merged commit 9ae6156 into develop Oct 11, 2019
@moigagoo moigagoo deleted the feature/first_step_toward_migrations branch October 11, 2019 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant