-
Notifications
You must be signed in to change notification settings - Fork 455
Add capability to register database migration from the application/module layer - Closes #3197 #3701
Conversation
Separate framework migrations from app/modules migrations in order to be able to migrate code and database all together
framework/src/modules/network/migrations/sql/20160723182900_create_schema.sql
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good; I would create following issues:
- Add logs showing the migration name and namespace
- Make migrations version dependent so we can plan for the case we want to revert a complete set of migrations (maybe this calls for a more in depth talk about migrations strategy)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please adjust all SQL file structure based on the feedback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently the migration entity have following interface methods:
- hasInternalMigrationsTable
- hasMigrationsTable
- getLastInternalMigrationId
- readPendingInternalMigrations
- readPending
- applyPendingInternalMigration
- applyPendingMigration
- applyInternal
- applyList
Which clearly shows a lot of duplicate stuff happening in there. First question What is internal migration? It just the normal migration in the framework scope.
So instead of creating a new concept and a lot of duplicate code, implement the migration process with the following conditions:
- Choose a namespace for framework migrations e.g.
lisk-framework
- Register framework migrations in application constructor.
- While migrating, implement the logic that migrations from the framework should be executed first. That's what you are currently trying to do.
The other solution would be (which I prefer):
- Don't treat that framework migrations (which are just schema definition of migrations table) as migration itself.
- Write idempotent SQL for what you currently have as framework migration
- Execute that SQL every time before running the actual migrations
Look at any other famous and established frameworks in this domain e.g. ActiveRecord, Eloquent, Doctrine you will notice none treat creating migrations table as a migration itself.
And one more thing, the migration you created for framework, containing:
UPDATE "migrations" SET namespace = 'network' WHERE id IN ('20161016133824', '20170113181857', '20171207000001', '20171227155620', '20180205000002', '20180327170000', '20181106000006', '20190103000001', '20190111111557');
UPDATE "migrations" SET namespace = 'chain' WHERE namespace = 'undefined';
These should be actually be splitted to their own specific modules. And how you do is to create a migration with id (timestamp) which executed first. Then rest of the migration from that module will be identified properly.
@nazarhussain I only agree with your last point (separate the |
Actually |
Remove internal_migrations table as per review request
@nazarhussain I refactored the code to get rid of the table which kept the framework migrations. Let me know what do you reckon. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lsilvs All seems good, just two small things to see as we discussed.
- Don't set 'undefined' as a value rather set the constraint after setting namspaces
- Rename the file 'migration.js' to 'migration_entity.js'
b9ababa
to
f9cd530
Compare
What was the problem?
All migrations were being handled by Chain Module and there was no way to run application/modules specific migrations.
How did I fix it?
Migrations logic was moved to Controller and now it has its own database table called
internal_migrations
to isolate it from app/modules migrations and allow updating database/code together.registerMigrations
was introduced in Application class in order to pass all registered migrations during Controller loading.Chain-specific migrations was moved to ChainModule class.
Network-specific migrations was moved to NetworkModule class.
Migration-specific migrations was moved to Framework.
Because
20160723182900_create_schema.sql
file had migrations belonging tomigrations
,chain
andnetwork
, it was split into 3 files and another migration (20190521000002_migrate_migrations_table.sql
) was created to take care of the split.See bellow current list of migrations and which module it belongs to:
How to test it?
dropdb lisk_dev
createdb lisk_dev
node framework/test/test_app/index.js -n devnet
Review checklist