New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New config option to disable validateMigrationsList() #3448
New config option to disable validateMigrationsList() #3448
Conversation
This is exactly the reason why I have been slightly against of having this feature included to knex. I haven't found many cases where having the switch actually would make sense and would not cause user's system to be inconsistent and broken. I know this is a bit harsh, but to be honest your process of using staging environment is horribly wrong. It is inviting strange errors, which will appear only in staging environment and it doesn't make sure that multiple merged features actually works correctly when combined. Effectively staging environment should mirror final production environment to verify that everything is fine before putting it to production. If you need to test out every branch first separately in staging environment with shared database you should also clean up all changes done to DB when you revert the certain features code form the environment.
Reseting database is the solution, you just need to take SQL dump of the database from the stage where it was before starting to test multiple feature branches. Then before or after running tests for each branch just restore database state from that SQL dump. That way you will always have consistent database state with code. If you just run bunch of migrations from various features can make completely working feature to seem like broken one (just to name one: migrations also can remove data that other branch might need). Anyhow IMO that kind of testing should be done in testing environment instead of staging. In staging you should apply each feature on top of each other the same way it will be done finally to the production system. |
Looks like in my description i've used a term "stage environment" in a misleading way. A little bit more info about our setup: I agree that sharing a database is kinda asking for troubles and not ideal, but we can't spin up a new database for every feature branch due to infrastracture limitations. We can probably make a separate database for
Reseting brings new problems to the table:
We try to make all the features (migrations) backward compatible as much as possible. It also helps with AB tests and rolling back releases from production in case of accident. As a result we prefer approach from this PR. |
It could be integrated as part of test command and all services could be stored / restored by the same script.
Ok, so this is environment where you are hand testing stuff... so there is no tests script where one could do the save/restore etc. Well sounds like you are out of luck then...
Good luck! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR is missing tests and documentation
@elhigu done Looks like build is failing due to totally unrelated thing
|
@arrilot Pull from master, that test should be fixed now (there is a failing CLI test now, feel free to ignore it) |
Done, thanks |
@elhigu here #1747 (comment)
So here we go
Motivation:
We use gitflow with features in different branches and deploy them separately to stage environment.
All branches on stage environment have one common database.
As a result we constantly face the same issue:
Rolling back migrations is not a solution because not every migration can be rolled back
Reseting database is not a solution because it erases data in database aswell
Merging branch A into B is not a solution because it breaks independency of features.
We know that switching off migration order brings some inconsistency issues but they are far less important for our local/stage environments.
Some other migrators like node-pg-migrate have such feature. (https://github.com/salsita/node-pg-migrate/blob/master/docs/cli.md check-order)