New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rockstor update doesn't fail if db migration fails. #1332

Closed
schakrava opened this Issue May 31, 2016 · 0 comments

Comments

Projects
None yet
1 participant
@schakrava
Member

schakrava commented May 31, 2016

Rockstor updates are distributed as rpms. As part of the rpm/yum update, db migrations are run if there are any schema changes. eg: recent network teaming related schema changes.

Rarely, db migration step of the update process fails. I have not seen it myself, but I know of at least 2 reports so far and I believe they are both running Rockstor off USB. For some reason, postgres barfs and we have updated rockstor package without db migration applied.

The fix usually is to run the db migration step as follows:

  1. stop rockstor -- sytecmtcl stop rocsktor
  2. run first migration -- /opt/rockstor/bin/django migrate storageadmin --database=default --noinput
  3. run second migration -- /opt/rockstor/bin/django migrate smart_manager --database=smart_manager --noinput

If steps 2 or 3 throw errors, then there are bigger problems. But for this particular bug, they should run without any errors or much of any useful information as output.

  1. start rockstor again -- systemctl start rockstor

We should tighten the yum update process to catch any db migration errors and put some retry and error handling logic. Also note that this may be best done after we update django #1237

@schakrava schakrava added the bug label May 31, 2016

@schakrava schakrava added this to the Pinnacles milestone May 31, 2016

@schakrava schakrava self-assigned this Jun 13, 2016

@schakrava schakrava closed this in 473eabb Apr 9, 2017

schakrava added a commit that referenced this issue Apr 9, 2017

Merge pull request #1694 from schakrava/db_migration_enhancement
Consolidate db migration logic into initrock. Fixes #1332
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment