[0.13] core: Track upgrade step within each schema version #474

digitalcircuit · 2019-01-29T06:52:39Z

In short

Track last successful upgrade step (upgrade_###_XXX.sql) within each schema version
- Store in schemaupgradestep, cleared when a schema version upgrade finishes
- Allows resuming interrupted multi-step schema upgrades
- Aids debugging after the fact
- Should not add any additional failure cases
Update logging to be more explicit about what schema version fails
Remove redundant updateSchemaVersion() call
- Already handled within loop, cleans up after f10304a

Criteria	Rank	Reason
Impact	★★★ 3/3	Reduces risk of breaking database with interrupted upgrades
Risk	★★★ 3/3	Schema steps might not be stored correctly, breaking upgrades
Intrusiveness	★☆☆ 1/3	Minor changes, shouldn't interfere with other pull requests

This is applied to master here.

Rationale

Single query schema versions were handled in pull request #364, and much of the same rational applies here.

When running upgrade queries, track the schema step filename for each intermediate step. This ensures that any interrupted upgrades have a greater chance of resuming correctly after core restart.

Almost all databases make single queries atomic (fully works or fully fails, no partial), and storing progress after each query makes upgrade interruptions much more likely to leave the database in a resumable intermediate schema version.

Examples

Simulated interruptions using debugger

NOTE₁: The storage backend will be disabled on the first error, falling back to an unconfigured state. You will need to restart the Quassel core to resume the upgrade. This feature/issue exists in 0.13 and earlier.

NOTE₂: All situations were tested in 0.13 using a rollup of 0.13 backport pull requests to save testing time.

Bundled PRs: #458, #462, #469, #471, #472, and the current PR. The current master branch was briefly tested before.

Good scenario 1: Some upgrade queries complete in one version

In abstractsqlstorage.cpp, set a breakpoint on setSchemaVersionUpgradeStep(queryResource.queryFilename);
Run upgrade until one instruction past this step
Stop debugger (simulating terminating core)
Verify coreinfo's schemaupgradestep gets set correctly
- E.g. upgrade_000_create_sender_tmp (from SQLite version 29)
- E.g. upgrade_010_alter_sender_64bit_ids (from PostgreSQL version 29)
Run, testing if migration resumes

🆕 New to this PR

Example log (SQLite)

2019-01-29 22:57:51 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
[stopped]
2019-01-29 22:58:27 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 22:58:27 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_000_create_sender_tmp)
2019-01-29 22:58:57 [Info ] Installed database schema successfully upgraded to version 31. 
2019-01-29 22:58:57 [Info ] SQLite storage backend is ready. Schema version: 31

Example log (PostgreSQL)

2019-01-29 23:31:41 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
[stopped]
2019-01-29 23:32:27 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:32:27 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_010_alter_sender_64bit_ids) 
2019-01-29 23:32:27 [Info ] Installed database schema successfully upgraded to version 29. 
2019-01-29 23:32:27 [Info ] PostgreSQL storage backend is ready. Schema version: 29

Good scenario 2: All upgrade queries complete in one version

In abstractsqlstorage.cpp, set a breakpoint on if (!updateSchemaVersion(ver, true)) {
Run upgrade until one instruction past this step
Stop debugger (simulating terminating core)
Verify coreinfo's schemaupgradestep gets cleared to empty string
Verify coreinfo's schemaversion gets incremented correctly
Run, testing if migration resumes

🏁 In 0.13 stable

Example log (SQLite)

2019-01-29 23:01:24 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[stopped]
2019-01-29 23:01:57 [Info ] Installed database schema (version 18) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 23:02:17 [Info ] Installed database schema successfully upgraded to version 31. 
2019-01-29 23:02:17 [Info ] SQLite storage backend is ready. Schema version: 31

Example log (PostgreSQL)

2019-01-29 23:33:53 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 29...  This may take a while for major upgrades.
[stopped]
2019-01-29 23:34:27 [Info ] Installed database schema (version 18) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:34:27 [Info ] Installed database schema successfully upgraded to version 29. 
2019-01-29 23:34:27 [Info ] PostgreSQL storage backend is ready. Schema version: 29

Bad scenario 3: Some upgrade queries complete in one version, but upgrade step not set

In abstractsqlstorage.cpp, set a breakpoint on setSchemaVersionUpgradeStep(queryResource.queryFilename);
Run upgrade, stopping on this instruction, before it is run
Stop debugger (simulating terminating core at a very inopportune time)
Verify coreinfo's schemaupgradestep gets set to previous step correctly
- E.g. starting off with empty string '``' (from SQLite/PostgreSQL version 29)
Run, testing if migration fails with informative message
Manually set coreinfo's schemaupgradestep to the last successful step
- E.g. upgrade_000_create_sender_tmp (from SQLite version 29)
- E.g. upgrade_000_update_sender_add_realname (from PostgreSQL version 27)
Run, testing if migration resumes

☑️ Broken before this PR, manual recovery made easier

Example log (SQLite)

2019-01-29 23:03:50 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
[stopped]
2019-01-29 23:05:35 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 23:05:35 [Error] unhandled Error in QSqlQuery!
2019-01-29 23:05:35 [Error]                   last Query:
 CREATE TABLE sender_tmp (senderid INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, sender TEXT NOT NULL, realname TEXT, avatarurl TEXT);
[...]
2019-01-29 23:05:35 [Error]                   DB Message: table sender_tmp already exists
2019-01-29 23:05:35 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 29 failed (step: upgrade_000_create_sender_tmp).
2019-01-29 23:05:35 [Warn ] Upgrade failed...
[stopped, manual correction applied]
2019-01-29 23:11:19 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 23:11:19 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_000_create_sender_tmp) 
2019-01-29 23:11:31 [Info ] Installed database schema successfully upgraded to version 31. 
2019-01-29 23:11:31 [Info ] SQLite storage backend is ready. Schema version: 31

Example log (PostgreSQL)

2019-01-29 23:37:06 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 29...  This may take a while for major upgrades.
[stopped]
2019-01-29 23:37:34 [Info ] Installed database schema (version 26) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:37:34 [Error] unhandled Error in QSqlQuery!
2019-01-29 23:37:34 [Error]                   last Query:
 ALTER TABLE sender ADD realname TEXT NULL;
[...]
2019-01-29 23:37:34 [Error]                   DB Message: ERROR:  column "realname" of relation "sender" already exists
(42701)
2019-01-29 23:37:34 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 27 failed (step: upgrade_000_update_sender_add_realname).
2019-01-29 23:37:34 [Warn ] Upgrade failed...
[stopped, manual correction applied]
2019-01-29 23:39:11 [Info ] Installed database schema (version 26) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:39:11 [Info ] Resuming interrupted upgrade for schema version 27 (last step: upgrade_000_update_sender_add_realname) 
2019-01-29 23:39:11 [Info ] Installed database schema successfully upgraded to version 29. 
2019-01-29 23:39:11 [Info ] PostgreSQL storage backend is ready. Schema version: 29

Bad scenario 4: An upgrade query fails in one version

Modify a query in a multi-step schema upgrade, deliberately introducing an error
- E.g. in 29/upgrade_010_copy_sender_sender_tmp.sql, INSERT INTO SELECT;
- E.g. in 29/upgrade_050_alter_buffer_64bit_ids.sql, INSERT INTO SELECT;
Run upgrade until error
Verify coreinfo's schemaupgradestep gets set correctly
- E.g. upgrade_000_create_sender_tmp (from SQLite version 29)
- E.g. upgrade_050_alter_buffer_64bit_ids (from PostgreSQL version 29)
Revert the deliberate error introduced in step 1
Run, testing if migration resumes

☑️ Broken before this PR, manual recovery made easier

This mimics the upgrade case for PostgreSQL 8.4, where a CREATE LANGUAGE plpgsql; command will need manually run, then the upgrade can resume automatically after that.

Example log (SQLite)

2019-01-29 23:13:16 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 23:13:20 [Error] unhandled Error in QSqlQuery!
2019-01-29 23:13:20 [Error]                   last Query:
 INSERT INTO SELECT;
[...]
2019-01-29 23:13:20 [Error]                   DB Message: near "SELECT": syntax error
2019-01-29 23:13:20 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 29 failed (step: upgrade_010_copy_sender_sender_tmp).
2019-01-29 23:13:20 [Warn ] Upgrade failed...
[stopped, error reverted]
2019-01-29 23:14:20 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 23:14:20 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_000_create_sender_tmp) 
2019-01-29 23:14:32 [Info ] Installed database schema successfully upgraded to version 31. 
2019-01-29 23:14:32 [Info ] SQLite storage backend is ready. Schema version: 31

Example log (PostgreSQL)

2019-01-29 23:49:47 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:49:47 [Error] unhandled Error in QSqlQuery!
2019-01-29 23:49:47 [Error]                   last Query:
 INSERT INTO SELECT;
[...]
2019-01-29 23:49:47 [Error]                   DB Message: ERROR:  syntax error at or near "SELECT"
LINE 1: INSERT INTO SELECT;
                    ^
(42601)
2019-01-29 23:49:47 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 29 failed (step: upgrade_050_alter_buffer_64bit_ids).
2019-01-29 23:49:47 [Warn ] Upgrade failed...
[stopped, error reverted]
2019-01-29 23:50:34 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:50:34 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_010_alter_sender_64bit_ids) 
2019-01-29 23:50:34 [Info ] Installed database schema successfully upgraded to version 29. 
2019-01-29 23:50:34 [Info ] PostgreSQL storage backend is ready. Schema version: 29

Bad scenario 5: Some upgrade queries complete in one version, Quassel core SQL resources renamed

In abstractsqlstorage.cpp, set a breakpoint on setSchemaVersionUpgradeStep(queryResource.queryFilename);
Run upgrade until one instruction past this step
Stop debugger (simulating terminating core at a very inopportune time)
Manually set coreinfo's schemaupgradestep to an invalid step
- E.g. upgrade_000_DOES_NOT_EXIST
Run, testing if migration fails with informative message

☑️ Broken before this PR, manual recovery made easier

Example log (SQLite)

2019-01-29 23:16:40 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
[stopped, error introduced]
2019-01-29 23:18:11 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 23:18:11 [Error] Unable to resume interrupted upgrade in Logging Backend!  Missing upgrade step in schema version 29 (expected step: upgrade_000_DOES_NOT_EXIST)
2019-01-29 23:18:11 [Warn ] Upgrade failed...

Example log (PostgreSQL)

2019-01-29 23:40:55 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
[stopped, error introduced]
2019-01-29 23:44:34 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 29...  This may take a while for major upgrades. 
2019-01-29 23:44:34 [Error] Unable to resume interrupted upgrade in Logging Backend!  Missing upgrade step in schema version 29 (expected step: upgrade_000_DOES_NOT_EXIST)
2019-01-29 23:44:34 [Warn ] Upgrade failed...

Non-deterministic interruption via `kill`

Steps

Show current schema version
Run core, forcibly (kill -9) interrupting upgrade
Run core, letting it finish

Statistics

Quassel core forcibly interrupted (kill -9) 45 times during 17 → 31 schema upgrade
- 8 manual recoveries required
- 30 automatic upgrade resumes
- 4 instances of queries taking longer than 0.1 seconds, manually run (auto-resume worked)
- 45 total upgrade attempts
Roughly 17.8% rate of error leading to manual recovery if someone kill -9's every 0.1 seconds
- Worst-case, improves over 66.7% manual recovery in same conditions (realistically much lower)
- No situations made worse

Commands run
run-profile.sh is a script to simplify managing multiple testing copies of Quassel and is equivalent to calling the right quasselcore with the right options

Quassel and SQLite commands run for testing and manual recovery

sqlite> select * from coreinfo;
schemaversion|17

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 1]
sqlite> select * from coreinfo;
schemaversion|17
schemaupgradestep|upgrade_000_alter_quasseluser_add_passwordversion

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 2]
sqlite> select * from coreinfo;
schemaversion|18
schemaupgradestep|upgrade_000_alter_ircserver_add_sslverify

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 3]
sqlite> select * from coreinfo;
schemaversion|19
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 4]
sqlite> select * from coreinfo;
schemaversion|19
schemaupgradestep|

[Manual recovery required: bad scenario 3]
[Note: in this case, the query finished but step wasn't set, so you can just copy the mentioned step from the error message - easier than before, where you'd have to manually run the remaining queries]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_alter_network_add_ratelimit_usecustom' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|19
schemaupgradestep|upgrade_000_alter_network_add_ratelimit_usecustom

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 5]
sqlite> select * from coreinfo;
schemaversion|19
schemaupgradestep|upgrade_001_alter_network_add_ratelimit_burstsize

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 6]
sqlite> select * from coreinfo;
schemaversion|19
schemaupgradestep|upgrade_002_alter_network_add_ratelimit_delay

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 7]
sqlite> select * from coreinfo;
schemaversion|19
schemaupgradestep|upgrade_003_alter_network_add_ratelimit_unlimited

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 8]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 9]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_alter_buffer_add_lastmsgid' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_000_alter_buffer_add_lastmsgid

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 10]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_001_update_buffer_set_lastmsgid

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 11]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_002_update_buffer_fix_lastseenmsgid_over_lastmsgid

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 12]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_003_create_table_buffer_new

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 13]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_004_insert_into_buffer_new_from_buffer

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 14]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_005_drop_table_buffer

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 15]
sqlite> select * from coreinfo;
schemaversion|20
schemaupgradestep|upgrade_006_alter_table_buffer_new_rename_to_buffer

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 16]
sqlite> select * from coreinfo;
schemaversion|21
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 17]
sqlite> select * from coreinfo;
schemaversion|21
schemaupgradestep|upgrade_001_add_trigger_backlog_lastmsgid_update_direct_update

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 18]
sqlite> select * from coreinfo;
schemaversion|22
schemaupgradestep|upgrade_000_alter_quasseluser_add_authenticator

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 19]
sqlite> select * from coreinfo;
schemaversion|23
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 20]
sqlite> select * from coreinfo;
schemaversion|23
schemaupgradestep|

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_create_senderprefixes' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|23
schemaupgradestep|upgrade_000_create_senderprefixes

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 21]
sqlite> select * from coreinfo;
schemaversion|24
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 22]
sqlite> select * from coreinfo;
schemaversion|24
schemaupgradestep|

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_alter_buffer_add_bufferactivity' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|24
schemaupgradestep|upgrade_000_alter_buffer_add_bufferactivity

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 23]
sqlite> select * from coreinfo;
schemaversion|25
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 24]
sqlite> select * from coreinfo;
schemaversion|25
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 25]
sqlite> select * from coreinfo;
schemaversion|25
schemaupgradestep|

[At this point, the migration query takes long enough to not finish within 0.1 seconds.  Migration step is correctly auto-resumed, and thus manually run to completion.]

sqlite> CREATE INDEX backlog_buffer_msg_idx ON backlog (bufferid, messageid);
sqlite> UPDATE coreinfo SET value = 26 WHERE key = 'schemaversion';
sqlite> select * from coreinfo;
schemaversion|26
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 26]
sqlite> select * from coreinfo;
schemaversion|26
schemaupgradestep|upgrade_000_alter_buffer_add_cipher

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 27]
sqlite> select * from coreinfo;
schemaversion|27
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 28]
sqlite> select * from coreinfo;
schemaversion|27
schemaupgradestep|

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_alter_buffer_add_highlightcount' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|27
schemaupgradestep|upgrade_000_alter_buffer_add_highlightcount

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 29]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 30]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_create_sender_tmp' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_000_create_sender_tmp

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 31]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_000_create_sender_tmp

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 32]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_000_create_sender_tmp

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 33]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_000_create_sender_tmp

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_010_copy_sender_sender_tmp' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_010_copy_sender_sender_tmp

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 34]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_010_copy_sender_sender_tmp

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 35]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_010_copy_sender_sender_tmp

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 36]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_010_copy_sender_sender_tmp

[At this point, the migration query takes long enough to not finish within 0.1 seconds.  Migration step is correctly auto-resumed, and thus manually run to completion.]

sqlite> DROP TABLE sender;
sqlite> UPDATE coreinfo SET value = 'upgrade_020_drop_sender' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_020_drop_sender

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 36]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_030_rename_sender_tmp_sender

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 37]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_030_rename_sender_tmp_sender

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 38]
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_030_rename_sender_tmp_sender

[At this point, the migration query takes long enough to not finish within 0.1 seconds.  Migration step is correctly auto-resumed, and thus manually run to completion.]

sqlite> CREATE UNIQUE INDEX sender_index ON sender(sender, realname, avatarurl);
sqlite> UPDATE coreinfo SET value = 'upgrade_040_update_sender_add_realname_avatarurl' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|28
schemaupgradestep|upgrade_040_update_sender_add_realname_avatarurl

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 39]
sqlite> select * from coreinfo;
schemaversion|29
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 40]
sqlite> select * from coreinfo;
schemaversion|29
schemaupgradestep|

[Manual recovery required: bad scenario 3]
sqlite> UPDATE coreinfo SET value = 'upgrade_000_create_corestate' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|29
schemaupgradestep|upgrade_000_create_corestate

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 41]
sqlite> select * from coreinfo;
schemaversion|30
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 42]
sqlite> select * from coreinfo;
schemaversion|30
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 43]
sqlite> select * from coreinfo;
schemaversion|30
schemaupgradestep|

[At this point, the migration query takes long enough to not finish within 0.1 seconds.  Migration step is correctly auto-resumed, and thus manually run to completion.]

sqlite> UPDATE backlog SET time = time * 1000;
sqlite> UPDATE coreinfo SET value = 'upgrade_000_update_buffer_set_time_extended' WHERE key = 'schemaupgradestep';
sqlite> select * from coreinfo;
schemaversion|30
schemaupgradestep|upgrade_000_update_buffer_set_time_extended

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 44]
sqlite> select * from coreinfo;
schemaversion|31
schemaupgradestep|

$ (sleep 0.1 && pkill -9 quasselcore ) & \
  ./run-profile.sh master core local
[terminated: 45]
sqlite> select * from coreinfo;
schemaversion|31
schemaupgradestep|

$ ./run-profile.sh master core local

sqlite> select * from coreinfo;
schemaversion|31
schemaupgradestep|

Interrupted upgrade log

Quassel log from interrupted upgrades

2019-01-29 18:30:15 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 1]
2019-01-29 18:31:04 [Info ] Installed database schema (version 17) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:31:04 [Info ] Resuming interrupted upgrade for schema version 18 (last step: upgrade_000_alter_quasseluser_add_passwordversion)
[terminated: 2]
2019-01-29 18:32:06 [Info ] Installed database schema (version 18) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:32:06 [Info ] Resuming interrupted upgrade for schema version 19 (last step: upgrade_000_alter_ircserver_add_sslverify)
[terminated: 3]
2019-01-29 18:33:47 [Info ] Installed database schema (version 19) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:33:47 [Error] unhandled Error in QSqlQuery!
2019-01-29 18:33:47 [Error]                   last Query:
 ALTER TABLE network ADD COLUMN usecustomessagerate INTEGER NOT NULL DEFAULT 0
2019-01-29 18:33:47 [Error]               executed Query: 
2019-01-29 18:33:47 [Error]                 bound Values: 
2019-01-29 18:33:47 [Error]                 Error Number: 1
2019-01-29 18:33:47 [Error]                Error Message: duplicate column name: usecustomessagerate Unable to execute statement
2019-01-29 18:33:47 [Error]               Driver Message: Unable to execute statement
2019-01-29 18:33:47 [Error]                   DB Message: duplicate column name: usecustomessagerate
2019-01-29 18:33:47 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 20 failed (step: upgrade_000_alter_network_add_ratelimit_usecustom).
2019-01-29 18:33:47 [Warn ] Upgrade failed...
[terminated: 4]
[Manual recovery required: bad scenario 3]
2019-01-29 18:38:46 [Info ] Installed database schema (version 19) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:38:46 [Info ] Resuming interrupted upgrade for schema version 20 (last step: upgrade_000_alter_network_add_ratelimit_usecustom)
[terminated: 5]
2019-01-29 18:39:59 [Info ] Installed database schema (version 19) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:39:59 [Info ] Resuming interrupted upgrade for schema version 20 (last step: upgrade_001_alter_network_add_ratelimit_burstsize)
[terminated: 6]
2019-01-29 18:43:31 [Info ] Installed database schema (version 19) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:43:31 [Info ] Resuming interrupted upgrade for schema version 20 (last step: upgrade_002_alter_network_add_ratelimit_delay)
[terminated: 7]
2019-01-29 18:45:36 [Info ] Installed database schema (version 19) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:45:36 [Info ] Resuming interrupted upgrade for schema version 20 (last step: upgrade_003_alter_network_add_ratelimit_unlimited)
[terminated: 8]
2019-01-29 18:46:08 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 18:46:08 [Error] unhandled Error in QSqlQuery!
2019-01-29 18:46:08 [Error]                   last Query:
 ALTER TABLE buffer
ADD COLUMN lastmsgid INTEGER NOT NULL DEFAULT 0;
2019-01-29 18:46:08 [Error]               executed Query: 
2019-01-29 18:46:08 [Error]                 bound Values: 
2019-01-29 18:46:08 [Error]                 Error Number: 1
2019-01-29 18:46:08 [Error]                Error Message: duplicate column name: lastmsgid Unable to execute statement
2019-01-29 18:46:08 [Error]               Driver Message: Unable to execute statement
2019-01-29 18:46:08 [Error]                   DB Message: duplicate column name: lastmsgid
2019-01-29 18:46:08 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 21 failed (step: upgrade_000_alter_buffer_add_lastmsgid).
2019-01-29 18:46:08 [Warn ] Upgrade failed...
[terminated: 9]
[Manual recovery required: bad scenario 3]
2019-01-29 19:11:44 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:11:44 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_000_alter_buffer_add_lastmsgid)
[terminated: 10]
2019-01-29 19:12:14 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:12:14 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_001_update_buffer_set_lastmsgid)
[terminated: 11]
2019-01-29 19:12:44 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:12:44 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_002_update_buffer_fix_lastseenmsgid_over_lastmsgid)
[terminated: 12]
2019-01-29 19:14:28 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:14:28 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_003_create_table_buffer_new)
[terminated: 13]
2019-01-29 19:15:15 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:15:15 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_004_insert_into_buffer_new_from_buffer)
[terminated: 14]
2019-01-29 19:15:45 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:15:45 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_005_drop_table_buffer)
[terminated: 15]
2019-01-29 19:16:14 [Info ] Installed database schema (version 20) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:16:14 [Info ] Resuming interrupted upgrade for schema version 21 (last step: upgrade_006_alter_table_buffer_new_rename_to_buffer)
[terminated: 16]
2019-01-29 19:16:42 [Info ] Installed database schema (version 21) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 17]
2019-01-29 19:17:20 [Info ] Installed database schema (version 21) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:17:20 [Info ] Resuming interrupted upgrade for schema version 22 (last step: upgrade_001_add_trigger_backlog_lastmsgid_update_direct_update)
[terminated: 18]
2019-01-29 19:17:53 [Info ] Installed database schema (version 22) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:17:53 [Info ] Resuming interrupted upgrade for schema version 23 (last step: upgrade_000_alter_quasseluser_add_authenticator)
[terminated: 19]
2019-01-29 19:19:01 [Info ] Installed database schema (version 23) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:19:01 [Error] unhandled Error in QSqlQuery!
2019-01-29 19:19:01 [Error]                   last Query:
 ALTER TABLE backlog
ADD COLUMN senderprefixes TEXT;
2019-01-29 19:19:01 [Error]               executed Query: 
2019-01-29 19:19:01 [Error]                 bound Values: 
2019-01-29 19:19:01 [Error]                 Error Number: 1
2019-01-29 19:19:01 [Error]                Error Message: duplicate column name: senderprefixes Unable to execute statement
2019-01-29 19:19:01 [Error]               Driver Message: Unable to execute statement
2019-01-29 19:19:01 [Error]                   DB Message: duplicate column name: senderprefixes
2019-01-29 19:19:01 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 24 failed (step: upgrade_000_create_senderprefixes).
2019-01-29 19:19:01 [Warn ] Upgrade failed...
[terminated: 20]
[Manual recovery required: bad scenario 3]
2019-01-29 19:20:42 [Info ] Installed database schema (version 23) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:20:42 [Info ] Resuming interrupted upgrade for schema version 24 (last step: upgrade_000_create_senderprefixes)
[terminated: 21]
2019-01-29 19:21:16 [Info ] Installed database schema (version 24) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:21:16 [Error] unhandled Error in QSqlQuery!
2019-01-29 19:21:16 [Error]                   last Query:
 ALTER TABLE buffer
ADD COLUMN bufferactivity integer NOT NULL DEFAULT 0
2019-01-29 19:21:16 [Error]               executed Query: 
2019-01-29 19:21:16 [Error]                 bound Values: 
2019-01-29 19:21:16 [Error]                 Error Number: 1
2019-01-29 19:21:16 [Error]                Error Message: duplicate column name: bufferactivity Unable to execute statement
2019-01-29 19:21:16 [Error]               Driver Message: Unable to execute statement
2019-01-29 19:21:16 [Error]                   DB Message: duplicate column name: bufferactivity
2019-01-29 19:21:16 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 25 failed (step: upgrade_000_alter_buffer_add_bufferactivity).
2019-01-29 19:21:16 [Warn ] Upgrade failed...
[terminated: 22]
[Manual recovery required: bad scenario 3]
2019-01-29 19:22:55 [Info ] Installed database schema (version 24) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:22:55 [Info ] Resuming interrupted upgrade for schema version 25 (last step: upgrade_000_alter_buffer_add_bufferactivity)
[terminated: 23]
2019-01-29 19:23:34 [Info ] Installed database schema (version 25) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 24]
2019-01-29 19:24:00 [Info ] Installed database schema (version 25) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 25]
2019-01-29 19:27:05 [Info ] Installed database schema (version 26) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 26]
2019-01-29 19:27:39 [Info ] Installed database schema (version 26) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:27:39 [Info ] Resuming interrupted upgrade for schema version 27 (last step: upgrade_000_alter_buffer_add_cipher)
[terminated: 27]
2019-01-29 19:28:10 [Info ] Installed database schema (version 27) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:28:10 [Error] unhandled Error in QSqlQuery!
2019-01-29 19:28:10 [Error]                   last Query:
 ALTER TABLE buffer
ADD COLUMN highlightcount integer NOT NULL DEFAULT 0
2019-01-29 19:28:10 [Error]               executed Query: 
2019-01-29 19:28:10 [Error]                 bound Values: 
2019-01-29 19:28:10 [Error]                 Error Number: 1
2019-01-29 19:28:10 [Error]                Error Message: duplicate column name: highlightcount Unable to execute statement
2019-01-29 19:28:10 [Error]               Driver Message: Unable to execute statement
2019-01-29 19:28:10 [Error]                   DB Message: duplicate column name: highlightcount
2019-01-29 19:28:10 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 28 failed (step: upgrade_000_alter_buffer_add_highlightcount).
2019-01-29 19:28:10 [Warn ] Upgrade failed...
[terminated: 28]
[Manual recovery required: bad scenario 3]
2019-01-29 19:29:42 [Info ] Installed database schema (version 27) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:29:42 [Info ] Resuming interrupted upgrade for schema version 28 (last step: upgrade_000_alter_buffer_add_highlightcount)
[terminated: 29]
2019-01-29 19:30:53 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:30:53 [Error] unhandled Error in QSqlQuery!
2019-01-29 19:30:53 [Error]                   last Query:
 CREATE TABLE sender_tmp (senderid INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, sender TEXT NOT NULL, realname TEXT, avatarurl TEXT);
2019-01-29 19:30:53 [Error]               executed Query: 
2019-01-29 19:30:53 [Error]                 bound Values: 
2019-01-29 19:30:53 [Error]                 Error Number: 1
2019-01-29 19:30:53 [Error]                Error Message: table sender_tmp already exists Unable to execute statement
2019-01-29 19:30:53 [Error]               Driver Message: Unable to execute statement
2019-01-29 19:30:53 [Error]                   DB Message: table sender_tmp already exists
2019-01-29 19:30:53 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 29 failed (step: upgrade_000_create_sender_tmp).
2019-01-29 19:30:53 [Warn ] Upgrade failed...
[terminated: 30]
[Manual recovery required: bad scenario 3]
2019-01-29 19:32:26 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:32:26 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_000_create_sender_tmp)
[terminated: 31]
2019-01-29 19:33:20 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:33:20 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_000_create_sender_tmp)
[terminated: 32]
2019-01-29 19:36:32 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:36:32 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_000_create_sender_tmp) 
2019-01-29 19:36:32 [Error] unhandled Error in QSqlQuery!
2019-01-29 19:36:32 [Error]                   last Query:
 INSERT INTO sender_tmp SELECT senderid, sender, NULL as realname, NULL as avatarurl FROM sender;
2019-01-29 19:36:32 [Error]               executed Query: 
2019-01-29 19:36:32 [Error]                 bound Values: 
2019-01-29 19:36:32 [Error]                 Error Number: 19
2019-01-29 19:36:32 [Error]                Error Message: UNIQUE constraint failed: sender_tmp.senderid Unable to fetch row
2019-01-29 19:36:32 [Error]               Driver Message: Unable to fetch row
2019-01-29 19:36:32 [Error]                   DB Message: UNIQUE constraint failed: sender_tmp.senderid
2019-01-29 19:36:32 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 29 failed (step: upgrade_010_copy_sender_sender_tmp).
2019-01-29 19:36:32 [Warn ] Upgrade failed...
[terminated: 33]
[Manual recovery required: bad scenario 3]
2019-01-29 19:41:48 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:41:48 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_010_copy_sender_sender_tmp)
[terminated: 34]
2019-01-29 19:42:18 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:42:18 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_010_copy_sender_sender_tmp)
[terminated: 35]
2019-01-29 19:46:02 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:46:02 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_010_copy_sender_sender_tmp)
[terminated: 36]
2019-01-29 19:52:39 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:52:39 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_020_drop_sender)
[terminated: 37]
2019-01-29 19:53:11 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:53:11 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_030_rename_sender_tmp_sender)
[terminated: 38]
2019-01-29 19:55:23 [Info ] Installed database schema (version 28) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:55:23 [Info ] Resuming interrupted upgrade for schema version 29 (last step: upgrade_040_update_sender_add_realname_avatarurl)
[terminated: 39]
2019-01-29 19:57:16 [Info ] Installed database schema (version 29) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:57:16 [Error] unhandled Error in QSqlQuery!
2019-01-29 19:57:16 [Error]                   last Query:
 CREATE TABLE core_state (
    key TEXT NOT NULL,
    value bytea,
    PRIMARY KEY (key)
)
2019-01-29 19:57:16 [Error]               executed Query: 
2019-01-29 19:57:16 [Error]                 bound Values: 
2019-01-29 19:57:16 [Error]                 Error Number: 1
2019-01-29 19:57:16 [Error]                Error Message: table core_state already exists Unable to execute statement
2019-01-29 19:57:16 [Error]               Driver Message: Unable to execute statement
2019-01-29 19:57:16 [Error]                   DB Message: table core_state already exists
2019-01-29 19:57:16 [Error] Unable to upgrade Logging Backend!  Upgrade query in schema version 30 failed (step: upgrade_000_create_corestate).
2019-01-29 19:57:16 [Warn ] Upgrade failed...
[terminated: 40]
[Manual recovery required: bad scenario 3]
2019-01-29 19:58:56 [Info ] Installed database schema (version 29) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 19:58:56 [Info ] Resuming interrupted upgrade for schema version 30 (last step: upgrade_000_create_corestate)
[terminated: 41]
2019-01-29 20:00:10 [Info ] Installed database schema (version 30) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 42]
2019-01-29 20:00:36 [Info ] Installed database schema (version 30) is not up to date. Upgrading to version 31...  This may take a while for major upgrades.
[terminated: 43]
2019-01-29 20:02:52 [Info ] Installed database schema (version 30) is not up to date. Upgrading to version 31...  This may take a while for major upgrades. 
2019-01-29 20:02:52 [Info ] Resuming interrupted upgrade for schema version 31 (last step: upgrade_000_update_buffer_set_time_extended) 
2019-01-29 20:02:52 [Info ] Installed database schema successfully upgraded to version 31. 
2019-01-29 20:02:52 [Info ] SQLite storage backend is ready. Schema version: 31 
2019-01-29 20:02:52 [Info ] Database authenticator is ready.
2019-01-29 20:02:52 [Info ] Listening for GUI clients on IPv6 :: port 4242 using protocol version 10 
[...]
[terminated: 44]
2019-01-29 20:03:30 [Info ] SQLite storage backend is ready. Schema version: 31 
2019-01-29 20:03:30 [Info ] Database authenticator is ready.
2019-01-29 20:03:30 [Info ] Listening for GUI clients on IPv6 :: port 4242 using protocol version 10 
[...]
[terminated: 45]
2019-01-29 20:04:43 [Info ] SQLite storage backend is ready. Schema version: 31 
2019-01-29 20:04:43 [Info ] Database authenticator is ready. 
2019-01-29 20:04:43 [Info ] Listening for GUI clients on IPv6 :: port 4242 using protocol version 10 
2019-01-29 20:04:43 [Info ] Listening for GUI clients on IPv4 0.0.0.0 port 4242 using protocol version 10 
2019-01-29 20:04:43 [Info ] Restoring previous core state... 
2019-01-29 20:05:19 [Info ] Client connected from 127.0.0.1 
2019-01-29 20:05:19 [Debug] Enabling compression...
2019-01-29 20:05:19 [Debug] Using the DataStream protocol...
2019-01-29 20:05:19 [Debug] Starting encryption for Client: "127.0.0.1"
2019-01-29 20:05:30 [Info ] Client 127.0.0.1 initialized and authenticated successfully as "digitalcircuit" (UserId: 1). 
2019-01-29 20:05:49 [Info ] Client 127.0.0.1 disconnected (UserId: 1).

Track the last successful upgrade step (upgrade_###_XXX.sql) within each schema version, storing it within 'coreinfo' table as 'schemaupgradestep'. When a schema upgrade finishes, clear 'schemaupgradestep' and set 'schemaversion'. This allows for resuming multi-step schema upgrades that were interrupted in the middle. Whenever starting a schema upgrade, also check the value of 'schemaupgradestep'. One of two states exist: 1. Empty ('') or nonexistent No interrupted schema upgrade, start the next schema version upgrade from the first query. 2. Contains text, e.g. 'upgrade_010_alter_sender_64bit_ids' The schema upgrade was interrupted, skip schema upgrade steps including the specified successful step, and resume from the next step. For case 2, if the schema upgrade step cannot be found, warn and bail out. This should only happen if: 1. The storage of successful query glitched, or the database was manually changed 2. Quassel changed the filenames of upgrade queries, and the local Quassel core version was replaced during an interrupted schema upgrade Modify SqliteStorage and PostgreSqlStorage to fetch/save the 'schemaupgradestep' key. Clearing this key is done atomically within updateSchemaVersion(). (Ideally, the whole upgrade would be wrapped in a transaction, but that doesn't seem to be easily possible.) Modify upgradeQueries() to return a list of query strings and resource filenames, used for tracking the upgrade step and providing clearer feedback on what steps fail.

Print the problematic setup step (e.g. 'setup_070_coreinfo') if storage backend initialization fails. This may help with debugging. Modify setupQueries() to return a list of query strings and resource filenames, used for the above.

Remove the redundant updateSchemaVersion() call at the end of schema migrations. This is already called within the final iteration of the loop. Cleans up a leftover from f10304a

Remove the redundant updateSchemaVersion() call at the end of schema migrations. This is already called within the final iteration of the loop. Cleans up a leftover from f10304a Closes quasselGH-474

digitalcircuit added 3 commits January 29, 2019 00:06

core: Display problem step if storage setup fails

04be3c7

Print the problematic setup step (e.g. 'setup_070_coreinfo') if storage backend initialization fails. This may help with debugging. Modify setupQueries() to return a list of query strings and resource filenames, used for the above.

core: Remove redundant updateSchemaVersion() call

0b930dc

Remove the redundant updateSchemaVersion() call at the end of schema migrations. This is already called within the final iteration of the loop. Cleans up a leftover from f10304a

digitalcircuit changed the title ~~[WIP] [0.13, rollup tests] core: Track upgrade step within each schema version~~ [WIP] [0.13] core: Track upgrade step within each schema version Jan 29, 2019

digitalcircuit mentioned this pull request Jan 29, 2019

core: Track upgrade step within each schema version #473

Merged

digitalcircuit mentioned this pull request Jan 29, 2019

OUTDATED: [rollup] common: Master of backports (up to 2019-1-28) for 0.13.1 #475

Closed

digitalcircuit changed the title ~~[WIP] [0.13] core: Track upgrade step within each schema version~~ [0.13] core: Track upgrade step within each schema version Jan 30, 2019

Sput42 approved these changes Feb 13, 2019

View reviewed changes

Sput42 merged commit 63725f7 into quassel:0.13 Feb 13, 2019

digitalcircuit deleted the backport-0.13-ft-db-track-multistep-schema branch February 14, 2019 04:35

digitalcircuit mentioned this pull request Dec 17, 2020

Prepare for release of Quassel v0.14 #555

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[0.13] core: Track upgrade step within each schema version #474

[0.13] core: Track upgrade step within each schema version #474

digitalcircuit commented Jan 29, 2019 •

edited

[0.13] core: Track upgrade step within each schema version #474

[0.13] core: Track upgrade step within each schema version #474

Conversation

digitalcircuit commented Jan 29, 2019 • edited

In short

Rationale

Examples

Simulated interruptions using debugger

Non-deterministic interruption via kill

Interrupted upgrade log

digitalcircuit commented Jan 29, 2019 •

edited

Non-deterministic interruption via `kill`