Changes to the schema

Mismatch databases / GIT

The (postgre)SQL database schemas do not match the GIT development model. It's quite hard to get changes right. With GIT, changes are first implemented in branches. When the branch gets accepted, it gets merged into the master branch (for Taranis branch 'develop'). But during development in a branch, you may need to use database changes already... Besides, different branches may work on different database changes at the same time, but the database has a linear incremental schema version.

Basic database change strategy

This works when you make a change in 'develop', for instance add an index or constraint. Add an upgrade script in install/db-upgrade-scripts/

• start with a copy of an existing script
• the script name starts with four digits: the first to are the digits of the next major release. For instance, when the next release is 3.6.0, than start with 3600. The last two digits are simply sequential. The four digits are followed by a dot and a short indication about the purpose.
• your script may also implement content changes or transformations. The script runs in a single transaction: when it fails anywhere, data changes will get reverted.
• run the script via "taranis install --no-internet" until it works.
• "git add install/db-upgrade-scripts/3600.*"
• apply the same changes to install/db-load/taranis-schema.sql
• do not forget to change the schema version in that same file:

- INSERT INTO taranis VALUES ('schema_version', 3456);
+ INSERT INTO taranis VALUES ('schema_version', 3600);

• "git commit -a"

Complex changes

Things really start to become complex when changes are need to support new code which is developed in a branch. Especially when you have multiple branches under development. This will require steps which require careful management. Use a wiki page for each branch to maintain a TODO list!

1. Create a change script (as above) which extends the database schema in branch 'develop' This is required to guarantee orderly schema changes. This change hence may be applied in a release which does not contain the new branch's code yet.
2. Create a script in your new branch which converts existing data into the new structure. The 'devel/' directory would be a good location for that script. Make it work a a way that you can call it as often as you want to.
3. When alpha testing the branch, inform the tester to run the data conversion first.
4. When the branch gets accepted in 'develop', move the data transformation script into either a database upgrade script or install/8??.update-to-3.6.0
5. Add a database script (as above) to branch 'develop' to cleanup the expired database structures.

It is really needed to run an realistic upgrade path test, as Taranis maintainers will experience, before the major release is published.