Migration Open-Pryv.io from 1.7.1 to 1.9.0 - Issue resolution

[perki]

This discussion to keep the resolution of issues found in migration open-pryv.io 1.7.1 to 1.9.0

Using a sample dataset produced with open-pryv.io 1.7.1 which can be found here we actually found 2 issues.

1. the migration was not triggered as the version tag was not present.
2. the migration of attachments was failing as the folder name changed from attachment-files to attachments

If you're running a dockerized version, make a backup with docker exec -t open-pryv-mongo mongodump -d pryv-node -o /data/backup/. The migration has to be done on a native set-up, but the migrated data will be usable by a docker installation.

1. Make a dump a of v1.7.1 the database with ./var-pryv/mongodb-bin/bin/mongorestore <db_backup_destination>
2. Prepare a fresh set-up of v1.9.0 and copy the following directories into v1.9.0 var-pryv dir
   • {v1.7.1}/var-pryv/attachement-files => {v1.9.0}/var-pryv/attachements
   • {v1.7.1}/var-pryv/users => {v1.9.0}/var-pryv/users
3. Start mongodb for v1.9.0
   • Restore data from the dump folder ./var-pryv/mongodb-bin/bin/mongorestore <db_backup_destination>
   • Connect to mongodb with {v1.9.0}/var-pryv/mongodb-bin/bin/mongosh
   • From mongoshell type use pryv-node to connect to pryv.io db.
   • Then check the current version with: db.versions.find({}) if empty add one with: db.versions.insertOne({ _id: '1.7.1', initialInstall: 0 })
4. Run the migration script: LOGS=info NODE_ENV=production ./components/api-server/bin/migrate --config {your configuration file}
   At this stage you see the migration process and user correctly migrated should be listed at step 1.8.0 and attachments (if any) at step 1.9.0

Here is a sample output

... some warnings
2024-02-23 15:51:48 [info]: [api-server:migration-process] Connecting to storage... 
2024-02-23 15:51:48 [info]: [api-server:migration-process] Storage connected. 
2024-02-23 15:51:48 [info]: [api-server:migration-process] Starting migration... 
2024-02-23 15:51:48 [info]: [api-server:migration-1.7.5] V1.7.1 => v1.7.5 Migration started 
2024-02-23 15:51:48 [info]: [api-server:migration-1.7.5] V1.7.1 => v1.7.5 Migration finished 
2024-02-23 15:51:48 [info]: [api-server:migration-1.8.0] V1.7.5 => v1.8.0 Migration started 
2024-02-23 15:51:48 [info]: [api-server:migration-1.8.0] Migrating userId: clsyq5bum00001q9ylf5d2exx username: testuser 
2024-02-23 15:51:48 [info]: [api-server:migration-1.8.0] Migrating password for userId: clsyq5bum00001q9ylf5d2exx 
2024-02-23 15:51:48 [info]: [api-server:migration-1.8.0]  migrating 5 fields for userId: clsyq5bum00001q9ylf5d2exx 
2024-02-23 15:51:48 [info]: [api-server:migration-1.8.0] V1.7.5 => v1.8.0 Migration finished 
2024-02-23 15:51:48 [info]: [api-server:migration-1.9.0] V1.9.0 => v1.9.0 Migration started 
2024-02-23 15:51:48 [info]: [api-server:migration-1.9.0:attachments] Attachmend moved for userId: clsyq5bum00001q9ylf5d2exx to: /Users/perki/code/open-pryv.io/var-pryv/users/x/x/e/clsyq5bum00001q9ylf5d2exx/attachments 
2024-02-23 15:51:48 [info]: [api-server:migration-1.9.0] V1.8.0 => v1.9.0 Migration finished 
2024-02-23 15:51:48 [info]: [api-server:migration-process] Storage migration complete. 
2024-02-23 15:51:48 [info]: [api-server:migration-process] Process exiting.