Possible database corruption on save in v1.18

[tansly]

Describe the bug
When I change the database on my iPhone (add some entry etc.), I can no longer access it when I reopen the database (says the password or keyfile is invalid). I transfer the file to my computer (using iCloud drive, Google drive etc.) and I can't open there as well (HMAC mismatch).

To Reproduce
Steps to reproduce the behavior:

1. Open a database (local, Google Drive or iCloud drive, don't matter)
2. Add new entry and save
3. Go back to databases list
4. Reopen the database
5. See the error (Invalid password or key file)
6. Retype the password, still error
7. Transfer the file to a computer
8. Try to open with KeepassXC, get HMAC mismatch

Expected behavior
I should be able to reopen the database with my password.

Screenshots
I can add if necessary, but I don't think it'll help.

User Information (please complete the following information):

• Device: iPhone 11
• OS: iOS 14.0.1
• App Version: 1.18

Additional context
I can add if necessary but I can't think of anything else right now.

[keepassium]

• Do you use a key file or YubiKey, or only password?
• What are the database encryption parameters?
• KeePassium makes automatic backup copies as a precaution for this nightmare scenario. Can you open an earlier backup copy with the same password?

[tansly]

Thanks for the quick response.

• I use only password.
• I'm not sure about which parameters are these. In KeepassXC, database format is seen as KDBX 4.0. And decryption time is 1.0s. Is there anywhere else I should look into?
• I can open the latest backup file.

[keepassium]

Yes, unfortunately the nightmare scenario is on... The new express-unlock feature messes up the saving.

So far, up to 1% of users could have installed 1.18 as part of the phased rollout. I have stopped the rollout and posted a warning on twitter and reddit.

The issue is caused by using old encryption keys with new randomized seeds. Restoring corrupted database is possible by planting old (last working) random seeds to the new database, with a hex editor. A less involved solution is to restore a backup file, but with loss of some latest changes.

Sorry about this. The fix in underway.

[keepassium]

Better news: it will be possible to restore corrupted databases directly in the app. This will involve the last good in-app backup file, so make sure your backup files are stored long enough (KeePassium settings → Database Backup → Keep Backup Files for → at least 7 days).

The fix has been implemented, but I'd like to test it really thoroughly this time. (The problem affects all DB formats and all 1.18 users, but it does not show up in premium/beta/test versions. That's why I did not notice it initially.)

[keepassium]

Uploading the update right now. Hopefully, the expedite AppStore review is fast.

[PanzerSajt]

I believe I have this exact issue. The sad part is that I exported the last DB and Key combination and reinstalled the 1.18 App version because I had issues with password autofill. So now I have no previous KeePassium based DB backup. I only have a one month old offline backup DB.

Is there a manual way to restore the current DB using the one month old DB? I really don't want to loose one month's data.

[keepassium]

@PanzerSajt, ouch. Sorry about that. I'm afraid, restoring the database requires the header of the most recent good database (the one after which things went wrong). Or at least the saved encryption key in the keychain. Reinstalling KeePassium erased both of these.

If you happen to store your database in a cloud, maybe it has a history of recent file versions. Or if you have the iCloud backup enabled, you can try to extract KeePassium's backups from there.

[PanzerSajt]

Unfortunately there was no automatic cloud backup or iCloud backup.

So it means I am stuck with the old DB.
Please let me know if there might be a new method which only requires the one month old DB backup with the corrupted DB. (manual hex editing is not a problem)

[keepassium]

Please let me know if there might be a new method which only requires the one month old DB backup with the corrupted DB. (manual hex editing is not a problem)

I will, but it does not look too promising so far. Here's some technical detail.

The encryption key is a function of the composite master key(database password mixed with the key file, if any) and some random sequences stored in the DB header. The function is called the key derivation function (KDF).

encryption_key = KDF (composite_key, random_sequences_in_header)

When you open a database, KeePassium uses the composite master key and the data from the DB header to calculate the encryption key. To speed up future decryption, the app stores both the composite master key and the derived encryption key. (This can be turned off by the "Remember master keys" option.)

Before saving a modified database, the random sequences are re-randomized, the app derives the new encryption key, and uses it to encrypt the database. The new random sequences are also saved in the DB header.

Due to my error, v1.18 does randomize and write the random sequences, but still uses the old (saved) encryption key to encrypt the database. As a result, the next time any app tries to decrypt the database, the derived key is based on the new random sequences and thus produces a different encryption key.

At the moment, I see only two possibilities to access a damaged database:

• Using the encryption key which is still stored in the system keychain. This skips the key derivation step, so the new random sequences are not taken into account. Basically, the app directly uses the same encryption key that was used to encrypt the database.
• Alternatively, we can re-calculate this encryption key. The key used for encrypting the broken database is based on the random sequences stored in the last good database. So if we unlock the last good database, the app will save the derived encryption key — the same which was used for encrypting the damaged database. After this, we can switch to the broken database, and it will open up.

I don't see how to make this work with an older database, but maybe I am missing something...

[keepassium]

1.5 hours ago (18:30 CET) Apple confirmed they will proceed with an expedited review. No further updates so far.

Status as of 4 October:
9:00 CET: No news from Apple, review still pending.
9:54 CET: KeePassium Pro 1.19 approved, released.
10:45 CET: KeePassium 1.19 approved, released.

Steps to restore damaged databases are coming soon.

[keepassium]

Version 1.19 with the fix has been released. Auto-updates are scheduled to start slowly, so please update manually.

Steps to restore a damaged database:

1. Update to KeePassium v1.19
2. Make a backup copy of the damaged database
3. Restore the last good backup file
   1. In KeePassium, open the list of databases
   2. Open list settings (bottom-left button)
      1. Turn on the "Show Backup Files" switch
      2. Set sorting to "Modification Date (Recent First)"
      3. Tap "Done" to close the settings
   3. In the list of databases, you will see backup files with timestamps
   4. Try to unlock backup files starting from the first one (skip the ".latest.kdbx", it is broken)
   5. Once you find the most recent good backup file, save it
      1. Long-press the file → Export → Save to Files
      2. Rename the file exactly as your damaged database (basically, remove the timestamp). It is important that both the damaged and the last-good database have the same file name.
      3. Save the file to "On My iPhone" (in different folder than the damaged database)
4. Add the last-good backup file to KeePassium
   1. Open the Files app and navigate to the last-good database (restored at the previous step)
   2. Long-press the file → Share → scroll the icons left → More → Open in KeePassium
   3. You should see KeePassium with the added last-good database. (You might need to pull down the list to refresh it.)
5. Open KeePassium settings → Data Protection, and make sure that "Remember Master Keys" switch is enabled.
6. Unlock the last-good database with your password/key file. It should open up normally, and KeePassium will remember the encryption key of this file.
7. Tap "< Back" to return to the list of databases
8. Select the damaged database. KeePassium will use the remembered encryption key of the last-good database, and the broken database should open up successfully.
9. Save the restored database (for example, long-press any group → Edit → Done)

After this, the restored database should work normally both in KeePassium and other apps.

Historically, all KeePassium updates used to pass through a week of beta testing by volunteers, as a precaution. Time time I became too self-confident, decided all the changes were minor and risk-free, and released directly to the App Store (thankfully, as a gradual release). Lesson learned: never skip pre-release beta testing.

Again, sorry for the mess.

[keepassium]

Can anybody confirm if the issue still occurs in 1.19, please?

[tansly]

Hey, I’ve updated the app and tested it. I could successfully edit and reopen the DB. Everything seems fine. Thanks for the quick fix!

[keepassium]

Thank you for checking, @tansly!

@PanzerSajt , if you run a macOS, check if it has a Time Machine backup of the previous DB version.