Need a way to opt back in to seeing VersionError

[ethan-bernstein]

The new ability in v4.0.1 to open avoid VersionError is really interesting. However, it seems to break the usage of version upgrade logic in certain scenarios that are key for our application.

A common use case for our app is to add a new calculated property in one of our tables along with an index on that property. As part of the version definition, we add upgrade logic to calculate the new property for existing objects in the table.

When we roll out a new version of our app, we roll back the deployment if we see too many errors. When that happens, we fix the bugs leading to those errors and then roll out the new version of the app again.

Consider what happens during temporary rollback if the new version of the app contains one of these database version updates.

When a client first runs the new version of the app, the database version is updated, and the upgrade logic populates the new property for existing objects in the table. With v4.0.1, when the client goes back to running the older version of the app, the app still opens the database and may start writing objects to the table without the new property. When the client starts running the newer version of the app, the version upgrade logic does not run a second time and so the table is left with some objects that are missing the calculated property.

Are we missing some way in which we can get the version upgrade logic to run again when we roll forward to the newer version of the app a second time?

Or is there some way we can opt back in to seeing VersionError when trying to open a database version newer than what the current application understands?

[dfahlander]

Thank you for an interesting use case. The whole point with the less stricter version handling in dexie 4 is to be able to roll back to previous version.

Currently, upgraders run and behave exactly like in dexie 3, but in the case where a previous version is loaded, we just "allow it" without running any downgraders (which would be impossible as the old version could not know how to downgrade).

This also means that the new version would not run upgraders again, like you explain thoroughly.

You could workaround this issue using the ready event. In the callback, you will get an instance of Dexie with exclusive access to read and write to the database before the rest of the application's queries are resumed. You can then check if upgrade would be needed using custom logic and perform that logic.

I will consider whether this would be solvable in dexie to actually store when a database had been "downgraded" and then detect this when upgraded again and re-run the upgraders. Thoughts are welcome.

[dfahlander]

Just mentioning also the plans for dexie@5 to specify downgrade behavior, for example:

db.version(3)
  .upgrade(async (tx) => {
    await tx.table('contacts')?.toCollection().modify((contact) => {
      const [firstName, lastName] = contact.displayName?.split(' ') ?? [];
      contact.firstName = firstName;
      contact.lastName = lastName;
      delete contact.displayName;
    });
  })
  .downgrade('recreate'); // In case of downgrade - recreate the database (delete and reopen on prev version)

Another option is "throw" which basically would throw a VersionError when downgrading. Unfortunately it would not be possible to store code for downgrading though as it would be unsafe and unreliable to execute code from a stored string.

https://dexie.org/roadmap/dexie5.0#example-youve-refactored-your-model-and-need-to-move-data-around-to-comply-with-the-new-model

[lvivski]

we just "allow it" without running any downgraders (which would be impossible as the old version could not know how to downgrade).

I think there is a potential solution for this. It won't cover all scenarios as some upgrades are subtractive in nature and cannot be reverted without data loss but can still allow more control for more advanced usage patterns.

We could have a JSON-based upgrade/downgrade DSL and store the downgrade configuration in DB itself. That way you'd be able to generate the downgrade code from this JSON change definition and even write unit tests to validate data integrity.

This doesn't even have to be the default approach that Dexie uses and can be written as an addon. Dexie would just need to provide hooks for intercepting upgrades/downgrades and a way to rerun upgrades after the previous downgrade.

[dfahlander]

That are wise ideas.