Evaluate performance of PouchDB indexeddb adapter

[garethbowen]

Describe the performance issue

PouchDB 9.0 has just been released with significant improvements to the indexeddb adapter which should make it perform better than the idb adapter we currently use.

Describe the improvement you'd like

Upgrade to Pouch 9.0 and run a suite of client side performance tests with representative data on the old adapter and the new adapter and publish the results. There is currently no migration from one to the other so you will have to completely reset the browser data when you switch adapter.

Measurements

• Most important will be client side apdex or similar to measure user satisfaction with the new adapter
• But we should also measure cpu, memory, and disk usage to make sure we don't incur unintended costs

Additional context

Once we've determined if and how much this improves performance we can decide which of several options to take next (in a separate issue):

1. Do nothing
2. Use the indexeddb adapter for new clients, but keep the idb adapter around for existing clients. I like this option because it's not a breaking change, and allows for progressive rollout. This could also be used as a workaround for heavy users to do a sync, wipe, initial replication to get the improved performance.
3. Write code to migrate between adapters, and drop support for the idb adapter. This is difficult because we have to be careful about not exhausting disk space during the migration, and may take a while to execute which would block CHWs from doing work until it's finished. For this reason it's probably a breaking change and would be delayed until the next major release.

[latin-panda]

@ralfudx, we will likely run the automation Apdex tests to compare it with our previous numbers. cc: @michaelkohn

[ralfudx]

Yeah this is noted

[latin-panda]

I am assigning this to me as I want to run the Apdex Automation now that we get consistent results, and compare before and after. First trying with ugly code

[latin-panda]

Apdex score

This test suite focuses on queries during navigation and searching by free text.

The biggest performance improvements are:

• tasks:refresh with 70% improvement
• report_list:query with 11.6%
• report_list:load with 25.5%

No slowdowns were detected.

Telemetry files

baseline-default-config-jon-july-11.json

pouch9-default-config-kimberly-july-12.json

Telemetry before vs after

PouchDB 9 integration

The pouch-adapter-indexeddb was integrated into the pouchdb-9 branch. However, this is not a final implementation; we still need to resolve the migrations before sync. and determining how to transition existing users from the old adapter to the new one. The focus of this work was to make it work for performance evaluation.

During the integration, I encounter these errors:

• onblocked, this should never happen
  • That's telemetry recording too much too fast and also closing the DB when the pouch-adapter-indexeddb already has closed it.
• Database has a global failure DOMException: Unable to add key to index 'seq': at least one key does not satisfy the uniqueness requirements.
  • Maybe related bug reported in PouchDB: indexeddb adapter: Unable to add key to index 'seq'
• invalid ref format
  • It happens when running migrations before sync.

After I fixed the telemetry, the 2 first stopped appearing.

Other helpful Apdex suites

These are other test suites that can be done for this work, but we need to configure the suite for the default config:

• Run initial replication multiple times
• Form submits for contacts, reports, and tasks.
• Perform action by reopening the app in every iteration

[garethbowen]

70% is almost too good to believe. Are you confident in this number? Is it possible some of the other changes you made to this branch are causing it? Is there any other explanation you can think of that might explain this?

determining how to transition existing users from the old adapter to the new one

The easiest way I can think of is to have new users use the new adapter and existing users continue to use the old adapter. I like this because...

1. It means if there are any undetected issues we can pick them up as it gradually rolls out, rather than disrupting thousands of users in one go. This was useful in transitioning from crosswalk to webview in android.
2. It isn't disruptive to existing users at all. The migration for a user with loads of docs may take some time when they are just trying to do their job. Furthermore the naive implementation would involve duplicating the data, which risks running of out disk space.
3. Users are already being told to sync and wipe and reload their app if they find it slow, so migration of existing users will slowly happen.
4. We can provide either manual migration via a button, or automatic migration at a later date.

There is a complexity cost to this but the way PouchDB is architected it's not a big cost.

[latin-panda]

70% is almost too good to believe. Are you confident in this number? Is it possible some of the other changes you made to this branch are causing it? Is there any other explanation you can think of that might explain this?

I've added this to this week's tasks; I'll dig more into tasks and rules engine