Add sqlcipher-loader module and migrate PIR/autofill

[aitorvs]

Task/Issue URL: https://app.asana.com/1/137249556945/project/488551667048375/task/1213721049904377?focus=true

Description

Introduces a dedicated sqlcipher-loader module that centralises SQLCipher native library loading and fixes a JNI deadlock causing a 7x spike in LIBRARY_LOAD_TIMEOUT_SQLCIPHER (350 → 2,500/day).

Root cause: removing PIR's eager System.loadLibrary("sqlcipher") call (f6c879e) inadvertently removed the pre-warm that autofill depended on. Autofill's SqlCipherLibraryLoader then performed the first full load on a background thread, and when PIR's lazy load raced concurrently, a JNI loading deadlock could occur — triggering the 10s timeout.

Fix:

• New sqlcipher-loader-api module exposes a SqlCipherLoader interface with a single waitForLibraryLoad(): Result<Unit> API.
• New sqlcipher-loader-impl provides RealSqlCipherLoader, which:
  • Implements MainProcessLifecycleObserver to eagerly pre-warm SQLCipher on the IO dispatcher at app startup, before autofill or PIR need it.
  • Uses a CompletableDeferred<Unit> (initialised at construction) so all callers share the same load — concurrent complete() calls are no-ops, making the race structurally impossible.
  • Fires LIBRARY_LOAD_FAILURE_SQLCIPHER (daily pixel) if the load throws.
• PIR and autofill both now inject SqlCipherLoader and call waitForLibraryLoad() instead of loading independently.
• SqlCipherLibraryLoader (autofill-local) and its test deleted.
• sqlCipherAsyncLoading feature flag and LIBRARY_LOAD_TIMEOUT_SQLCIPHER pixel removed — no timeout needed when the load is predictable and early.

Steps to test this PR

SQLCipher loads correctly

• Install the app and open a page with a password field — autofill suggestion should appear normally
• Open the PIR screen — it should load without errors
• Check logcat for SqlCipher: native library loaded successfully appearing once at startup (not twice, not on demand)

No regression on autofill

• Save a login and verify it autofills on the target site
• Confirm no LIBRARY_LOAD_TIMEOUT_SQLCIPHER or LIBRARY_LOAD_FAILURE_SQLCIPHER pixels fire under normal conditions

Verify build

• ./gradlew :sqlcipher-loader-impl:testDebugUnitTest
• ./gradlew :autofill-impl:testDebugUnitTest
• ./gradlew :pir-impl:testDebugUnitTest

SQLCipher loads correctly on PIR process

• Obtain subscription
• Start a PIR scan via the PIR dashboard
• Logcat should show "SqlCipher: Attempting to load native library loaded on the PIR process”
• Logcat should show " SqlCipher-Init: Library load wait completed successfully”
• Logcat should show "PIR-DB: sqlcipher native library loaded ok"

---

Note

Medium Risk
Moderate risk because it changes when/how the SQLCipher native library is loaded and gates creation of encrypted databases for both Autofill and PIR, which can impact data access and startup behavior.

Overview
Centralizes SQLCipher native library loading into a new sqlcipher-loader module (SqlCipherLoader API + RealSqlCipherLoader impl) that eagerly starts async loading via process lifecycle observers and provides a shared waitForLibraryLoad for all callers (with timeout/failure pixels).

Migrates Autofill and PIR secure DB factories to inject SqlCipherLoader instead of doing their own loads, deleting Autofill’s local SqlCipherLibraryLoader and its feature flag (sqlCipherAsyncLoading) and moving the SQLCipher load pixels out of autofill.json5 into sqlcipher_loader.json5. The app wiring is updated to depend on the new modules and to strip ATB params for the new SQLCipher pixels.

^{Written by Cursor Bugbot for commit 33c8a5f. This will update automatically on new commits. Configure here.}

[karlenDimla]

• Add sqlcipher-loader module and migrate PIR/autofill #8009 👈 (View in Graphite)
• develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: runCatching swallows CancellationException breaking structured concurrency
  • waitForLibraryLoad() now uses explicit try/catch that rethrows CancellationException and only wraps non-cancellation failures in Result.failure.
• ✅ Fixed: Missing ATB parameter removal for migrated pixel
  • Added a PixelParamRemovalPlugin contribution in sqlcipher-loader-impl to remove ATB from library_load_failure_sqlcipher before sending.

Or push these changes by commenting:

@cursor push 6a071da47a

Preview (6a071da47a)

diff --git a/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt b/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
--- a/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
+++ b/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
@@ -24,6 +24,8 @@
 import com.duckduckgo.app.statistics.pixels.Pixel
 import com.duckduckgo.app.statistics.pixels.Pixel.PixelType.Daily
 import com.duckduckgo.common.utils.DispatcherProvider
+import com.duckduckgo.common.utils.plugins.pixel.PixelParamRemovalPlugin
+import com.duckduckgo.common.utils.plugins.pixel.PixelParamRemovalPlugin.PixelParameter
 import com.duckduckgo.di.scopes.AppScope
 import com.duckduckgo.library.loader.LibraryLoader
 import com.duckduckgo.sqlcipher.loader.api.SqlCipherLoader
@@ -31,6 +33,7 @@
 import com.squareup.anvil.annotations.ContributesBinding
 import com.squareup.anvil.annotations.ContributesMultibinding
 import dagger.SingleInstanceIn
+import kotlinx.coroutines.CancellationException
 import kotlinx.coroutines.CompletableDeferred
 import kotlinx.coroutines.CoroutineScope
 import kotlinx.coroutines.launch
@@ -75,7 +78,14 @@
         if (!libraryLoaded.isCompleted) {
             appCoroutineScope.launch(dispatchers.io()) { doLoad() }
         }
-        return runCatching { libraryLoaded.await() }
+        return try {
+            libraryLoaded.await()
+            Result.success(Unit)
+        } catch (cancellationException: CancellationException) {
+            throw cancellationException
+        } catch (throwable: Throwable) {
+            Result.failure(throwable)
+        }
     }
 
     private fun doLoad() {
@@ -100,3 +110,15 @@
 enum class SqlCipherPixelName(override val pixelName: String) : Pixel.PixelName {
     LIBRARY_LOAD_FAILURE_SQLCIPHER("library_load_failure_sqlcipher"),
 }
+
+@ContributesMultibinding(
+    scope = AppScope::class,
+    boundType = PixelParamRemovalPlugin::class,
+)
+object SqlCipherPixelsRequiringDataCleaning : PixelParamRemovalPlugin {
+    override fun names(): List<Pair<String, Set<PixelParameter>>> {
+        return listOf(
+            LIBRARY_LOAD_FAILURE_SQLCIPHER.pixelName to PixelParameter.removeAtb(),
+        )
+    }
+}

[karlenDimla]

Tested and everything looks good for me:

• Basic Autofill flows
• PIR flows (PIR disabled, PIR enabled and initial scan + background scan)
• Commented out doLoad to hit timeout and emit pixel

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: No guard prevents concurrent doLoad() native library loading
  • Added an AtomicBoolean compare-and-set guard in doLoad() so only one native sqlcipher load can be started across all call sites.

Or push these changes by commenting:

@cursor push 5d5d97d9d9

Preview (5d5d97d9d9)

diff --git a/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt b/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
--- a/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
+++ b/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
@@ -38,6 +38,7 @@
 import kotlinx.coroutines.TimeoutCancellationException
 import kotlinx.coroutines.launch
 import kotlinx.coroutines.withTimeout
+import java.util.concurrent.atomic.AtomicBoolean
 import logcat.LogPriority.ERROR
 import logcat.logcat
 import javax.inject.Inject
@@ -54,6 +55,7 @@
 ) : SqlCipherLoader, MainProcessLifecycleObserver, PirProcessLifecycleObserver {
 
     private val libraryLoaded = CompletableDeferred<Unit>()
+    private val loadStarted = AtomicBoolean(false)
 
     // Eagerly kick off the load at app startup, well before autofill or PIR need it.
     override fun onCreate(owner: LifecycleOwner) {
@@ -98,6 +100,9 @@
     }
 
     private fun doLoad() {
+        if (!loadStarted.compareAndSet(false, true)) {
+            return
+        }
         logcat { "SqlCipher: starting async library load" }
         LibraryLoader.loadLibrary(
             context,
@@ -110,7 +115,6 @@
 
                 override fun failure(throwable: Throwable) {
                     logcat(ERROR) { "SqlCipher: native library load failed: ${throwable.javaClass.simpleName} - ${throwable.message}" }
-                    // Guard ensures the pixel fires exactly once even if doLoad() races.
                     if (libraryLoaded.completeExceptionally(throwable)) {
                         pixel.fire(LIBRARY_LOAD_FAILURE_SQLCIPHER, type = Daily())
                     }

[landomen]

Checked autofill and PIR flows, including background workers, works as expected.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Missing try-catch in doLoad leaves deferred permanently pending
  • Wrapped the synchronous LibraryLoader.loadLibrary invocation in a try/catch that completes the deferred exceptionally and fires the failure pixel so waiters fail immediately instead of timing out.

Or push these changes by commenting:

@cursor push 8a17146051

Preview (8a17146051)

diff --git a/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt b/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
--- a/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
+++ b/sqlcipher-loader/sqlcipher-loader-impl/src/main/java/com/duckduckgo/sqlcipher/loader/impl/RealSqlCipherLoader.kt
@@ -103,23 +103,30 @@
 
     private fun doLoad() {
         logcat { "SqlCipher: starting async library load" }
-        LibraryLoader.loadLibrary(
-            context,
-            SQLCIPHER_LIB_NAME,
-            object : LibraryLoader.LibraryLoaderListener {
-                override fun success() {
-                    logcat { "SqlCipher: native library loaded successfully" }
-                    libraryLoaded.complete(Unit)
-                }
+        try {
+            LibraryLoader.loadLibrary(
+                context,
+                SQLCIPHER_LIB_NAME,
+                object : LibraryLoader.LibraryLoaderListener {
+                    override fun success() {
+                        logcat { "SqlCipher: native library loaded successfully" }
+                        libraryLoaded.complete(Unit)
+                    }
 
-                override fun failure(throwable: Throwable) {
-                    logcat(ERROR) { "SqlCipher: native library load failed: ${throwable.javaClass.simpleName} - ${throwable.message}" }
-                    if (libraryLoaded.completeExceptionally(throwable)) {
-                        pixel.fire(LIBRARY_LOAD_FAILURE_SQLCIPHER, type = Daily())
+                    override fun failure(throwable: Throwable) {
+                        logcat(ERROR) { "SqlCipher: native library load failed: ${throwable.javaClass.simpleName} - ${throwable.message}" }
+                        if (libraryLoaded.completeExceptionally(throwable)) {
+                            pixel.fire(LIBRARY_LOAD_FAILURE_SQLCIPHER, type = Daily())
+                        }
                     }
-                }
-            },
-        )
+                },
+            )
+        } catch (throwable: Throwable) {
+            logcat(ERROR) { "SqlCipher: native library load setup failed: ${throwable.javaClass.simpleName} - ${throwable.message}" }
+            if (libraryLoaded.completeExceptionally(throwable)) {
+                pixel.fire(LIBRARY_LOAD_FAILURE_SQLCIPHER, type = Daily())
+            }
+        }
     }
 
     private companion object {

[CDRussell]

LGTM.

Tested scenarios where autofill wouldn't be available due to both:

• sqlcipher library failure
• sqlcipher library timeout

Nit: this PR has thrown out tests that would still be useful to have that were captured in SqlCipherLibraryLoaderTest. I think Claude would easily port them over for you.