Enable encryption for mobile targets #2070

winghouchan · 2025-05-18T13:23:04Z

Why?

#1423 introduced support for React Native however encryption was disabled due to the difficulty in getting successful builds with encryption enabled:

Encryption seems to be compiling sqlcipher (?) via cmake, which also includes OpenSSL flags, the problem the compilation step is hard to integrate with Android (and that's why I had to disable it). Someone else that knows the internals of libsql should give it a try.

– #1423 (comment)

With encryption disabled, libSQL on React Native does not have feature parity with other platforms.

What?

This PR enables encryption for builds against mobile targets. However, while the encryption feature is enabled, there is additional work to get the encryption feature fully complete:

P.S. if you do turn on encryption, it will require a modification to the bindings to send the encryption key from JS, so, open a PR when the time comes and I will take a look.

– #1423 (comment)

How?

The encryption feature of the libSQL dependency in the C bindings package has been enabled for all targets.
Build scripts and configurations in the libSQL FFI package have been updated so that libSQL products for each target and the OP SQLite Example App build successfully; and the tests against the OP SQLite Example App pass.

I'm not actually familiar with Android/iOS/C/Rust build systems, or libSQL / OP SQLite internals so configuration decisions were based on a feedback loop of running make <android|ios> in bindings/c and making updates until builds in libSQL and the OP SQLite Example App were successful. I'm unsure if there's anything else to check and will have to defer to Turso team members and @ospfranco for their knowledge.

winghouchan · 2025-05-18T14:03:32Z

bindings/c/Makefile

@@ -3,6 +3,7 @@ CFLAGS := -Iinclude
 LDFLAGS := -lm
 ARCHS_IOS = x86_64-apple-ios aarch64-apple-ios aarch64-apple-ios-sim
 ARCHS_ANDROID = aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
+ANDROID_PLATFORM := 31


Note

This value comes from the value already passed to the platform option when running cargo ndk. It has been pulled out and assigned here so that it can also be passed as an environment variable for libSQL FFI's build script which uses it to set the ANDROID_PLATFORM variable when running CMake.

winghouchan · 2025-05-18T14:15:35Z

libsql-ffi/build.rs

@@ -452,7 +452,7 @@ fn build_multiple_ciphers(target: &str, out_path: &Path) {
    let bundled_dir = format!("{out_dir}/sqlite3mc");
    let sqlite3mc_build_dir = env::current_dir().unwrap().join(out_dir).join("sqlite3mc");

-    let mut cmake_opts: Vec<&str> = vec![];
+    let mut cmake_opts: Vec<String> = vec![];


Note

cmake_opts changed from Vec<&str> to Vec<String> to allow for interpolating options values into a string with their option names.

winghouchan · 2025-05-18T14:41:12Z

libsql-ffi/build.rs

+    let ndk_cmake_toolchain_path = env::var("CARGO_NDK_CMAKE_TOOLCHAIN_PATH").ok();
+    let toolchain_path = ndk_cmake_toolchain_path
+        .clone()
+        .map(PathBuf::from)
+        .unwrap_or_else(|| sqlite3mc_build_dir.join("toolchain.cmake"));
+    let cmake_toolchain_opt = ndk_cmake_toolchain_path
+        .clone()
+        .map(|path| format!("-DCMAKE_TOOLCHAIN_FILE={}", path))
+        .unwrap_or_else(|| "-DCMAKE_TOOLCHAIN_FILE=toolchain.cmake".to_string());


Note

Use the Android NDK toolchain if available (determined by the value of the CARGO_NDK_CMAKE_TOOLCHAIN_PATH). This seems to be necessary for a successful build for Android.

winghouchan · 2025-05-18T14:41:46Z

libsql-ffi/build.rs

+    cmake_opts.push("-DCMAKE_BUILD_TYPE=Release".to_string());
+    cmake_opts.push("-DSQLITE3MC_STATIC=ON".to_string());
+    cmake_opts.push("-DCODEC_TYPE=AES256".to_string());
+    cmake_opts.push("-DSQLITE3MC_BUILD_SHELL=OFF".to_string());
+    cmake_opts.push("-DSQLITE_SHELL_IS_UTF8=OFF".to_string());
+    cmake_opts.push("-DSQLITE_USER_AUTHENTICATION=OFF".to_string());
+    cmake_opts.push("-DSQLITE_SECURE_DELETE=OFF".to_string());
+    cmake_opts.push("-DSQLITE_ENABLE_COLUMN_METADATA=ON".to_string());
+    cmake_opts.push("-DSQLITE_USE_URI=ON".to_string());
+    cmake_opts.push("-DCMAKE_POSITION_INDEPENDENT_CODE=ON".to_string());

    if target.contains("musl") {
-        cmake_opts.push("-DCMAKE_C_FLAGS=\"-U_FORTIFY_SOURCE\" -D_FILE_OFFSET_BITS=32");
-        cmake_opts.push("-DCMAKE_CXX_FLAGS=\"-U_FORTIFY_SOURCE\" -D_FILE_OFFSET_BITS=32");
+        cmake_opts.push("-DCMAKE_C_FLAGS=\"-U_FORTIFY_SOURCE\" -D_FILE_OFFSET_BITS=32".to_string());
+        cmake_opts
+            .push("-DCMAKE_CXX_FLAGS=\"-U_FORTIFY_SOURCE\" -D_FILE_OFFSET_BITS=32".to_string());


Note

Changing cmake_opts from Vec<&str> to Vec<String> necessitated calling to_string on these string literals.

This seems very verbose but I'm not familiar with Rust so please let me know if there is a better way.

winghouchan · 2025-05-18T14:56:10Z

libsql-ffi/build.rs

+    if target.contains("android") {
+        let android_abi = match target {
+            "aarch64-linux-android" => "arm64-v8a",
+            "armv7-linux-androideabi" => "armeabi-v7a",
+            "i686-linux-android" => "x86",
+            "x86_64-linux-android" => "x86_64",
+            _ => panic!("Unsupported Android target: {}", target),
+        };
+        let android_platform = std::env::var("ANDROID_PLATFORM")
+            .expect("ANDROID_PLATFORM environment variable must be set");
+
+        cmake_opts.push(cmake_toolchain_opt);
+        cmake_opts.push(format!("-DANDROID_ABI={}", android_abi));
+        cmake_opts.push(format!("-DANDROID_PLATFORM=android-{}", android_platform));
+    }


Important

This is probably the critical change to build for Android. It configures CMake in the following ways:

Sets CMAKE_TOOLCHAIN_FILE to the path to the Android NDK toolchain file (available via the CARGO_NDK_CMAKE_TOOLCHAIN_PATH environment variable).

Sets the ANDROID_ABI variable according to the defined target.

Sets the ANDROID_PLATFORM variable to the platform level specified in the C bindings package manifest, via an environment variable named ANDROID_PLATFORM.

winghouchan · 2025-05-18T15:01:33Z

libsql-ffi/build.rs

+    if target.contains("ios") {
+        cmake_opts.push("-DCMAKE_SYSTEM_NAME=iOS".to_string());
+
+        let (arch, processor, sysroot) = if target.contains("x86_64") {
+            ("x86_64", "x86_64", "iphonesimulator")
+        } else if target.contains("aarch64") {
+            let sysroot = if target.contains("sim") {
+                "iphonesimulator"
+            } else {
+                "iphoneos"
+            };
+            ("arm64", "arm64", sysroot)
+        } else {
+            panic!("Unsupported iOS target: {}", target);
+        };
+
+        cmake_opts.push(format!("-DCMAKE_OSX_ARCHITECTURES={}", arch));
+        cmake_opts.push(format!("-DCMAKE_SYSTEM_PROCESSOR={}", processor));
+        cmake_opts.push(format!("-DCMAKE_OSX_SYSROOT={}", sysroot));


Important

This is probably the critical change to build for iOS. It configures CMake in the following ways:

Sets the CMAKE_OSX_ARCHITECTURES variable to the architecture according to the target.

Sets the CMAKE_SYSTEM_PROCESSOR variable to the processor according to the target.

Sets the CMAKE_OSX_SYSROOT variable to iphonesimulator if the target is a simulator or the x86_64 architecture (there are no physical x86_64 iPhones) or iphoneos if the target is a physical device.

winghouchan · 2025-05-18T15:11:37Z

@ospfranco: given my unfamiliarity, would you mind expanding on this comment on #1423 please?

P.S. if you do turn on encryption, it will require a modification to the bindings to send the encryption key from JS, so, open a PR when the time comes and I will take a look.

winghouchan · 2025-05-18T15:34:52Z

Related: #1384

winghouchan · 2025-05-18T15:42:51Z

I see there's some logic for cross compilation:

libsql/libsql-ffi/build.rs

Lines 457 to 509 in c6ebbe1

    
           let target_postfix = target.to_string().replace("-", "_"); 
        
           let cross_cc_var_name = format!("CC_{}", target_postfix); 
        
           println!("cargo:warning=CC_var_name={}", cross_cc_var_name); 
        
           let cross_cc = env::var(&cross_cc_var_name).ok(); 
        
           let cross_cxx_var_name = format!("CXX_{}", target_postfix); 
        
           let cross_cxx = env::var(&cross_cxx_var_name).ok(); 
        
           let toolchain_path = sqlite3mc_build_dir.join("toolchain.cmake"); 
        
           let cmake_toolchain_opt = "-DCMAKE_TOOLCHAIN_FILE=toolchain.cmake".to_string(); 
        
           let mut toolchain_file = OpenOptions::new() 
        
               .create(true) 
        
               .write(true) 
        
               .append(true) 
        
               .open(toolchain_path.clone()) 
        
               .unwrap(); 
        
           if let Some(ref cc) = cross_cc { 
        
               let system_name = if cc.contains("linux") { 
        
                   "Linux" 
        
               } else if cc.contains("darwin") { 
        
                   "Darwin" 
        
               } else if cc.contains("w64") { 
        
                   "Windows" 
        
               } else { 
        
                   panic!("Unsupported cross target {}", cc) 
        
               }; 
        
               let system_processor = if cc.contains("x86_64") { 
        
                   "x86_64" 
        
               } else if cc.contains("aarch64") { 
        
                   "arm64" 
        
               } else if cc.contains("arm") { 
        
                   "arm" 
        
               } else { 
        
                   panic!("Unsupported cross target {}", cc) 
        
               }; 
        
               cmake_opts.push(&cmake_toolchain_opt); 
        
               writeln!(toolchain_file, "set(CMAKE_SYSTEM_NAME \"{}\")", system_name).unwrap(); 
        
               writeln!( 
        
                   toolchain_file, 
        
                   "set(CMAKE_SYSTEM_PROCESSOR \"{}\")", 
        
                   system_processor 
        
               ) 
        
               .unwrap(); 
        
               writeln!(toolchain_file, "set(CMAKE_C_COMPILER {})", cc).unwrap(); 
        
           } 
        
           if let Some(cxx) = cross_cxx { 
        
               writeln!(toolchain_file, "set(CMAKE_CXX_COMPILER {})", cxx).unwrap(); 
        
           }

Would it be better to somehow work within this, instead of the changes currently proposed (79ff682)?

penberg · 2025-05-20T07:33:45Z

@levydsa do you remember why we disabled encryption for mobile targets?

levydsa · 2025-05-20T15:06:26Z

@penberg, because it was hard to cross-compile sqlite3mc to mobile targets. But apparently, its possible with a few tweaks.

tyrauber · 2025-05-22T17:10:36Z

Thank you @winghouchan, for picking this up. libsql is fantastic on mobile devices (thanks to op-sqlite), but the lack of database encryption is definitely problematic. I would love to see this get merged in.

Enable encryption for mobile targets

79ff682

winghouchan force-pushed the enable-encryption-for-mobile-targets branch from 0d0250d to 79ff682 Compare May 18, 2025 13:26

winghouchan mentioned this pull request May 18, 2025

[Question]: libsql encryption support OP-Engineering/op-sqlite#224

Closed

winghouchan commented May 18, 2025

View reviewed changes

winghouchan mentioned this pull request Jun 11, 2025

Simplify build script with cmake crate #2099

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable encryption for mobile targets #2070

Enable encryption for mobile targets #2070

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

winghouchan May 18, 2025

Uh oh!

winghouchan May 18, 2025

Uh oh!

winghouchan May 18, 2025

Uh oh!

winghouchan May 18, 2025

Uh oh!

winghouchan May 18, 2025

Uh oh!

winghouchan May 18, 2025

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

penberg commented May 20, 2025

Uh oh!

levydsa commented May 20, 2025

Uh oh!

tyrauber commented May 22, 2025

Uh oh!

Uh oh!

Enable encryption for mobile targets #2070

Are you sure you want to change the base?

Enable encryption for mobile targets #2070

Uh oh!

Conversation

winghouchan commented May 18, 2025

Why?

What?

How?

Uh oh!

winghouchan May 18, 2025

Choose a reason for hiding this comment

Uh oh!

winghouchan May 18, 2025

Choose a reason for hiding this comment

Uh oh!

winghouchan May 18, 2025

Choose a reason for hiding this comment

Uh oh!

winghouchan May 18, 2025

Choose a reason for hiding this comment

Uh oh!

winghouchan May 18, 2025

Choose a reason for hiding this comment

Uh oh!

winghouchan May 18, 2025

Choose a reason for hiding this comment

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

winghouchan commented May 18, 2025

Uh oh!

penberg commented May 20, 2025

Uh oh!

levydsa commented May 20, 2025

Uh oh!

tyrauber commented May 22, 2025

Uh oh!

Uh oh!