Questions regarding the future of GRDB + SQLCipher

[groue]

Hello all,

GRDB 7, presently in the making, will drop CocoaPods support. 💔

Some folks will be able to migrate to SPM. But folks who use GRDB + SQLCipher will be stuck with GRDB 6.

This sad situation is a conjunction of several blockers:

• SPM can't build SQLCipher.
• CocoaPods won't publish a new GRDB version from Xcode 15.3 (GRDB 7 will require Swift 5.10+ and Xcode 15.3+).
• I'd rather not ship binary builds of SQLCipher, because there are always people who want a different build (different compiler version, different platform, whatever). Binary builds are a chore and a time sink, but even more so for a library that aims at running on whatever device can run Swift and SQLite. And, well, I have to admit I'm not a build expert.

I will not remove the SQLCipher and CocoaPods support from the repository, because the CocoaPods blocker might be resolved one day.

---

I have a question for the community: do you have any advice or experience that could help unblocking the situation and allow SQLCipher users to enjoy the upcoming GRDB 7?

Also, if you are using SQLCipher and are concerned with this announcement, did you consider using data protection instead?

Please share your experiences below 🙏

[groue]

Hello @ayoy. May I ask you, one of the authors of https://github.com/duckduckgo/GRDB.swift (the GRDB fork that ships a pre-built XCFramework of GRDB + SQLCipher), if you can imagine anything that could be done from here, the main repository?

[ayoy]

Hi @groue! Thanks for the ping and congratulations on dropping CocoaPods :)

At DuckDuckGo we've initially used GRDB with SQLCipher in SPM as source code, and have switched to binary framework only to speed up our compile times. But my point is that it's perfectly doable to have it distributed via SPM as source code.

That said, it will definitely be tricky to make SQLCipher-enabled GRDB available alongside vanilla GRDB in the same SPM package (code repository). And even if it's doable, it would still require you to include SQLCipher sources in your repository, and it will still force users to use the fixed set of compile flags for SQLCipher.

You'll find our source distribution in the SQLCipher branch of our fork (we keep it as a backup in case binary frameworks stop working). If you're not so keen on digging into the build system, I'd be happy to try it myself (i.e. to see if it's possible to have GRDB and GRDB+SQLCipher targets available in a single SPM package). If that works for you, just let me know when you have the GRDB 7 branch ready to fork and play with.

Thanks for your great work! 💪

[groue]

Right now I'm focusing on the features (with a lot of boxing with Swift Concurrency 🥊😜), and probably I'll release the lib without SQLCipher support initially. And we also need to give time to people who want to comment, contribute, or mourn here. But I'll let you know, yes, because you know a heck a lot more than I do!

[groue]

Hello @michaelkirk and @charlesmchen. We had conversation about GRDB in the Signal app in the past. Signal currently uses GRDB+SQLCipher through CocoaPods, so I thought you'd be happy to know about the situation. I'm curious about your opinions.

[groue]

cc @max-signal, @harry-signal, @sashaweiss-signal, @pete-signal, because you are the main contributors to Signal in the last year.

[bok-]

We use GRDB + SQLCipher with a small script that wraps CocoaPods to generate XCFrameworks, which I'd love to do away with. Unfortunately I don't have any good ideas off the top of my head as to how you could accomplish that with SQLCipher not supporting SPM themselves, and assuming you had that - without forcing it to be a dependency for everyone regardless of whether they want it or not, I guess for the latter you could create a GRDB-SQLCipher wrapper package that depends on both, but without a way to pass configuration , compiler directives or anything really between packages it feels pretty difficult. If you do include it in the main package you can probably usage the package access modifier to allow the main package to expose things to allow a GRDB+SQLCipher target to overwrite it, but that feels like it could be a large refactor.

We do also use data protection but its not something you can rely on in a number of situations — such as when the user disables their iPhone's passcode. Yes the user is compromising their own information security so its their own fault, but we would likely still suffer reputation damage if that happened (I work for a large bank), so we want to make sure we have full control over the encryption of the database. Hope that context helps some.

[groue]

We do also use data protection but its not something you can rely on in a number of situations — such as when the user disables their iPhone's passcode. Yes the user is compromising their own information security so its their own fault, but we would likely still suffer reputation damage if that happened (I work for a large bank), so we want to make sure we have full control over the encryption of the database. Hope that context helps some.

Thanks @bok- :-) That's the context I was missing.

Maybe you can answer two other questions (I don't want to infringe any industrial property):

• If you need to change the key, are you satisfied with the recommended technique, or did you have to do something differently?
• How do you build the key? (now I may cross some limits, so please don't answer if you have any reasonable doubt 😅)

[bok-]

Apologies for the late reply! I personally think it's not strictly industrial property when you take a standard approach, so I don't think it's infringing anything to describe what we do.

I decided early on that changing the paraphrase on the database was an expensive operation so we wanted to avoid that. As a result we always encrypt the database using a symmetric key generated by the best option CryptoKit gives you out of the box. We had some performance problems with this initially but found it was because we supplied the key to GRDB in the wrong format. Since we fixed that it has been perfect.

That key is stored in keychain with flags for no backup + no migration. So during normal operation that's accessible to the app for synchronisation or user access or whatever.

If we decide that a user's device is not safe because they don't have a passcode set, or it's jailbroken or other things OWASP says to check for (and some of our own custom stuff) then we take the symmetric key from the keychain, and re-encrypt it with a key derived from the PIN the user uses to access our app. That also means any situation where the PIN was previously optional it is now required. It also means we don't need to change the paraphrase protecting our GRDB database, just adding an extra step to decrypt the key we pass to GRDB.

[groue]

This is a great answer, thank you very much 🙏

[groue]

It should be possible for applications to keep on using GRDB+SQLCipher with CocoaPods even if it is impossible to publish a new version of the GRDB pod, by declaring a dependency on a branch:

# Same as pod 'GRDB.swift/SQLCipher', '~> 6.0'
# We depend on a branch due to https://github.com/CocoaPods/CocoaPods/issues/11839
# that prevents GRDB from publishing tagged releases on cocoapods.org.
pod 'GRDB.swift/SQLCipher', git: 'https://github.com/groue/GRDB.swift.git', branch: 'GRDB6'
pod 'SQLCipher', '~> 4.0'

The GRDB6 will only ever point to tagged releases. For example, on 2024/03/24, GRDB6 is identical to v6.26.0.

The same technique will be available for GRDB7 (on the GRDB7 branch - which is currently a work in progress and is NOT production-ready).

[rakuyoMo]

Hello! I have a question: When I rely on a sqlcipher fork that supports spm through spm (such as https://github.com/Tencent/sqlcipher/blob/v1.4.4/Package.swift), can GRDB's encryption function be used normally?

(Current version, and upcoming GRDB7)

[groue]

Who knows? Try and share your experience.

[rakuyoMo]

I want to say I failed

I use Tuist+SPM, which is equivalent to directly operating multiple xcprojects to some extent.

It may work. I modify the settings of GRDB to depend on a SQLCipher binary package, but it is just a component, so I need to modify every project that depends on this component.

Maybe using the above-mentioned https://github.com/duckduckgo/GRDB.swift or other forks containing SQLCipher binary packages is a better choice.

[rakuyoMo]

There are so many forks of GRDB (I guess because it's good enough), so many that I only found duckduckgo/GRDB.swift. If anyone knows of other good GRDB forks that include SQLCipher, please share them.

[groue]

The Network Graph of the repository can be useful when you are looking at specific forks. You can see individual commits, and their purpose (if they are well written). It's still the search of a needle in a haystack, but with a magnifying glass ;-)

[rakuyoMo]

Have you ever considered separating the SQLCipher-related code from GRDB's main repository (Package.swift) and putting it in an independent repository?

As mentioned above, I tried to rely on the SQLCipher binary library, but there were many changes to be made. And it would be very troublesome to synchronize GRDB upstream updates in the future.

If the SQLCipher-related functions are separated from GRDB, then users including me can rely on GRDB and the SQLCipher binary library to make their own component library, which is more flexible. And the amount of SQLCipher-related function code is not large, which is convenient to rely on in the form of source code (or other methods) in this secondary development component. Unless there are special circumstances, users do not need to care about GRDB updates, and can use the latest version of GRDB in the project without updating this secondary development component.

[rakuyoMo]

Let me be more specific. Suppose there are two spm libraries, GRDB and GRDB-SQLCipher. GRDB is all the code in GRDB.swift except for the code related to SQLCipher. GRDB-SQLCipher contains the code related to SQLCipher.

Users can create a new component, such as MyGRDB, which depends on GRDB and a SQLCipher binary package, and also adds the content in GRDB-SQLCipher in the form of source code.

If GRDB is updated in the future (for example, to version 7.1), users do not need to update MyGRDB (assuming that GRDB 7.1 will not affect GRDB-SQLCipher), and can directly use GRDB 7.1 in the main project.

[rakuyoMo]

Otherwise, as it is now, I need to fork GRDB.swift and add a dependency on SQLCipher in Package.swift (and I may need to modify some code, I'm not sure).

Then I need to add some version control, add a tag or create a branch. Finally, use the tag or branch to reference it in the project.

If GRDB is updated later, I need to pay some cost to synchronize the upstream update, then create a new tag or branch, and finally reference the new version in my project.

This process is not as simple and once and for all as the above method for me.

Of course, you or other users may not think so, which is normal. These words only represent my thoughts. We are just discussing, discussing a possibility, right?

[groue]

No, that's not a possibility.

An hypothetical GRDB-SQLCipher package can not instruct its dependency GRDB to stop linking against the system SQLite, and link against SQLCipher instead. That's just not an SPM feature.

[groue]

Thanks for searching, though :-) But, yeah, we have technical (and time) constraints to deal with.

[ThePragmaticArt]

Wait, couldn't the library be separated into 3 layers?

At a Foundation layer, you would have the DI interfaces to interface with SQLCipher/SQLite shared interfaces if necessary. At a Domain layer the core GRDB interfaces. Optionally, there can be a separate interface with default SQLite definitions for injection purposes which the consuming project would be responsible for importing -> Injecting into GRDB itself?

This would at least make GRDB agnostic of binary implementations and hopefully provide injection points to override some behaviors via SQLCipher if desired.

If someone would like to use vanilla SQLIte, there's just an initial definition that would be required to be DI'ed into the library. Having a non 1:1 coupling between the core GRDB interfaces and core SQLite interfaces should leave the responsibility on the integration libraries to translate things correctly (given an opportunity) where-as GRDB can focus on the vanilla SQLite translations within their own library exclusively.

Ideally the concept wouldn't be very far off from normal DI mechanisms or plugin approaches: https://github.com/apple/swift-log.

The biggest headache today is that Realm is essentially dead (in a sense) and all 3rd party encryption libraries for CoreData are dead (given how long that's been a thing it's unlikely to be revived). The only remaining option for enterprise is SQLite encryption or plain encrypted files on disk...

[morajabi]

@rakuyoMo We've updated DuckDuckGo's awesome fork to use latest GRDB 7.0.0-beta.6, feel free to use!

https://github.com/inline-chat/GRDB.swift

[aehlke]

@ChaosCoder @morajabi is it documented / specified somewhere in your repos which xcode version is used for the xcframework? just want to understand how to determine and how to update it myself, thanks

[ChaosCoder]

@morajabi Those points are what we fixed with our fork:

• Your fork was out of date and did not have any of the releases from the last months. Especially there was no release version of GRDB 7.
• The xcframework was compiled with a version of Xcode/Swift, that had the following bug: App crashes with EXC_BAD_ACCESS on first database read in Xcode 16 Beta 6 #1596 and could therefor not be used
• The script did not produce a xcframework out of the box for the new GRDB releases
• You don't have the issues-tab activated in GitHub, so I could not open an issue with what I found

If you want to maintain your fork, maybe you could try to automate the release process so it is tied to the release of new GRDB and/or SQLCipher versions?

@aehlke I used Xcode 16.2, but you are right, it was not specified. I just updated the readme so it includes this information. The script itself just uses the Xcode that was selected via xcode-select on your machine and now includes this information automatically. See TICESoftware@e374dc6

[aehlke]

Thanks so much. I'm not using SQLCipher but a lot of my build time is taken up by GRDB so I'd like to use xcframeworks. Appreciate it

[ayoy]

Just a note for anyone relying on the DuckDuckGo fork – I’ve finally updated it to GRDB 7.4.1. Sorry for the delay here, but we just didn’t need to upgrade earlier.

[morajabi]

@ChaosCoder Thanks for the detailed comment. I fixed all of the issues you mentioned and released a new version.

If anyone uses GRDBQuery, I think they'd have to also use the patched fork that points to the forked GRDB.

Also, as a side note, It's amazing to see the active community around GRDB. Special thanks to @ayoy for creating the initial fork.

[DrewKiino]

would love to see a native SPM approach to this as we will be integrating GRDB in an enterprise app and we will be relying on encryption later down the line - fingers crossed

[aehlke]

use a fork #1495 (reply in thread)