Trouble building GRDB with Carthage

[blue-toe]

Hi,

I'm having trouble compiling GRDB.swift.

I'm using carthage and never had trouble until recent releases when I've been getting this:

The following build commands failed:
CompileC /Users/raymond/Library/Developer/Xcode/DerivedData/GRDB-gdcllwrgfoeepnekbbfhunfrgxgp/Build/Intermediates/sqlcipher.build/Release-iphoneos/sqlcipher.build/Objects-normal/arm64/sqlite3.o sqlite3.c normal arm64 c com.apple.compilers.llvm.clang.1_0.compiler
CompileC /Users/raymond/Library/Developer/Xcode/DerivedData/GRDB-gdcllwrgfoeepnekbbfhunfrgxgp/Build/Intermediates/sqlcipher.build/Release-iphoneos/sqlcipher.build/Objects-normal/armv7/sqlite3.o sqlite3.c normal armv7 c com.apple.compilers.llvm.clang.1_0.compiler
CompileC /Users/raymond/Library/Developer/Xcode/DerivedData/GRDB-gdcllwrgfoeepnekbbfhunfrgxgp/Build/Intermediates/sqlcipher.build/Release-iphoneos/sqlcipher.build/Objects-normal/armv7s/sqlite3.o sqlite3.c normal armv7s c com.apple.compilers.llvm.clang.1_0.compiler
(3 failures)
configure: error: in `/Users/raymond/Documents/Xcode/KSClaim/Carthage/Checkouts/GRDB.swift/SQLCipher/src':
configure: error: C compiler cannot create executables
clang: error: no such file or directory: '/Users/raymond/Documents/Xcode/KSClaim/Carthage/Checkouts/GRDB.swift/SQLCipher/src/sqlite3.c'
clang: error: no input files
clang: error: no such file or directory: '/Users/raymond/Documents/Xcode/KSClaim/Carthage/Checkouts/GRDB.swift/SQLCipher/src/sqlite3.c'
clang: error: no input files
clang: error: no such file or directory: '/Users/raymond/Documents/Xcode/KSClaim/Carthage/Checkouts/GRDB.swift/SQLCipher/src/sqlite3.c'
clang: error: no input files
A shell task failed with exit code 65:
** BUILD FAILED **

[groue]

Hello @blue-toe

Assuming you are using the latest versions of Carthage and GRDB, please try again with one extra option:

carthage update --use-submodules

Tell me if this works (it does for me). I'll then update the documentation.

[blue-toe]

Hi Groue,

Thanks for the ultra quick response!

I am using the latest Carthage and did an "git init" in my project directory.

Executing "carthage update --use-submodules" as you suggested seems to work. I note that the OS X platform schemes are also built prior to the iOS schemes.

When I just use "carthage update --platform iOS --use-submodules", the build process will fail.

[groue]

@blue-toe: actually I misunderstood the effect of the --use-submodules option: it has nothing to do with building frameworks that rely on submodules, as GRDB does, but instead checks out dependencies as submodules: https://github.com/Carthage/Carthage#using-submodules-for-dependencies.

So I think we may face a bug in Carthage: you should open an issue at https://github.com/carthage/carthage, with GRDB as the evidence of the bug.

As a workaround, it looks like we have both settled that you can build the GRBD frameworks (included unneeded ones) with --use-submodules, and remove the unwanted frameworks and submodules next after.

[robertjpayne]

@groue I'm looking at this too right now, Carthage 0.16.2 correctly checks out the submodules the issue is something in the build steps that isn't properly running the amalgamation configuration.

[groue]

Hello @robertjpayne. I agree with you. The trouble may well lie in the Xcode project that builds SQLite with SQLCipher support, and not in Carthage. I don't know yet.

[robertjpayne]

@groue it's that amalgamation step, it just doesn't generate the sqlite.c file where the next make step expects it.

[robertjpayne]

@groue need to be able to run xcodebuild -workspace GRDB.xcworkspace -scheme GRDBCipheriOS -sdk iphoneos -configuration Release clean build on a fresh checkout from git, it clearly fails really fast on missing sqlite.c

[robertjpayne]

@groue aha the build step fails from the CLI:

checking build system type... i386-apple-darwin15.4.0
checking host system type... i386-apple-darwin15.4.0
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `**********/Carthage/Checkouts/GRDB.swift/SQLCipher/src':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** No rule to make target `sqlite3.c'.  Stop.

Particularly looks like configure: error: C compiler cannot create executables

[robertjpayne]

@groue anyways have to go for tonight, need to track down why ./configure fails from CLI but not Xcode.

[groue]

Thanks for your work Robert! Have a good night, I'll have a look as well.

[robertjpayne]

@groue turns out it's because SQLCipher's static lib target expects to inherit the current sdk/platform and architecture(s) from the parent target in Xcode.

Without this it defaults to x86_64 which is an invalid combo with SDK iphoneos.

The solution for this is going to be pretty awful since the ./configure probably shouldn't be shared between architectures.

[robertjpayne]

@groue so I think this will work:

set -e

if [ "$PLATFORM_NAME" -eq "macosx" ]
then
SDK_PLATFORM_NAME="macosx"
else
SDK_PLATFORM_NAME="iphonesimulator"
fi

SDKROOT="$(xcrun --sdk $SDK_PLATFORM_NAME --show-sdk-path)"
CC="$(xcrun --sdk $SDK_PLATFORM_NAME -f clang)"
CXX="$(xcrun --sdk $SDK_PLATFORM_NAME -f clang++)"
CFLAGS="-isysroot $SDKROOT -DSQLITE_HAS_CODEC -DSQLITE_TEMP_STORE=2"
CXXFLAGS=$CFLAGS

export CC CXX CFLAGS CXXFLAGS

./configure --enable-tempstore=yes --with-crypto-lib=commoncrypto

make sqlite3.c

exit 0

[groue]

@robertjpayne I agree that something is wrong with the environment variables. It would be great if we had not to change the script whenever we need to build for another SDK (iPhone device, tvOS, watchOS...).

[robertjpayne]

@groue you don't need to change the script, this works for all the platforms, it only uses the 'hacked' platform for the configuration step that puts together the amalgamation file, the compile step still happens for the destination target/platform/arch

[groue]

@robertjpayne There's another track I'm following: I've noticed that the sqlipher target embedded in the sqlcipher Xcode project is an OSX target that happens to be used for iOS as well. This already produces some oddities, such as GRDBCipher iOS/OSX being kind of ambiguous on their target platform (Xcode doesn't automatically select the correct platform when testing those targets). I'm looking at what happens when I define two separate targets with clear and distinct platforms. I expect Xcode (IDE & CLI) to produce the correct environment variables then.

[groue]

you don't need to change the script, this works for all the platforms, it only uses the 'hacked' platform for the configuration step that puts together the amalgamation file, the compile step still happens for the destination target/platform/arch

OK, I see. New platforms would just compile fine, with an amalgamation built on a "hacked" platform, as you say. You already have more success than I do :-)

[groue]

@robertjpayne I've tried to replace the amalgamation script with yours, and I'm still pretty confused. Basically, it sometimes works, sometimes does not (by "it works" I mean successfully running carthage update in a directory that only contains a Cartfile).

[groue]

@robertjpayne @blue-toe GRDB v0.59.1 is out, with fixed Carthage support.

Robert, you were very very close, and I thank you very much. The only missing line in your build script was [ -e Makefile ] && make distclean so that carthage update would reliably build GRDB.

Carthage is weird in that it does not build frameworks in a stable order. And depending whether GRDBCipherOSX or GRDBCipheriOS would build first, the build would fail or succeed. The extra make distclean restores SQLCipher on a clean state before each build.