2020-08-17: [VWS/Covid19NotificationApp] Reproducible Builds

[raboof]

Describe the bug, issue or concern

As discussed previously in general at minvws/nl-covid19-notification-app-coordination#6, while it is great that the provenance chain has been audited (https://github.com/minvws/nl-covid19-notification-app-provenance), an even stronger verification that no-where in the (technical) process a step was compromised would be to use Reproducible Builds to allow all citizens to independently verify that the published binaries in the play store are actually consistent with the tagged source code.

To Reproduce

Personally I'm not an Android developer, so I hope to learn from others in this thread.

The steps to check reproducibility are generally:

• download the packages from the play store
• check out the tag from git
• build the code
• compare the result with the package downloaded from the play store.

I can build the code with ./gradlew bundleRelease (after generating a keystore and correctly setting KEYSTORE_KEY_ALIAS, KEYSTORE_KEY_PASSWORD and KEYSTORE_PASSWORD environment variables). This produces a app/build/outputs/bundle/prodRelease/covid-notificatie-v1.0.0-1-prod-release.aab , but I'm not sure yet if that's the right starting point and how to compare that to the packages from the play store.

I can understand if this is not the main focus of the VWS team right now, but I hope this issue might at least be a good place for contributors to share their knowledge and findings.

Expected behavior

It would be great if we could come up with the steps needed to independently validate the published packages correspond to the published source code. Of course it's only one link in the chain, but an important one!

Governance

• I've read the contributing document https://github.com/minvws/.github/blob/master/CONTRIBUTING.md
• I've read and understand the Code of Conduct https://github.com/minvws/.github/blob/master/CODE_OF_CONDUCT.md
• I understand that any contributions or suggestions I made may make it into the actual code. I've read the License https://github.com/minvws/nl-covid19-notification-app-coordination/blob/master/LICENSE.txt and the contributor license agreement https://github.com/minvws/nl-covid19-notification-app-coordination/blob/master/CLA.md

[PanderMusubi]

This #9 would have been a nice way to do that, however it has some show stoppers. Would have a check effect as F-Droid also has its own reproducible builds.

[hvisser]

The steps you describe are mostly correct, you'd also have to set BUILD_ID because that's also embedded into the app, and build a release build with an alternative key store that you can supply as you mention (the git hash is only in a release build).

Getting the app from the store is tricky, there's no official way to retrieve it from the Play Store AFAIK.

As for coming up with the steps, we'd appreciate help from the community here. Our primary proof of what is shipped is currently through the auditing process and in that process the process from source to build to deployment has also be vetted.

[ruuda]

but I'm not sure yet if that's the right starting point and how to compare that to the packages from the play store.

You can list the path to the apk of an installed package with pm path $PACKAGE. You can get a shell to run that command in with adb shell, and then pull the apk from the phone with adb pull. You can list installed packages with pm list. The package name in this case is nl.rijksoverheid.en. In my case it reports two apks, base.apk and split_config.xxhdpi.apk.

The build process should also produce an apk at some point. The apks can then be compared with diffoscope, which has special support for apks, and it will point out differences at the lowest possible levels.

[hvisser]

It's probably better to diff base.apk only, since the produced APK will be a universal APK, while the Play Store will serve only the base apk + the splits that are needed for the device.

Here's a starting point to try:

• Get the base.apk from your device. Use adb shell pm path nl.rijksoverheid.en to list the path, then use adb pull path/to/base.apk to get the apk.
• Clone the repo and checkout the v1.0.0 tag git checkout v1.0.0
• Build and sign the release bundle of the app. If you have Android Studio installed and have built an app before this should probably work KEYSTORE_FILE=~/.android/debug.keystore KEYSTORE_PASSWORD=android KEYSTORE_KEY_ALIAS=androiddebugkey KEYSTORE_KEY_PASSWORD=android BUILD_ID=68251 ./gradlew bundleProdRelease -- you may need to create a dummy key store file first otherwise.
• Get bundletool: https://github.com/google/bundletool/releases
• Create the base apk from the bundle: java -jar bundletool-all-1.1.0.jar build-apks --bundle=app/build/outputs/bundle/prodRelease/covid-notificatie-v1.0.0-68251-prod-release.aab --output reproduce/coronamelder.apks
• unzip the reproduce/coronamelder.apks file.
• Run diffscope (assuming you unzipped in the reproduce directory: docker run --rm -t -w $(pwd) -v $(pwd):$(pwd):ro \ registry.salsa.debian.org/reproducible-builds/diffoscope base.apk reproduce/splits/base-master.apk

Running this there are still some resource differences (Play Store can optimise these) and only one diff in a decompiled source file, which might be due to some compilation differences on my local machine or R8, but I'm not sure. Pretty close though!

The other APKs on the device are for languages and resources, again those might be optimised by the Play Store and therefore differ from what bundletool is producing.

Would be great if folks can try, comment and further document it.

[dennisjaheruddin]

Not sure if this mitigates the problem, but if one cannot pull a clean APK from the store, a decent alternative might be to have reproducible builds for an APK outside of the app store. The users who want that extra bit of proof can then manually install the app, and the vast majority who just wants convenience can of course still use the store.