-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reproducible Builds #31
Comments
Hello, thanks for your feedback! However, maybe the last build of the app, in the release section, was built before the merge of the develop branch to master. I don't quite remember and I don't know if that matters. |
Check my nick, make a good guess 😉 Yes, I'm the very Izzy behind it. So if you have questions towards the listing there, or how your app and its metadata go there, be welcome to ask. But what was checked here was the APK at your release – against the APK built from the commit the release tag points to.
Oh, I can assure you it matters a lot. That's "§1" of the RB rules: if you want to make sure RB fails, do what you just described 😜 If you want it to succeed, always attach the APK built from a clean tree behind the commit the release tag points to. See the above linked hints on reproducible builds, under §1 😉
You can leave the "changes on the IzzyOnDroid side" to me, I'll take care for them.
Yay, cool! Can't ask for more. Except maybe for giving me a ping once the next release is out, so I can check again? Thanks a lot! |
I have released a new version of Soul Searching. The apk was built on the commit the release follows (commit 96c1316 on master branch). I have checked the page on reproducible builds and I think that it should be good now 👍 . Let me know if everything is okay in your side. |
Unfortunately, not RB. Quite a load of differences in the APKs… wait:
Umpf. But you
Hm, "first basic rule": always build the APK from a clean tree at the commit the tag points to. Not the next tag 😝 Because that means we have to manually specify the commit to build from – so the recipe cannot be reused for the next release, which then will fail because it's verification APK would be built from the same commit again. But then, still no RB. Sure the tree was clean (no uncommitted changes, no cached artifacts)? - Archive: 61b218f72fa5eadf9b696d88e9437e17b840dc739be3a0c8bed4d96eb31c89ab-com.github.enteraname74.soulsearching-v0.7.0-upstream.apk
- Zip file size: 16307178 bytes, number of entries: 525
- -rw-r--r-- 0.0 unx 56 b- 52 defN 1981-01-01 01:01:02 2ee0d5b5 META-INF/com/android/build/gradle/app-metadata.properties
- -rw-r--r-- 0.0 unx 5085 b- 5085 stor 1981-01-01 01:01:02 b4fbfadc assets/dexopt/baseline.prof
- -rw-r--r-- 0.0 unx 631 b- 631 stor 1981-01-01 01:01:02 be0582ed assets/dexopt/baseline.profm
- -rw-r--r-- 0.0 unx 4325596 b- 2107333 defN 1981-01-01 01:01:02 24fe876f classes.dex
+ Archive: 0da70385d0a58d8361240329355418ccb07f6a58f995367cc965968e526dd795-com.github.enteraname74.soulsearching-v0.7.0-unsigned.apk
+ Zip file size: 16238762 bytes, number of entries: 525
+ -rw-r--r-- 0.0 unx 56 b- 51 defN 1981-01-01 01:01:02 2ee0d5b5 META-INF/com/android/build/gradle/app-metadata.properties
+ -rw-r--r-- 0.0 unx 5225 b- 5225 stor 1981-01-01 01:01:02 f5907ccb assets/dexopt/baseline.prof
+ -rw-r--r-- 0.0 unx 647 b- 647 stor 1981-01-01 01:01:02 b342095f assets/dexopt/baseline.profm
+ -rw-r--r-- 0.0 unx 4325596 b- 2051274 defN 1981-01-01 01:01:02 24fe876f classes.dex
-rw-r--r-- 0.0 unx 35 b- 37 defN 1981-01-01 01:01:02 dfa4bb33 assets/$catalog.json
-rw-r--r-- 0.0 unx 35 b- 37 defN 1981-01-01 01:01:02 2b201084 assets/META-INF/$catalog.json
-rw-r--r-- 0.0 unx 33 b- 35 defN 1981-01-01 01:01:02 31cab400 assets/META-INF/proguard/$catalog.json
- -rw-r--r-- 0.0 unx 178 b- 104 defN 1981-01-01 01:01:02 a83b8f6d assets/META-INF/proguard/korio.pro
- -rw---- 2.0 fat 1738 b- 788 defN 1981-01-01 01:01:02 2d73be70 DebugProbesKt.bin
+ -rw-r--r-- 0.0 unx 178 b- 103 defN 1981-01-01 01:01:02 a83b8f6d assets/META-INF/proguard/korio.pro
+ -rw---- 2.0 fat 1738 b- 782 defN 1981-01-01 01:01:02 2d73be70 DebugProbesKt.bin
-rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 996dd6a3 META-INF/androidx.activity_activity-compose.version
-rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 996dd6a3 META-INF/androidx.activity_activity-ktx.version
-rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 996dd6a3 META-INF/androidx.activity_activity.version
@@ -77,7 +77,7 @@
-rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 2dd8d84e META-INF/services/hb.d
-rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 aeceed5e META-INF/services/pc.y
-rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 85e3be9d META-INF/services/uc.r
- -rw---- 2.0 fat 2518 b- 1497 defN 1981-01-01 01:01:02 ddcb631c META-INF/versions/9/previous-compilation-data.bin
+ -rw---- 2.0 fat 2518 b- 1499 defN 1981-01-01 01:01:02 ddcb631c META-INF/versions/9/previous-compilation-data.bin
-rw---- 2.0 fat 103798 b- 103798 stor 1981-01-01 01:01:02 4c8596db drawable/add_new_songs_settings.png
-rw---- 2.0 fat 39122 b- 39122 stor 1981-01-01 01:01:02 a23f18b2 drawable/app_icon.png
-rw---- 2.0 fat 574782 b- 574782 stor 1981-01-01 01:01:02 f41284e1 drawable/dynamic_main.png
@@ -88,20 +88,20 @@
-rw---- 2.0 fat 1079063 b- 1079063 stor 1981-01-01 01:01:02 b880c9c2 drawable/perso_first_example.png
-rw---- 2.0 fat 571985 b- 571985 stor 1981-01-01 01:01:02 75fbd617 drawable/perso_second_example.png
-rw---- 2.0 fat 562131 b- 562131 stor 1981-01-01 01:01:02 d9d09875 drawable/perso_third_example.png
- -rw---- 2.0 fat 1465 b- 853 defN 1981-01-01 01:01:02 476527d0 drawable/saxophone.svg
+ -rw---- 2.0 fat 1465 b- 848 defN 1981-01-01 01:01:02 476527d0 drawable/saxophone.svg
-rw---- 2.0 fat 39122 b- 39122 stor 1981-01-01 01:01:02 a23f18b2 drawable/saxophone_png.png
-rw---- 2.0 fat 569179 b- 569179 stor 1981-01-01 01:01:02 7acb1f0e drawable/system_dark_theme_main.png
-rw---- 2.0 fat 646221 b- 646221 stor 1981-01-01 01:01:02 0f8a32a6 drawable/system_dark_theme_player.png
-rw---- 2.0 fat 1182750 b- 1182750 stor 1981-01-01 01:01:02 a8ad2d3e drawable/system_light_theme_main.png
-rw---- 2.0 fat 644432 b- 644432 stor 1981-01-01 01:01:02 00689bf3 drawable/system_light_theme_player.png
- -rw---- 2.0 fat 987 b- 344 defN 1981-01-01 01:01:02 2507751e kotlin-tooling-metadata.json
- -rw---- 2.0 fat 928 b- 570 defN 1981-01-01 01:01:02 8e787046 kotlin/annotation/annotation.kotlin_builtins
- -rw---- 2.0 fat 3685 b- 1554 defN 1981-01-01 01:01:02 f41c30cf kotlin/collections/collections.kotlin_builtins
- -rw---- 2.0 fat 200 b- 161 defN 1981-01-01 01:01:02 c69fe143 kotlin/coroutines/coroutines.kotlin_builtins
- -rw---- 2.0 fat 646 b- 395 defN 1981-01-01 01:01:02 d8d2330d kotlin/internal/internal.kotlin_builtins
- -rw---- 2.0 fat 18661 b- 5363 defN 1981-01-01 01:01:02 b623e8f5 kotlin/kotlin.kotlin_builtins
- -rw---- 2.0 fat 3399 b- 1320 defN 1981-01-01 01:01:02 cb3b8aed kotlin/ranges/ranges.kotlin_builtins
- -rw---- 2.0 fat 2426 b- 1318 defN 1981-01-01 01:01:02 7e3e3d18 kotlin/reflect/reflect.kotlin_builtins
+ -rw---- 2.0 fat 987 b- 341 defN 1981-01-01 01:01:02 2507751e kotlin-tooling-metadata.json
+ -rw---- 2.0 fat 928 b- 561 defN 1981-01-01 01:01:02 8e787046 kotlin/annotation/annotation.kotlin_builtins
+ -rw---- 2.0 fat 3685 b- 1521 defN 1981-01-01 01:01:02 f41c30cf kotlin/collections/collections.kotlin_builtins
+ -rw---- 2.0 fat 200 b- 157 defN 1981-01-01 01:01:02 c69fe143 kotlin/coroutines/coroutines.kotlin_builtins
+ -rw---- 2.0 fat 646 b- 394 defN 1981-01-01 01:01:02 d8d2330d kotlin/internal/internal.kotlin_builtins
+ -rw---- 2.0 fat 18661 b- 5257 defN 1981-01-01 01:01:02 b623e8f5 kotlin/kotlin.kotlin_builtins
+ -rw---- 2.0 fat 3399 b- 1292 defN 1981-01-01 01:01:02 cb3b8aed kotlin/ranges/ranges.kotlin_builtins
+ -rw---- 2.0 fat 2426 b- 1286 defN 1981-01-01 01:01:02 7e3e3d18 kotlin/reflect/reflect.kotlin_builtins
-rw---- 0.0 fat 8964 b- 2190 defN 1981-01-01 01:01:02 f5cb1a28 AndroidManifest.xml Looking at the file sizes, this seems to be a compression issue. For example, the above diff shows @obfusk had some trick for that which is missing in my notes, as this is the first time I encounter a case like that. Maybe she shares with us 😃 |
The v0.7.0 tag points to 96c1316, which seems correct?
Yes and no. The CRC32 is identical, so the diff shows the uncompressed data is the same, but the size isn't. Except for
Eh... that's not supposed to happen. Somehow the upstream APK has compressed data that cannot be reproduced with If so, there is no way to fix this. Assuming it's not the JDK or similar and a clean CLI build doesn't help either, upgrading AGP might be worth trying. |
This looks to be the result of a different zlib/deflate implementation being used to create the APK. @enteraname74 can you provide us some details on the OS and build environment you used to create the APK? |
Sorry for the late answer, I was busy on other things. Anyway, here is the config I use for building a release of Soul Searching: OS: Fedora 40 |
It looks like Fedora switched from zlib to zlib-ng, which might be the cause of the different compressed data. Unfortunately, we don't currently support Fedora for our rebuilder so we can't easily confirm this. But we'll look into it. Thanks! |
Which Java 17? The Fedora package? Or another JDK? |
Output of
|
Are you sure Android Studio (assuming you use that) is configured to use that JDK? Because it often uses a bundled one instead and I think I have the problem figured out and the OpenJDK package from Fedora seems unaffected as it doesn't use the system zlib. |
You're right, I checked the JDK used by the project, and it was a jetbrains runtime (jbr-17) instead of openjdk that was used. |
Thanks! Any chance you could run In case you're interested in what my investigation turned up: https://lists.reproducible-builds.org/pipermail/rb-general/2024-September/003547.html |
Thanks for sharing the work behind your investigation!
|
Thanks for the output. That confirms JetBrains Java Runtime indeed uses the system zlib ( |
The APK at v0.8.1 seems still to be build using zlib-ng, as the different compression-levels show – and there are more differences even: -rw-r--r-- 0.0 unx 56 b- 52 defN 1981-01-01 01:01:02 981abecf META-INF/com/android/build/gradle/app-metadata.properties
- -rw-r--r-- 0.0 unx 120 b- 116 defN 1981-01-01 01:01:02 6eec8c50 META-INF/version-control-info.textproto
- -rw-r--r-- 0.0 unx 5155 b- 5155 stor 1981-01-01 01:01:02 531a53af assets/dexopt/baseline.prof
- -rw-r--r-- 0.0 unx 684 b- 684 stor 1981-01-01 01:01:02 c85926a5 assets/dexopt/baseline.profm
- -rw-r--r-- 0.0 unx 4179224 b- 2044812 defN 1981-01-01 01:01:02 9f3c7853 classes.dex
+ -rw-r--r-- 0.0 unx 120 b- 118 defN 1981-01-01 01:01:02 6eec8c50 META-INF/version-control-info.textproto
+ -rw-r--r-- 0.0 unx 5336 b- 5336 stor 1981-01-01 01:01:02 2ba952ba assets/dexopt/baseline.prof
+ -rw-r--r-- 0.0 unx 704 b- 704 stor 1981-01-01 01:01:02 ca29097a assets/dexopt/baseline.profm
+ -rw-r--r-- 0.0 unx 4179224 b- 1988883 defN 1981-01-01 01:01:02 9f3c7853 classes.dex
-rw-r--r-- 0.0 unx 35 b- 37 defN 1981-01-01 01:01:02 dfa4bb33 assets/$catalog.json
-rw-r--r-- 0.0 unx 35 b- 37 defN 1981-01-01 01:01:02 2b201084 assets/META-INF/$catalog.json
-rw-r--r-- 0.0 unx 33 b- 35 defN 1981-01-01 01:01:02 31cab400 assets/META-INF/proguard/$catalog.json
- -rw-r--r-- 0.0 unx 178 b- 104 defN 1981-01-01 01:01:02 a83b8f6d assets/META-INF/proguard/korio.pro
- -rw---- 2.0 fat 1728 b- 784 defN 1981-01-01 01:01:02 c8e37c1f DebugProbesKt.bin
+ -rw-r--r-- 0.0 unx 178 b- 103 defN 1981-01-01 01:01:02 a83b8f6d assets/META-INF/proguard/korio.pro
+ -rw---- 2.0 fat 1728 b- 777 defN 1981-01-01 01:01:02 c8e37c1f DebugProbesKt.bin
-rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 b2408560 META-INF/androidx.activity_activity-compose.version
-rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 b2408560 META-INF/androidx.activity_activity-ktx.version
-rw---- 2.0 fat 6 b- 8 defN 1981-01-01 01:01:02 b2408560 META-INF/androidx.activity_activity.version
@@ -78,7 +78,7 @@
-rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 452db45d META-INF/services/i6.v
-rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 6e00e79e META-INF/services/j6.a
-rw---- 2.0 fat 5 b- 7 defN 1981-01-01 01:01:02 a4d2c6b6 META-INF/services/v5.d
- -rw---- 2.0 fat 2518 b- 1497 defN 1981-01-01 01:01:02 ddcb631c META-INF/versions/9/previous-compilation-data.bin
+ -rw---- 2.0 fat 2518 b- 1499 defN 1981-01-01 01:01:02 ddcb631c META-INF/versions/9/previous-compilation-data.bin
-rw---- 2.0 fat 39122 b- 39122 stor 1981-01-01 01:01:02 a23f18b2 composeResources/com.github.enteraname74.soulsearching.coreui.core_ui.generated.resources/drawable/app_icon.png
-rw---- 2.0 fat 103798 b- 103798 stor 1981-01-01 01:01:02 4c8596db composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/add_new_songs_settings.png
-rw---- 2.0 fat 574782 b- 574782 stor 1981-01-01 01:01:02 f41284e1 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/dynamic_main.png
@@ -89,20 +89,20 @@
-rw---- 2.0 fat 1079063 b- 1079063 stor 1981-01-01 01:01:02 b880c9c2 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/perso_first_example.png
-rw---- 2.0 fat 571985 b- 571985 stor 1981-01-01 01:01:02 75fbd617 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/perso_second_example.png
-rw---- 2.0 fat 562131 b- 562131 stor 1981-01-01 01:01:02 d9d09875 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/perso_third_example.png
- -rw---- 2.0 fat 1465 b- 853 defN 1981-01-01 01:01:02 476527d0 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/saxophone.svg
+ -rw---- 2.0 fat 1465 b- 848 defN 1981-01-01 01:01:02 476527d0 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/saxophone.svg
-rw---- 2.0 fat 39122 b- 39122 stor 1981-01-01 01:01:02 a23f18b2 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/saxophone_png.png
-rw---- 2.0 fat 569179 b- 569179 stor 1981-01-01 01:01:02 7acb1f0e composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/system_dark_theme_main.png
-rw---- 2.0 fat 646221 b- 646221 stor 1981-01-01 01:01:02 0f8a32a6 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/system_dark_theme_player.png
-rw---- 2.0 fat 1182750 b- 1182750 stor 1981-01-01 01:01:02 a8ad2d3e composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/system_light_theme_main.png
-rw---- 2.0 fat 644432 b- 644432 stor 1981-01-01 01:01:02 00689bf3 composeResources/com.github.enteraname74.soulsearching.shared_ui.generated.resources/drawable/system_light_theme_player.png
- -rw---- 2.0 fat 987 b- 345 defN 1981-01-01 01:01:02 31ea6192 kotlin-tooling-metadata.json
- -rw---- 2.0 fat 928 b- 570 defN 1981-01-01 01:01:02 8e787046 kotlin/annotation/annotation.kotlin_builtins
- -rw---- 2.0 fat 3685 b- 1554 defN 1981-01-01 01:01:02 f41c30cf kotlin/collections/collections.kotlin_builtins
- -rw---- 2.0 fat 200 b- 161 defN 1981-01-01 01:01:02 c69fe143 kotlin/coroutines/coroutines.kotlin_builtins
- -rw---- 2.0 fat 646 b- 395 defN 1981-01-01 01:01:02 d8d2330d kotlin/internal/internal.kotlin_builtins
- -rw---- 2.0 fat 18661 b- 5363 defN 1981-01-01 01:01:02 b623e8f5 kotlin/kotlin.kotlin_builtins
- -rw---- 2.0 fat 3399 b- 1320 defN 1981-01-01 01:01:02 cb3b8aed kotlin/ranges/ranges.kotlin_builtins
- -rw---- 2.0 fat 2426 b- 1318 defN 1981-01-01 01:01:02 7e3e3d18 kotlin/reflect/reflect.kotlin_builtins
+ -rw---- 2.0 fat 987 b- 343 defN 1981-01-01 01:01:02 31ea6192 kotlin-tooling-metadata.json
+ -rw---- 2.0 fat 928 b- 561 defN 1981-01-01 01:01:02 8e787046 kotlin/annotation/annotation.kotlin_builtins
+ -rw---- 2.0 fat 3685 b- 1521 defN 1981-01-01 01:01:02 f41c30cf kotlin/collections/collections.kotlin_builtins
+ -rw---- 2.0 fat 200 b- 157 defN 1981-01-01 01:01:02 c69fe143 kotlin/coroutines/coroutines.kotlin_builtins
+ -rw---- 2.0 fat 646 b- 394 defN 1981-01-01 01:01:02 d8d2330d kotlin/internal/internal.kotlin_builtins
+ -rw---- 2.0 fat 18661 b- 5257 defN 1981-01-01 01:01:02 b623e8f5 kotlin/kotlin.kotlin_builtins
+ -rw---- 2.0 fat 3399 b- 1292 defN 1981-01-01 01:01:02 cb3b8aed kotlin/ranges/ranges.kotlin_builtins
+ -rw---- 2.0 fat 2426 b- 1286 defN 1981-01-01 01:01:02 7e3e3d18 kotlin/reflect/reflect.kotlin_builtins
-rw---- 0.0 fat 8964 b- 2192 defN 1981-01-01 01:01:02 fa206a13 AndroidManifest.xml Was that APK already built with OpenJDK, or still with JetBrains? Apart from that, there are also differences in size of uncompressed data (e.g. the baselines) indicating there's probably more to be fixed. Funny thing with the baselines is that though their uncompressed size differs, diffing the content shows no difference – and those files are not compressed at all… So I'm rather clueless there. |
Apart from the |
And the |
Hey @enteraname74: CONGRATS! 🥳 I've just tested your latest release (v0.9.1), and it is RB!!! So here comes your welcome toot, the green shield is up, and this issue can be closed. Thanks for your hard work on this! 🤗 |
I've checked your app if its build is reproducible (see: Reproducible bulds, special client support and more in our repo), but while I was able to successfully generate the APK using
./gradlew assembleRelease
, the resulting APK was not identical to yours (the Dex diff is rather huuuge). Was that APK really built from the commit the tag points to? If so, did I miss some build options? And if not, which commit was it?APK diff:
We'd appreciate if you could help making your build reproducible. We've prepared some hints on reproducible builds for that.
Looking forward to your reply!
The text was updated successfully, but these errors were encountered: