/bugtracker

fully reproducible builds #743

Closed
thestinger opened this Issue Sep 22, 2017 · 5 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
1 participant
@thestinger
@thestinger
Contributor

thestinger commented Sep 22, 2017 •
edited
Edited 1 time
  • @thestinger thestinger edited Sep 22, 2017

Non-determinism in the builds for Oreo-based CopperheadOS:

META/file_contexts.bin: Binary files differ
SYSTEM/lib/libblas.so: Binary files differ
SYSTEM/lib/libwebrtc_audio_preprocessing.so: Binary files differ
SYSTEM/lib/vndk-sp/libblas.so: Binary files differ
SYSTEM/lib64/libpdfium.so: Binary files differ
SYSTEM/lib64/libwebrtc_audio_preprocessing.so: Binary files differ
SYSTEM/priv-app/Dialer/oat/arm64/Dialer.odex: Binary files differ
SYSTEM/priv-app/Dialer/oat/arm64/Dialer.vdex: Binary files differ

@thestinger thestinger added Type: enhancement History: oreo-port labels Sep 22, 2017

@thestinger thestinger changed the title from reproducible builds to fully reproducible builds Sep 22, 2017

@thestinger thestinger added Priority: low and removed History: oreo-port labels Sep 22, 2017

@thestinger

This comment has been minimized.

Show comment Hide comment
@thestinger

thestinger Sep 22, 2017

Contributor

C and C++ compilation output differences can be tracked back to intermediate files.

Differences in static library intermediates:

Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/lcms2-2.6/src/cmsgamma.o and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/lcms2-2.6/src/cmsgamma.o differ
Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/libpdfiumlcms.a and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/libpdfiumlcms.a differ
diff '--color=auto' -r out_one/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c out_two/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:59 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:21 2017. */
diff '--color=auto' -r out_one/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h out_two/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:59 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:21 2017. */
Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o differ
Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a differ

Differences in shared library intermediates:

Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/LINKED/libpdfium.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/LINKED/libpdfium.so differ
Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/PACKED/libpdfium.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/PACKED/libpdfium.so differ
diff '--color=auto' -r out_one/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c out_two/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
diff '--color=auto' -r out_one/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h out_two/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so differ
Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so differ

Differences in 32-bit static library intermediates:

Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/double.o and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/double.o differ
Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/libF77blas.a and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/libF77blas.a differ
Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o differ
Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a differ

Differences in 32-bit shared library intermediates:

Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/LINKED/libblas.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/LINKED/libblas.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/PACKED/libblas.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/PACKED/libblas.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/LINKED/libblas.vndk-sp.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/LINKED/libblas.vndk-sp.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/PACKED/libblas.vndk-sp.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/PACKED/libblas.vndk-sp.so differ
diff '--color=auto' -r out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
diff '--color=auto' -r out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so differ
Contributor

thestinger commented Sep 22, 2017 •
edited
Edited 1 time
  • @thestinger thestinger edited Sep 22, 2017

C and C++ compilation output differences can be tracked back to intermediate files.

Differences in static library intermediates:

Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/lcms2-2.6/src/cmsgamma.o and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/lcms2-2.6/src/cmsgamma.o differ
Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/libpdfiumlcms.a and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libpdfiumlcms_intermediates/libpdfiumlcms.a differ
diff '--color=auto' -r out_one/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c out_two/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:59 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:21 2017. */
diff '--color=auto' -r out_one/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h out_two/target/product/sailfish/obj/STATIC_LIBRARIES/librilutils_static_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:59 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:21 2017. */
Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o differ
Binary files out_one/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a and out_two/target/product/sailfish/obj/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a differ

Differences in shared library intermediates:

Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/LINKED/libpdfium.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/LINKED/libpdfium.so differ
Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/PACKED/libpdfium.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libpdfium_intermediates/PACKED/libpdfium.so differ
diff '--color=auto' -r out_one/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c out_two/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
diff '--color=auto' -r out_one/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h out_two/target/product/sailfish/obj/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so differ
Binary files out_one/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so differ

Differences in 32-bit static library intermediates:

Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/double.o and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/double.o differ
Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/libF77blas.a and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libF77blas_intermediates/libF77blas.a differ
Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/entropy_coding.o differ
Binary files out_one/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a and out_two/target/product/sailfish/obj_arm/STATIC_LIBRARIES/libwebrtc_isac_intermediates/libwebrtc_isac.a differ

Differences in 32-bit shared library intermediates:

Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/LINKED/libblas.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/LINKED/libblas.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/PACKED/libblas.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas_intermediates/PACKED/libblas.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/LINKED/libblas.vndk-sp.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/LINKED/libblas.vndk-sp.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/PACKED/libblas.vndk-sp.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libblas.vndk-sp_intermediates/PACKED/libblas.vndk-sp.so differ
diff '--color=auto' -r out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.c
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
diff '--color=auto' -r out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/librilutils_intermediates/proto/hardware/ril/librilutils/proto/sap-api.pb.h
2c2
< /* Generated by nanopb-0.2.8-dev at Thu Sep 21 23:13:44 2017. */
---
> /* Generated by nanopb-0.2.8-dev at Thu Sep 21 20:20:11 2017. */
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/LINKED/libwebrtc_audio_preprocessing.so differ
Binary files out_one/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so and out_two/target/product/sailfish/obj_arm/SHARED_LIBRARIES/libwebrtc_audio_preprocessing_intermediates/PACKED/libwebrtc_audio_preprocessing.so differ
@thestinger

This comment has been minimized.

Show comment Hide comment
@thestinger

thestinger Sep 22, 2017

Contributor

Seems like one Clang non-determinism issue tied to vector instructions and then some other ART issue where --force-determinism isn't working for Dialer ahead-of-time compilation.
Contributor

thestinger commented Sep 22, 2017

Seems like one Clang non-determinism issue tied to vector instructions and then some other ART issue where --force-determinism isn't working for Dialer ahead-of-time compilation.
@thestinger

This comment has been minimized.

Show comment Hide comment
@thestinger

thestinger Sep 22, 2017

Contributor

ART issue can apparently happen with other apps, but it seems more likely as the apps get larger.
Contributor

thestinger commented Sep 22, 2017

ART issue can apparently happen with other apps, but it seems more likely as the apps get larger.

@thestinger thestinger added the upstream label Nov 4, 2017

@thestinger

This comment has been minimized.

Show comment Hide comment
@thestinger

thestinger Nov 5, 2017

Contributor

Clang issue is resolved in oreo-dr1-release. ART dexpreopt issue still appears to be present.
Contributor

thestinger commented Nov 5, 2017

Clang issue is resolved in oreo-dr1-release. ART dexpreopt issue still appears to be present.
@thestinger

This comment has been minimized.

Show comment Hide comment
@thestinger

thestinger Dec 14, 2017

Contributor

These issues are resolved.
Contributor

thestinger commented Dec 14, 2017

These issues are resolved.

@thestinger thestinger closed this Dec 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment