Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmake: strip_nondeterminism_if_found() on all .a library files #17494

Closed

Conversation

marc-hb
Copy link
Collaborator

@marc-hb marc-hb commented Jul 12, 2019

Post process libraries if find_program(strip-nondeterminism) succeeds,
do nothing otherwise.

This has been successfully tested on Ubuntu 18.04 with a full
sanitycheck run including all the default samples/ and tests/

Two very important caveats:

  1. This required a couple of fixes on top of "apt-get install
    non-determinism" version 0.040-1.1~build1. These fixes are available
    at https://salsa.debian.org/reproducible-builds/strip-nondeterminism/merge_requests/4

  2. The entire ar.pm handler has been removed in more recent (1.0.0+)
    strip-nondeterminism releases! The commit message 366d60c9cc16 says:
    "Drop .ar handler; binutils is reproducible". The corresponding bugs
    and discussions seem to show a concern than ar.pm could hide bugs in
    binutils or llvm-ar. Supporting older or non-debian systems doesn't
    seem to be desired.

So the preferred alternative is to use a toolchain like the Zephyr SDK
that supports creating deterministic .a files and (if not the default)
pass the -D option to (GNU) ar thanks to
CMAKE_LANG_CREATE_STATIC_LIBRARY

Signed-off-by: Marc Herbert marc.herbert@intel.com

Post process libraries if find_program(strip-nondeterminism) succeeds,
do nothing otherwise.

This has been successfully tested on Ubuntu 18.04 with a full
sanitycheck run including all the default samples/ and tests/

Two very important caveats:

1. This required a couple of fixes on top of "apt-get install
   non-determinism" version 0.040-1.1~build1. These fixes are available
   at
   https://github.com/marc-hb/strip-nondeterminism/commits/ar-handler-fixes

2. The entire ar.pm handler has been removed in more recent (1.0.0+)
   strip-nondeterminism releases! The commit message 366d60c9cc16 says:
   "Drop .ar handler; binutils is reproducible". The corresponding bugs
   and discussions seem to show a concern than ar.pm could hide bugs in
   binutils or llvm-ar. Supporting older or non-debian systems doesn't
   seem to be desired.

So the preferred alternative is to use a toolchain like the Zephyr SDK
that supports creating deterministic .a files and (if not the default)
pass the -D option to (GNU) ar thanks to
CMAKE_LANG_CREATE_STATIC_LIBRARY

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
@marc-hb
Copy link
Collaborator Author

marc-hb commented Jul 12, 2019

ar -D is submitted for the Zephyr SDK in PR #17495

@marc-hb
Copy link
Collaborator Author

marc-hb commented Jul 14, 2019

See https://salsa.debian.org/reproducible-builds/strip-nondeterminism/issues/3
"Process to remove unused handlers over time"

@lamby
Copy link

lamby commented Aug 2, 2019

FYI I plan to work on this over the next week or so and thus a custom package will not be required.

@github-actions github-actions bot added has-conflicts Issue/PR has conflicts with another issue/PR and removed has-conflicts Issue/PR has conflicts with another issue/PR labels Jun 29, 2020
@nashif nashif added the Stale label Jul 8, 2020
@nashif
Copy link
Member

nashif commented Jul 10, 2020

stale for almost a year...

@nashif nashif closed this Jul 10, 2020
@lamby
Copy link

lamby commented Jul 11, 2020

Nod. From quickly loading this back into my brain, what we did in strip-nondeterminism was to re-add the .ar handler and allow it to be explicitly turned on.

@marc-hb
Copy link
Collaborator Author

marc-hb commented Jul 11, 2020

In the mean time crosstool-ng finally caught up with other toolchains and finally turn on --enable-deterministic-archives by default so while rare sometimes bugs do fix themselves :-)
zephyrproject-rtos/sdk-ng#81

At least for the Zephyr SDK that is. Other toolchains are free to resubmit this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Build System area: Kernel has-conflicts Issue/PR has conflicts with another issue/PR Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants