Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdenv: make darwin builds reproducable #77632

Merged
merged 5 commits into from Jan 31, 2020
Merged

Conversation

@LnL7
Copy link
Member

@LnL7 LnL7 commented Jan 13, 2020

Motivation for this change

Fixes #21629

Here's an example using hello (without the stdenv changes) https://gist.github.com/LnL7/7e705ea7f4099c074a004c897810bc73

And a run of r13y over the darwin stdenv
https://dgn9s9wft3u6a.cloudfront.net/store/ppm2pvkw1ngvmw063qbkxsg7r0nvdcxi-report/index.html

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

/cc @NixOS/darwin-maintainers

@LnL7 LnL7 added this to Stdenv in Darwin Jan 13, 2020
@ofborg ofborg bot added the 6.topic: erlang label Jan 14, 2020
@LnL7 LnL7 force-pushed the LnL7:darwin-macos-version-min branch from 75c2904 to 1eed714 Jan 14, 2020
@LnL7 LnL7 changed the title [WIP] stdenv: make darwin builds reproducable stdenv: make darwin builds reproducable Jan 14, 2020
@ofborg ofborg bot requested review from tobim and Mic92 Jan 14, 2020
Copy link
Contributor

@tobim tobim left a comment

According to https://cmake.org/cmake/help/latest/variable/CMAKE_OSX_DEPLOYMENT_TARGET.html
the CMAKE_OSX_DEPLOYMENT_TARGET will be initialized from MACOSX_DEPLOYMENT_TARGET, so it should be safe to remove those flags from the affected packages.

let version = "10.12"; in

# Ensure appleSdkVersion is up to date.
assert stdenv.isDarwin -> stdenv.appleSdkVersion == version;

This comment has been minimized.

@tobim

tobim Jan 14, 2020
Contributor

Why don't we just use stdenv.appleSdkVersion?

This comment has been minimized.

@LnL7

LnL7 Jan 14, 2020
Author Member

The attribute is macOS specific, without the condition evaluating darwin only packages on linux fails on the assertion instead of meta.platforms with a nice message.

This comment has been minimized.

@tobim

tobim Jan 19, 2020
Contributor

What I was trying to ask is: if version must be equal to stdenv.appleSdkVersion, then why is one of them not initialized from the other, as in version = stdenv.appleSdkVersion or "invalid platform";?

@LnL7
Copy link
Member Author

@LnL7 LnL7 commented Jan 14, 2020

Indeed, but it's explicitly disabled for some reason.

@matthewbauer Why did you disabled the deployment target for cmake 8beb809? We do want binaries to have a minimum version.

@tobim
Copy link
Contributor

@tobim tobim commented Jan 14, 2020

libuv fails on 1 test (was on 1eed714):

not ok 42 - fs_event_error_reporting
# exit code 11
# Output from process `fs_event_error_reporting`:
# 2020-01-14 12:38 run-tests[62081] (FSEvents.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
# we never want to use the global macOS SDK
cmakeFlags="-DCMAKE_OSX_SYSROOT= $cmakeFlags"

# disable OSX deployment target
# we don't want our binaries to have a "minimum" OSX version
cmakeFlags="-DCMAKE_OSX_DEPLOYMENT_TARGET= $cmakeFlags"

This comment has been minimized.

@matthewbauer

matthewbauer Jan 14, 2020
Member

I'm surprised this isn't needed? How does CMake choose this value when it's unset?

This comment has been minimized.

@LnL7

LnL7 Jan 14, 2020
Author Member

If not set explicitly the value is initialized by the MACOSX_DEPLOYMENT_TARGET environment variable, if set, and otherwise computed based on the host platform.

So if I understand this correctly this was equivalent to unsetting MACOSX_DEPLOYMENT_TARGET from the stdenv.

# Ensure consistent LC_VERSION_MIN_MACOSX and remove LC_UUID.
export MACOSX_DEPLOYMENT_TARGET=${macosVersionMin}
export NIX_LDFLAGS+=" -macosx_version_min ${macosVersionMin} -sdk_version ${appleSdkVersion} -no_uuid"

This comment has been minimized.

@matthewbauer

matthewbauer Jan 14, 2020
Member

I'm not sure how the linker works when you pass multiple values here. For instance, is -macosx_version_min 10.12 -macosx_version_min 10.14 a valid argument list?

This comment has been minimized.

@LnL7

LnL7 Jan 14, 2020
Author Member

MACOSX_DEPLOYMENT_TARGET and -macos_version_min are closely related but not the same. AFAIK the variable is used by build systems and headers for version specific pragmas while the linker flag just configures the load command.

Alltho since it looks like the compiler always adds the flag based on the variable, overriding this original one, it's not really necessary. Added it just in case but should be fine to leave out if you're concerned about it.

# Workaround for https://openradar.appspot.com/22671534 on 10.11.
export gl_cv_func_getcwd_abort_bug=no
stripAllFlags=" " # the Darwin "strip" command doesn't know "-s"

This comment has been minimized.

@matthewbauer

matthewbauer Jan 14, 2020
Member

Making this a space is kind of hacky, but I think it is because of the - in ${stripAllFlags:--s} in setup.sh?

This comment has been minimized.

@LnL7

LnL7 Jan 14, 2020
Author Member

Yeah, but doesn't that do the opposite of what we want here?

This comment has been minimized.

@matthewbauer

matthewbauer Jan 14, 2020
Member

Docs say that :- means:

If parameter is unset or null, the expansion of word is substituted. Otherwise, the value of parameter is substituted.

So yeah it sounds like the space is needed

@FRidh FRidh added this to Needs review in Staging Jan 15, 2020
@LnL7
Copy link
Member Author

@LnL7 LnL7 commented Jan 15, 2020

@tobim libuv builds fine here

@LnL7
Copy link
Member Author

@LnL7 LnL7 commented Jan 15, 2020

Looks like anything that links against impure system frameworks still isn't reproducable, but that's somewhat to be expected.

-/nix/store/1nkad14ypnqhlwn4w4rs287679y09ly3-libuv-1.34.0/lib/libuv.1.dylib:
+/nix/store/hywq7d7bmql8a1w0m28y59pim76dldav-libuv-1.34.0/lib/libuv.1.dylib:
 Load command 0
       cmd LC_SEGMENT_64
   cmdsize 552
@@ -260,7 +260,7 @@
       cmdsize 104
          name /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (offset 24)
    time stamp 2 Thu Jan  1 01:00:02 1970
-      current version 1575.23.0
+      current version 1349.8.0
 compatibility version 150.0.0
 Load command 11
           cmd LC_RPATH

I've thought before about doing the same as Libsystem for all impure libraries we interact with, also solving some sandboxing problems, but that would be quite hard to maintain.

@LnL7
Copy link
Member Author

@LnL7 LnL7 commented Jan 31, 2020

Unless anybody has concerns I'm going to merge this soon.

@tobim
Copy link
Contributor

@tobim tobim commented Jan 31, 2020

@tobim libuv builds fine here

The build failure on my machine is gone now too.

Fixes #21629

Passing these extra linker flags removes both the semi-random uuid
included in most binaries as well as making the sdk version consistent
instead of based on the current os version.

    Load command 8
         cmd LC_UUID
     cmdsize 24
        uuid 70FAF921-5DC8-371C-B814-4F121FADFDF4

    Load command 9
          cmd LC_VERSION_MIN_MACOSX
      cmdsize 16
      version 10.12
          sdk 10.13

The -macosx_version_min flag isn't strictly necessary since that's
already handled by MACOSX_DEPLOYMENT_TARGET.
LnL7 added 4 commits Jan 13, 2020
We _do_ want minimum versions in our packages.
This was initially introduced in 92188d9,
not clear how relevant this still is but i686 isn't supported anymore so
disable it explicitly.
@LnL7 LnL7 force-pushed the LnL7:darwin-macos-version-min branch from ddc17f6 to 68513e4 Jan 31, 2020
@LnL7 LnL7 requested review from FRidh, jonringer and ttuegel as code owners Jan 31, 2020
@LnL7 LnL7 changed the base branch from master to staging Jan 31, 2020
@LnL7 LnL7 merged commit 6f38c93 into NixOS:staging Jan 31, 2020
15 of 16 checks passed
15 of 16 checks passed
cmake, stdenv on x86_64-darwin
Details
Evaluation Performance Report Evaluator Performance Report
Details
cmake, stdenv on aarch64-linux Success
Details
cmake, stdenv on x86_64-linux Success
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
Staging automation moved this from Needs review to Done Jan 31, 2020
@LnL7 LnL7 deleted the LnL7:darwin-macos-version-min branch Jan 31, 2020
@marsam marsam mentioned this pull request Feb 18, 2020
@eraserhd eraserhd mentioned this pull request Feb 20, 2020
6 of 10 tasks complete
@primeos
Copy link
Member

@primeos primeos commented Aug 8, 2020

FYI: Due to regressions in CMake 3.18, CMAKE_OSX_ARCHITECTURES=x86_64 now causes problems for Linux builds. I've opened #94952 to see if it makes sense to set this only on Darwin, but I'm not familiar with the details and reasons here (maybe we want a unified set of flags for all platforms?). Feel free to voice your opinion in #94952 if you have time for a quick look. Thanks :)

@LnL7 LnL7 moved this from Stdenv to Done in Darwin Oct 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Darwin
  
Done
Staging
  
Done
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.