lib/systems: add UEFI systems for i686, aarch64, x86_64 as examples #226145

RaitoBezarius · 2023-04-14T12:33:13Z

Latest attempt at #228374

Description of changes

It also adds the metadata for Rust.

So @alyssais suggested me the next step should be to create a UEFI stdenv.
I'm not very knowledgeable on adding such stdenvs in nixpkgs, should this be part of pkgs/stdenv ? or pkgs/build-support/uefi ?

It's uncertain to me.

Thanks to @rrbutani help, we were able to get Lanzaboote's stub to build!

Steps to make it acceptable:

Acceptable hack for UEFI systems
Acceptable solution for NIX_ENFORCE_PURITY
Acceptable solution for cc-wrapper at linkage time
Acceptable solution to order -z relro -z ... properly and re-enable hardening
Acceptable solution for cargo-auditable
Acceptable solution for unsupported hardening flags

Things done

lib/systems/examples.nix

alyssais · 2023-04-14T21:17:57Z

Since we're not sure yet what the best triple to use is going to be (since GNU is unlikely to ever support UEFI in the standard toolchain, we might want to use the LLVM ones here), I don't think it's useful to add the examples until we can actually prove we can build stuff on them. The examples should come at the end of the bringup process.

for GCC: https://wiki.osdev.org/UEFI_App_Bare_Bones `nix-build --expr "(import ./. { crossSystem = (import ./lib).systems.examples.i686-uefi; }).stdenv"` `nix-build --expr "(import ./. { crossSystem = (import ./lib).systems.examples.i686-uefi; }).buildPackages.rustc"` `nix-build --expr "(import ./. { crossSystem = (import ./lib).systems.examples.x86_64-uefi; }).lanzaboote"` `rustc` wants a linker that supports MSVC link flags; the `cc-wrapper` wrapped clang does not (though I think the `lld` underneath would be fine)

isEfi does not really make sense as a name when nixpkgs can target (U)EFI environments. Therefore, we rename it `hasUefi`: does the platform has a known UEFI firmware/environment? `isUefi` should be used to say: is this platform an UEFI environment?

RaitoBezarius · 2023-04-15T15:43:15Z

Since we're not sure yet what the best triple to use is going to be (since GNU is unlikely to ever support UEFI in the standard toolchain, we might want to use the LLVM ones here), I don't think it's useful to add the examples until we can actually prove we can build stuff on them. The examples should come at the end of the bringup process.

I transformed this PR into something that will introduce Lanzaboote's stub as an example.

flokli · 2023-04-15T15:57:32Z

pkgs/tools/misc/lanzaboote/default.nix

@@ -0,0 +1,31 @@
+{ rustPlatform, stdenv, fetchFromGitHub, lib, breakpointHook }: rustPlatform.buildRustPackage {


breakpointHook

pkgs/tools/misc/lanzaboote/default.nix

SuperSandro2000 · 2023-04-15T17:44:29Z

lib/systems/examples.nix

+
+  # See: https://github.com/rust-lang/rust/pull/56769/files#r241916113
+  i686-uefi = {
+    # config = "i686-unknown-windows-gnu";


What's with those comments?

Please read #226291 first :P

SuperSandro2000 · 2023-04-15T17:45:02Z

lib/systems/inspect.nix

@@ -6,6 +6,8 @@ with lib.lists;
 let abis_ = abis; in
 let abis = lib.mapAttrs (_: abi: builtins.removeAttrs abi [ "assertions" ]) abis_; in

+# TODO: isUefi
+


Can you write at least a half sentence otherwise we forget what we wanted to do here

It was supposed to add a isUefi function here so we can test for it

SuperSandro2000 · 2023-04-15T17:46:36Z

pkgs/tools/misc/lanzaboote/default.nix

+
+  cargoHash = "sha256-FlnheCgowYsEHcFMn6k8ESxDuggbO4tNdQlOjUIj7oE=";
+
+  # TODO: limit supported platforms to UEFI


Missing the full meta and meta should be last.

Yes, this is more a test package PoC to show the "value" of those changes rather than the real package yet

RaitoBezarius · 2023-04-15T17:53:00Z

Converting to draft to signal WIP state.

RaitoBezarius · 2023-04-25T12:56:10Z

pkgs/development/compilers/llvm/15/default.nix

-    clangUseLLVM = wrapCCWith rec {
+    clangUseLLVM =
+    if stdenv.targetPlatform.libc == null then
+      builtins.trace "yo" tools.clangNoLibc # TODO: gate correctly...


@rrbutani what did you mean here for the gating? It seems like it's correct to me?

I think I wasn't sure whether we wanted to check for libc == null here or specifically exempt UEFI targets like we do in all-packages.nix (which will complain if it sees libc == null for a target it doesn't have special handling for):
https://github.com/NixOS/nixpkgs/blob/132b7981ef4ea88d9810ff286c1bee06bb8bf022/pkgs/top-level/all-packages.nix#L20381-L20404

But I think you're right and that this is fine.

rrbutani · 2023-04-25T16:19:32Z

pkgs/development/compilers/llvm/15/default.nix

      cc = tools.clang-unwrapped;
      libcxx = targetLlvmLibraries.libcxx;
      bintools = bintools';
      extraPackages = [
        libcxx.cxxabi
        targetLlvmLibraries.compiler-rt
-      ] ++ lib.optionals (!stdenv.targetPlatform.isWasm) [
+      ] ++ lib.optionals (!stdenv.targetPlatform.isWasm && !stdenv.targetPlatform.isUefi) [


If we're gating the wrapped CC with libc/libcxx/compiler-rt/libunwind on having libc != null, I think this isn't needed anymore for the UEFI targets?

rrbutani · 2023-04-25T16:20:27Z

pkgs/development/compilers/llvm/15/default.nix

@@ -224,11 +227,12 @@ in let
          "-Wno-unused-command-line-argument"
          "-B${targetLlvmLibraries.compiler-rt}/lib"
        ]
-        ++ lib.optional (!stdenv.targetPlatform.isWasm) "--unwindlib=libunwind"
+        ++ lib.optional (!stdenv.targetPlatform.isWasm && !stdenv.targetPlatform.isUefi) "--unwindlib=libunwind"


ditto; is this needed?

rrbutani · 2023-04-25T16:20:35Z

pkgs/development/compilers/llvm/15/default.nix

        ++ lib.optional
          (!stdenv.targetPlatform.isWasm && stdenv.targetPlatform.useLLVM or false)
          "-lunwind"
        ++ lib.optional stdenv.targetPlatform.isWasm "-fno-exceptions";
+      dontWrap = stdenv.targetPlatform.isUefi;


ditto; is this needed?

ghost · 2023-04-27T09:33:25Z

lib/systems/default.nix

@@ -50,7 +50,7 @@ rec {
        else if final.isFreeBSD             then "fblibc"
        else if final.isNetBSD              then "nblibc"
        else if final.isAvr                 then "avrlibc"
-        else if final.isNone                then "newlib"
+        else if final.isUefi                then "none" # TODO: execute can actually be QEMU?


I don't understand what this TODO is saying.

Can we emulate UEFI programs on QEMU? I guess we could have a remote builder for UEFI if we had baremetal compilers in UEFI, etc. :D ?

Can we emulate UEFI programs on QEMU?

Probably. I just can't figure out what this has to do with the choice of libc.

ghost

Please break this up into a minimum of three separate PRs: lib/systems, cc-wrapper, and everything else.

The lib/systems part needs some changes (see below) but is basically feasible. I have very serious concerns about the cc-wrapper approach.

ghost · 2023-04-27T10:06:55Z

lib/systems/examples.nix

+  # See: https://github.com/rust-lang/rust/pull/56769/files#r241916113
+  i686-uefi = {
+    # config = "i686-unknown-windows-gnu";
+    config = "i686-pc-win32-coff";


Suggested change

config = "i686-pc-win32-coff";

config = "i686-w64-mingw32";

This is the config.sub triple; see #226291 (comment)

ghost · 2023-04-27T10:07:12Z

lib/systems/examples.nix

+
+  x86_64-uefi = {
+    # config = "x86_64-unknown-windows-msvc";
+    config = "x86_64-pc-win32-coff";


Suggested change

config = "x86_64-pc-win32-coff";

config = "x86_64-w64-mingw32";

This is the config.sub triple; see #226291 (comment)

ghost · 2023-04-27T10:07:40Z

lib/systems/examples.nix

+
+  aarch64-uefi = {
+    # config = "aarch64-unknown-windows-msvc";
+    config = "aarch64-pc-win32-coff";


Suggested change

config = "aarch64-pc-win32-coff";

config = "aarch64-w64-mingw32";

This is the config.sub triple; see #226291 (comment)

ghost · 2023-04-27T10:09:04Z

lib/systems/inspect.nix

@@ -82,7 +82,10 @@ rec {
    isMusl         = with abis; map (a: { abi = a; }) [ musl musleabi musleabihf muslabin32 muslabi64 ];
    isUClibc       = with abis; map (a: { abi = a; }) [ uclibc uclibceabi uclibceabihf ];

-    isEfi = [
+    isUefi         = { /* kernel = kernels.windows; replace with none? */ abi = abis.coff; vendor = vendors.pc; }; # this may not be restrictive enough? TODO


Suggested change

isUefi = { /* kernel = kernels.windows; replace with none? */ abi = abis.coff; vendor = vendors.pc; }; # this may not be restrictive enough? TODO

isUefi = { abi = abis.uefi; kernel = kernels.none; };

The most distinctive thing about UEFI, from a "linker's point of view" is that it has its own calling convention (ABI). Indeed, when Linux prostrates itself to venerate the firmware, it assumes the EfiApi calling convention.

ghost · 2023-04-27T10:10:01Z

lib/systems/parse.nix

+    # Not really an ABI, rather an object format.
+    coff         = {
+      assertions = [
+        { assertion = platform: platform.isWindows || platform.isUefi;
+          message = ''
+            The "COFF" object file format is for Windows or UEFI targets.
+          '';
+        }
+      ];
+    };


Suggested change

# Not really an ABI, rather an object format.

coff = {

assertions = [

{ assertion = platform: platform.isWindows || platform.isUefi;

message = ''

The "COFF" object file format is for Windows or UEFI targets.

'';

}

];

};

ghost · 2023-04-27T10:11:45Z

lib/systems/inspect.nix

+    isUefi         = { /* kernel = kernels.windows; replace with none? */ abi = abis.coff; vendor = vendors.pc; }; # this may not be restrictive enough? TODO
+
+    # Those are platforms for known UEFI firmwares
+    hasUefi = [


This doesn't belong in inspect.nix. The patterns are only for telling you about the platform inspected, not about what other platforms are associated with it. For example canExecute isn't in inspect.nix. hasUefi is in the same bucket as canExecute; it is a binary relation on platforms.

ghost · 2023-04-27T10:13:00Z

pkgs/build-support/cc-wrapper/default.nix

+# It is sometimes useful to expose naked binaries in the standard "wrapped" format.
+# For UEFI for example.
+, dontWrap ? false


This is totally gross.

👎

RaitoBezarius · 2023-04-28T15:55:30Z

Only for reference, superseded by #228374.

RaitoBezarius requested review from alyssais, Ericson2314 and matthewbauer as code owners April 14, 2023 12:33

RaitoBezarius mentioned this pull request Apr 14, 2023

project: move to nixpkgs Rust infrastructure nix-community/lanzaboote#147

Merged

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Apr 14, 2023

rrbutani reviewed Apr 14, 2023

View reviewed changes

lib/systems/examples.nix Outdated Show resolved Hide resolved

rrbutani reviewed Apr 14, 2023

View reviewed changes

lib/systems/examples.nix Outdated Show resolved Hide resolved

RaitoBezarius mentioned this pull request Apr 15, 2023

UEFI targets in nixpkgs infrastructure #226291

Open

RaitoBezarius force-pushed the uefi-systems branch from 5bec3d5 to 59a58e1 Compare April 15, 2023 15:29

RaitoBezarius requested review from zowoq, winterqt, figsoda, Mic92 and a team as code owners April 15, 2023 15:29

github-actions bot added 6.topic: python 6.topic: rust 6.topic: systemd labels Apr 15, 2023

rrbutani and others added 4 commits April 15, 2023 17:30

lib/systems: isEfi -> hasUefi

6ef7bab

isEfi does not really make sense as a name when nixpkgs can target (U)EFI environments. Therefore, we rename it `hasUefi`: does the platform has a known UEFI firmware/environment? `isUefi` should be used to say: is this platform an UEFI environment?

lanzaboote: init at 0.3.0

c29dfa7

fix

619a58b

RaitoBezarius force-pushed the uefi-systems branch from 59a58e1 to 619a58b Compare April 15, 2023 15:30

github-actions bot removed the 6.topic: python label Apr 15, 2023

ofborg bot added the 2.status: merge conflict label Apr 15, 2023

flokli requested changes Apr 15, 2023

View reviewed changes

SuperSandro2000 reviewed Apr 15, 2023

View reviewed changes

RaitoBezarius marked this pull request as draft April 15, 2023 17:52

RaitoBezarius added the significant Novel ideas, large API changes, notable refactorings, issues with RFC potential, etc. label Apr 19, 2023

RaitoBezarius mentioned this pull request Apr 21, 2023

Don't use crane for production nix-community/lanzaboote#98

Closed

RaitoBezarius commented Apr 25, 2023

View reviewed changes

RaitoBezarius added 3 commits April 25, 2023 16:03

stuff

de884df

fixup! stuff

f657286

cc-wrapper: introduce do not wrap

e779121

rrbutani reviewed Apr 25, 2023

View reviewed changes

alyssais mentioned this pull request Apr 26, 2023

UEFI cross: don't overthink it #228374

Merged

12 tasks

ghost reviewed Apr 27, 2023

View reviewed changes

RaitoBezarius closed this Apr 28, 2023

Mic92 deleted the uefi-systems branch April 28, 2023 19:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lib/systems: add UEFI systems for i686, aarch64, x86_64 as examples #226145

lib/systems: add UEFI systems for i686, aarch64, x86_64 as examples #226145

RaitoBezarius commented Apr 14, 2023 •

edited

Loading

alyssais commented Apr 14, 2023

RaitoBezarius commented Apr 15, 2023

flokli Apr 15, 2023

SuperSandro2000 Apr 15, 2023

RaitoBezarius Apr 15, 2023

SuperSandro2000 Apr 15, 2023

RaitoBezarius Apr 15, 2023

SuperSandro2000 Apr 15, 2023

RaitoBezarius Apr 15, 2023

RaitoBezarius commented Apr 15, 2023

RaitoBezarius Apr 25, 2023

rrbutani Apr 25, 2023

rrbutani Apr 25, 2023

rrbutani Apr 25, 2023

rrbutani Apr 25, 2023

ghost Apr 27, 2023

RaitoBezarius Apr 27, 2023

ghost Apr 27, 2023

ghost left a comment

ghost Apr 27, 2023

ghost Apr 27, 2023

ghost Apr 27, 2023

ghost Apr 27, 2023

ghost Apr 27, 2023

ghost Apr 27, 2023

ghost Apr 27, 2023

RaitoBezarius commented Apr 28, 2023

		@@ -0,0 +1,31 @@
		{ rustPlatform, stdenv, fetchFromGitHub, lib, breakpointHook }: rustPlatform.buildRustPackage {


		cargoHash = "sha256-FlnheCgowYsEHcFMn6k8ESxDuggbO4tNdQlOjUIj7oE=";

		# TODO: limit supported platforms to UEFI

	config = "x86_64-pc-win32-coff";
	config = "x86_64-w64-mingw32";

	config = "aarch64-pc-win32-coff";
	config = "aarch64-w64-mingw32";

	isUefi = { /* kernel = kernels.windows; replace with none? */ abi = abis.coff; vendor = vendors.pc; }; # this may not be restrictive enough? TODO
	isUefi = { abi = abis.uefi; kernel = kernels.none; };

lib/systems: add UEFI systems for i686, aarch64, x86_64 as examples #226145

lib/systems: add UEFI systems for i686, aarch64, x86_64 as examples #226145

Conversation

RaitoBezarius commented Apr 14, 2023 • edited Loading

Latest attempt at #228374

Description of changes

Things done

alyssais commented Apr 14, 2023

RaitoBezarius commented Apr 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RaitoBezarius commented Apr 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RaitoBezarius commented Apr 28, 2023

RaitoBezarius commented Apr 14, 2023 •

edited

Loading