NixOS on RISC-V #101651
Comments
veprbl
added
the
6.topic: cross-compilation
|
My risc-v board crapped out and I haven't been keeping this up to date, sorry. At the time though I was able to run full cross-compiled NixOS in qemu, not too far from native compliation if I was patient enough. |
|
I managed to cross-compile Nix itself for riscv64: with (import <nixpkgs> {
overlays = [
(self: super: {
coreutils = super.coreutils.overrideAttrs (old: {
# https://github.com/gentoo/gentoo/raw/1070aa3245b38881a050507e365a2c9c60c23534/sys-apps/coreutils/files/coreutils-8.32-ls-restore-8.31-behavior.patch
patches = old.patches ++ [ ./coreutils-8.32-ls-restore-8.31-behavior.patch ];
});
})
];
}).pkgsCross.riscv64;
nix.override {
boost = boost17x;
}Run under QEMU user mode, I would expect most things to behave as if we were compiling natively on RISC-V: However, when building NixOS, I received the same error as @malte-v. Digging deeper reveals it's actually from Edit: I'm mistaken about what's going on. I'll try some more stuff before posting anything.We can quickly check by adding trace prints to Building NixOS then gives me: And indeed, building (My Nixpkgs is pinned at Well, Any idea on how to fix this cleanly? Changing null to some dummy value in |
|
Ok, I think I misunderstood. Of course building natively is not possible without the bootstrap binaries. When I attempted to build the bootstrap tools myself from x86_64, I got: Looking at the store paths, indeed I see libraries in the Those strings are coming from the Unfortunately Afterwards, during the tests I encountered two problems:
Then I successfully cross-compiled My (very ugly) changes so far, which i have pushed to my fork:
Edit: Remove part about prebuilt bootstrap-tools as now I know the reason for the |
|
(Deleted my last post as I thought I had some progress, but I don't) Ok, I'm moving closer to native compilation, but encountered this error when building xz: Expand for full logs with NIX_DEBUG=1This looks very similar to #79793 (comment), but the cause is different. If you expand the full logs you will see that From Expand for config.log with NIX_DEBUG=1The dependency looks like: I don't know what the correct way to solve this is, as a toolchain with libatomic is only built in the later stages. But we don't actually need Would like to hear your opinions. |
|
@zhaofengli Fyi the gcc reference cycles should be fixed by #106954 |
|
Any progress on getting this to work? I'm currently in the same boat as the original author of this issue with the same |
|
Are there more RISC-V specific issues that need to be resolved? The cross toolchain is working now: Beyond that, NixOS cross-compilation in general is pretty unstable, and will probably continue to be as long as there are no channel blockers for it. |
|
I think the problem is rather that there aren't a lot of maintainers who need/want cross-compilation. Making the channels block on cross-compilation issues would only cause headaches I think. Some sort of health indicator other than "look at hydra builds" might be helpful but that'd probably also just be red all the time. |
|
Possibly relevant #105294 (comment) |
|
I've made a bit of progress on my end, but I am still running into tons of issues. My goal is to build a minimal U-Boot compatible root filesystem image for RISC-V. Note that full native compilation is not one of my goals. Here's what I have run into so far:
Here is the configuration.nix I am using, and I am building it using the command |
|
Hmm, I've run into issues with numactl too. I'll open a PR later to enable it on powerpc64 and riscv since it seems to be supported on both of those platforms. I don't know what to do about kexec. That looks like a tough one. I'll look into pipewire though |
|
Fyi #96336 (comment) |
Yeah I don't understand how @zhaofengli was able to cross compile nix with boost. |
For Nix I bumped the boost dependency to boost17x (see my comment). I haven't touched this thing for a while, but you can try my old tree. Hopefully we can get somewhere further before I receive the HiFive Unmatched board later next month. |
|
@Rytone it's not clear to me from your configuration: how are you controlling the uboot configuration settings when cross compiling for a riscv target? Suppose I wanted to build just uboot targeting riscv with a custom Relatedly, any idea how to build just opensbi for riscv with nix? (I haven't looked deeply into this; I'm about to do a deep dive). |
|
@DieracDelta To be completely honest, I'm not totally sure what I'm doing makes any sense, so take this with a grain of salt: What I [think I] am doing right now is building a NixOS image that will be detected by U-Boot where I have built an OpenSBI+U-Boot firmware image following these and these instructions, not at all integrated into the NixOS build. Ultimately, I want to be able to run something like this (I think): Also, related to picking this up again, I am now running into the issues in #116207 when trying to do a fully bootstrapped build, and when just cross-compiling, I run into other random errors depending on what NixOS revision I use. |
|
That's awesome. @Rytone could you elaborate more on what you mean by fully bootstrapped? I'm able to just spin up a nix shell with From that nix-shell, I can build opensbi and u-boot targeting riscv. I'd like to make derivations for opensbi and u-boot that allows for extra targets to be added as well as |
|
Yeah, not sure if bootstrapped is the right term, but its what I went with 😅. By "fully bootstrapped" I meant what I have in this configuration where the majority of the RISC-V packages are built under RISC-V virtualization rather than being cross-compiled. The alternative is to import a fresh copy of nixpkgs in the call to With regards to building OpenSBI and U-Boot, I built them like you did just using a shell, though I didn't try using clang at all. I agree that in the long run being able to build them as a Nix derivation would be nicer. |
|
Quick update: Just realized that while my cross-compilation technique is able to build all of the derivations necessary for a RISC-V NixOS installation, it is unable to completely build the image itself as it tries to run some final setup of the RISC-V image under an x86 VM. I looked and it doesn't seem the VM infrastructure is suited for this sort of compilation... I tried hacking it to spin up a RISC-V VM instead but it got stuck booting. Going forward I would say abandon that idea of cross-compiling everything and instead focus on getting the proper "Nix way" to work. |
|
I got confused with my paths. It turns out I am having trouble getting a nix shell with a gcc cross compiler toolchain with x8664 as my host and riscv64 as my target. It seems to be more involved than just defining a cross system like this: riscvPkgs = import nixpkgs {
localSystem = "${system}";
crossSystem = {
config = "riscv64-unknown-linux-gnu";
abi = "lp64d";
};
}; And then passing in |
|
https://github.com/oxalica/rust-overlay/blob/0bb9ef6d8b34e5579d7384201f3106a49ce3deca/examples/cross-aarch64/shell.nix fixed my issue. TLDR is to put For reference, here's a more complete picture of what I'm doing: let
riscvPkgs = import nixpkgs {
localSystem = "${system}";
crossSystem = {
config = "riscv64-unknown-linux-gnu";
abi = "lp64d";
};
};
in pkgs.mkShell {
nativeBuildInputs = [ riscvPkgs.buildPackages.gcc riscvPkgs.buildPackages.clang ];
};This gets me both the clang and gcc compilers, both prefixed with Relatedly, any ideas where the gnu-toolchain source is being passed in? I've got a fork that's somewhat diverged from the master branch that I'd like to build. I was hoping to override the src attribute of its derivation, but I'm not finding any derivation when grepping Nixpkgs... |
|
Firefox is now running on native RISC-V NixOS. For this I cheated a little bit by using the X86 version of Node.js during build since a native build of node hangs on node_mksnapshot (more investigation needed). It's only used during the build process and does not affect the results.
If you want to try out Firefox and SuperTuxKart on RISC-V NixOS yourself, I put together a new graphical image. With VirtIO-GPU + VirGL (use the included |
|
I was planning to get one of the StarFive VisionFive RISC-V boards, I'm surprised there's already been an attempt at this. |
|
Got it ordered, should be here in about a week. |
|
Could you try |
|
Sure thing. It cross-compiles well, so I'll test it as soon as I get a working test environment on it when it arrives. |
|
Here's some preliminary footwork: #168826 |
|
|
We're now in nixos-hardware at https://github.com/NixOS/nixos-hardware/tree/master/starfive/visionfive/v1. |
|
Just got my VisionFive 2, where do I start? Is there a Matrix room specifically for RISC-V? |
|
I got mine today as well. |
|
Matrix Room: Exotic Nix Targets is used for RISC-V. |
|
Homemade (without using the starfive proprietary tools) firmware for visionfive2: https://hydra.nichi.co/job/nixos/riscv/firmware-vf2/latest/ |
|
Can anyone interested look at #221306 to make gn compile on RISC-V? |
|
Star64 nixos-hardware pr: NixOS/nixos-hardware#662 And here's an image I built directly on the board: https://app.cachix.org/api/v1/cache/fgaz/serve/hj72i7znp0rvc4dxs2xslw6b3ik3wifb/sd-image/nixos-sd-image-23.11pre-git-riscv64-linux-pine64-star64.img.zst (8GB RAM, user/pw: nixos) |
|
@fgaz Both merged now |
|
@nyabinary yes I have working images here: https://sr.ht/~fgaz/nixos-star64 |
|
And we have a new matrix room at #riscv:nixos.org |
|
Is there any reason to keep this issue open still? I fear it's going to become an unactionable grab bag of comments and unrelated issues. AFAIK, NixOS kinda just works, as much as any other similarly-tiered platforms at the moment, no? |
|
Correct. There's no further reason to keep an issue much more than "wow, look how NixOS came to be on RISC-V". We can move on to more actionable fine grained issues when it pertains to RISC-V support in NixOS. |
Issue description
Judging from the screenshot in #36187, @shlevy has already been successful in running some form of NixOS on RISC-V. I'd just like to know what the status of this is, as naively trying to build an ISO installer failed:
There seems to be some issue with Perl. I guess 306d5cd addressed this, but now it's broken again. I'm still pretty much a biginner to Nix(OS) and unfortunately have no idea how to fix it :( Maybe we could get NixOS on RISC-V running again? I don't know how much work it would be though, since #36187 got merged 2.5 years ago and a lot has happened since then.
The text was updated successfully, but these errors were encountered: