Installation issues

[tgunnoe]

Thank's for your work here. I've finally started my attempt to get nixos onto the PBP this last weekend, but have been encountering issues that made this a week process for me.

I would rank my experience at midlevel when it comes to nixos, having used it exclusively since last year, but never on any non-x86 devices. Here's the issues I was encountering.

1. There's a lot of confusion to me when it comes to the boot process with boot order selection. The Image Build section of the readme here readies nixOS on your SD card (build on an x86 desktop), but it most likely wont boot unless the eMMC is disabled or disconnected. I tried updating the eMMC with the PBP official Manjaro with hopes that it contains the right boot order on uboot to try the SD card first, to no avail. Before that I tried a few things with flashing the SPI-NOR with some fixes from the PBP forums with no progress on getting the boot order to check the SD first. The lack of good instructions on the PBP wiki/forums is why I believe I couldn't get this to work. So to fix it, i just flashed the NixOS image from this repo directly onto the eMMC and it worked.

2. I cloned this repo and imported it into configuration.nix as stated in the readme. I had to disable a lot of things in configuration.nix, like desktops/sway/nonessential program in order to get my core functioning utilities and I had to nixos-rebuild boot instead of a typical switch to get an updated generation. Now it seems I'm having problems compiling the kernel:

• first of all, I had to add swap space (via USB) or the build would run out of memory pretty soon.

• I continually get this warning: You are using an unsupported version of LLVM! Currently only 6.0 is supported. We will try though.... I'm not sure what process is using LLVM to compile

• following that, the PBP kept shutting down at some point in the process. I assume it was overheating so i set jobs as such -j1, set a fan on it, and this increased the process to almost 24 hours, which still resulted in a failed build at the end. And it seems the build can't resume where it left off? This was about the 4th attempt of a build.

• So I can't compile this kernel on my much faster x86 desktop? I dont completely understand the state of cross compiling with nix. I hear that in nixpkgs it works well but for NixOS it does not. I am wondering if there is a better way to do this than this day process of compiling which ends with failure every time.

I'm mostly looking for any suggestions on what I could do better but wanted to share my experiences with the install process in general.

Thanks!

[akirakyle]

The kernel should compile within the PBP's 4GB of memory just fine. It takes ~2hrs for me. LLVM isn't part of that process so it sounds like something else is trying to compile. I would suggest figuring out what that is first and not build that. I've recently seen a similar warning when trying to install something that depends on the GHC on npkgs-unstable, however I would suggest using the nixos-20.03 channel as you probably won't have as many issues getting binary substitutes (though not a certainty - see NixOS/nixpkgs#83049).

The PBP is likely shutting down due to running out of battery, not overheating. Unfortunately the power supply cannot supply enough to run the PBP with the screen on and all cores maxed out. Turing the screen backlight down helps, along with using fewer build cores. The nature of nix being a functional package manager and its transactional nature means that it cannot resume a failed build, it must start again to have any hope of maintaining a reproducible guarantee.

[samueldr]

Hi,

I'll try answering some of the concerns as best as I can.

Boot order

The boot ordering is the usual rk3399 one. That, alone isn't helpful, but is a good first nugget of information to find relevant data.

There are two boot orders, the CPU boot order, and u-boot's boot order.

The CPU wants to start a program. It will look in a couple locations to find a program. First in SPI, then on the eMMC, finally on the SD card. It will start the first valid program it finds.

This program, in our case, is u-boot. This, in turn, can do anything, it's not limited by the CPU boot order. The defaults in upstream u-boot is to look on the storage that is the eMMC first, and then on SD card.

On that storage it will look for different things, but will not boot another u-boot; it will most likely be looking for a partition with either a u-boot boot script or an extlinux.conf file.

This may explain some of the issues booting USB, you are fighting two distinct pieces that both could want to do conflicting actions!

I believe most distros made for the Pinebook Pro end up tweaking u-boot's boot order for that "alternative boot order"; it is alternative compared to u-boot's defaults.

There's not much that this repo or NixOS can do about all that. It mostly depends on what is already on the machine. Though as you have done, wiping the eMMC is a safe bet.

---

System configuration

Hm, not sure what to say. It's likely to be aarch64 issues, as it's less well-tested than x86_64 on a desktop-like setting.

Regarding swap space, and your installed system, it's not the place for this setup to make opinionated decisions. Everyone's needs are different.

---

Troubles building

Regarding over-heating, it's entirely possible! The SoC can get toasty when pushed. It can even start drawing power from the battery even when plugged, it's not sipping power when it's going full hog. It has been reported by others on non-nixos that it can get in a thermal shutdown situation. Though I personally haven't seen that. Though note that I'm not building big builds on the machine.

In that situation you could look into disabling one of the A72 core so it doesn't end up used during long spans of "high stress".

You could be cross-compiling the kernel on the desktop machine, but due to the way Nix works, the compiler is an input, and you get two different compilers, so a different output, so you would have to do some trickery to make your config use the cross-compiled kernel on a native rebuild. (I haven't done so, so I can't be of much help.)

I don't know what's up with your "LLVM-using" build. That's probably not PBP related, but rather an aarch64 thing.

[tgunnoe]

I've recently seen a similar warning when trying to install something that depends on the GHC on npkgs-unstable

Yes! this is exactly what it is. I'm on unstable The path of build failing that I just discovered is GHC -> cachix -> system-path -> nixos-system. I assumed it was the kernel because I couldn't imagine something else taking that long, but it looks like I'm good on the kernel:

$ uname -a
Linux chapterhouse 5.6.0 #1 NixOS SMP Thu Jan 1 00:00:01 UTC 1970 aarch64 GNU/Linux

The culprit seems to be my use of cachix in order to use this repository (all wayland/sway related): https://github.com/colemickens/nixpkgs-wayland

2. I had to disable a lot of things in configuration.nix, like desktops/sway/nonessential program in order to get my core functioning utilities

Here I did get a new generation, which included the kernel I am assuming.

however I would suggest using the nixos-20.03

I'm going to have to make a choice then, because I'd like to user the nixpkgs-wayland overlay as I do on my other machines which is built against unstable, but there's likely more of those packages moved to the latest stable, or I could pick a commit to match up. I am completely unfamiliar with GHC and why it would not be building.

The PBP is likely shutting down due to running out of battery, not overheating.

I dont think so; I charge over USB-C but will check in any case.

The nature of nix being a functional package manager and its transactional nature means that it cannot resume a failed build, it must start again to have any hope of maintaining a reproducible guarantee.

Thank you, I've never considered this!

Boot order

Thanks for that thorough explanation! I was semi-aware of the chain what manages boot order (CPU - SPI, eMMC then SD etc, but I could find no information on the public images built prebuilt for the PBP on what their uboot boot order is. For example the manjaro image installed on eMMC didn't allow me to boot the NixOS image build from your repository. It's most helpful to know that the CPU will hit uboot first in its device order. Ideally I imagine uboot on SPI with a boot selection (using grub on spi like libreboot?), but I understand that's a work in progress.

Thanks for the suggestions, I now know that I need to figure out several things, including how to better adapt the sway/wayland parts to the PBP.

Regarding over-heating, it's entirely possible! The SoC can get toasty when pushed. It can even start drawing power from the battery even when plugged

I'm going to enable some thermal monitoring controls before I try to make a new build of something (maybe hardware.fancontrol.config or hardware.sensor.iio.enable?

[theotherjimmy]

Hi @tgunnoe. I use sway on my pbp without the nixpkgs-wayland overlay without any issues. That being said, I'm using a more recent version of mesa than is provided by any version in nixpkgs by extending the overlay in my copy of wip-pinebook-pro.

I charge over USB-C but will check in any case.

Note that the barrel jack and USB-C are both limited to providing 15W, which is slightly less power than the system consumes with max brightness and 100% cpu at maximum clocks. Further, if the battery gets too warm, the battery controller will stop charging it (which is indicated by a blinking red led on the side of the pbp instead of the normal solid one).

OTOH, the pbp has only 4GB of RAM, and many compilers can easily chew through this in no time (especially Rustc or GHC). I have found that enabling zram has mitigated this for me, and I also recommend enabling earlyoom to make sure that you're pbp does not lock up and instead kills a process that's consuming a large about of memory.

[jeregrine]

I was able to get this working over the weekend with the latest pinebook pro running manjaro. The default build.sh didn't work. Something related to nix-build not being found when running it via exec. I needed to just run it directly and set the env vars before.

I also needed to build on 20.03 instead of 19 and yea I needed to turn the brightness down.

[tgunnoe]

I have it working with channel 20.03 now, and my sway setup is not too affected.

Anyone else experiencing battery charging issues? Its happened several times now, but I noticed it this morning after opening the lid again. The display was off, and I suppose there's still a suspend issue and it wasn't suspended, but the power drained overnight by about 30 percent and no charging light after plugging in. I have to restart it to resume charging.

Further, if the battery gets too warm, the battery controller will stop charging it (which is indicated by a blinking red led on the side of the pbp instead of the normal solid one).

Could be this, but I don't have a blinking light and it doesnt seem hot.

So how to go about getting an aarch64-friendly nixpkgs? Does one exist? I know many programs that can definitely compile to aarch64 but almost none are available through the normal nixpkgs channel, most being derivations for x86-based.

[theotherjimmy]

So how to go about getting an aarch64-friendly nixpkgs? Does one exist?

That's regular nixpkgs, so nope.

almost none are available through the normal nixpkgs channel

For a while there, there was an issue that was blocking building of something like 1/2 of the aarch64 builds in hydra. I think that may have been resolved recently though. It's worth looking into the hydra jobs to see what's going wrong.

[theotherjimmy]

As far as battery issues go, there are some pretty severe limitations to the suspend support right now (sorry). You can't suspend if either:

• You use NVMe (not even s2idle works here :/)
• You use mainline TF-A and/or u-boot

OTOH, this overlay should configure s2idle as the default suspend target, to prevent suspend from hanging your PBP. In my testing s2idle drains 1/4 to 1/3 of the battery in 10ish hours, matching what you have seen.

Not sure about the charging problem though. I get issues when I'm debugging suspend, but that's a bit of a special case.

[tgunnoe]

I'm starting to get the hang of it.

I'm bringing this repository into my setup like so https://github.com/tgunnoe/configurations/blob/master/machines/pinebook-pro/configuration.nix#L14

That's regular nixpkgs, so nope.

If the requirements for a program are built on the system successfully (such as cmake/mesa/pulseaudio/sdl etc) maybe it's just the derivations that list their platforms in a non-agnostic way? (that is, x86-64)

As far as battery issues go, there are some pretty severe limitations to the suspend support right now (sorry). You can't suspend if either:

* You use NVMe (not even s2idle works here :/)

* You use mainline TF-A and/or u-boot

So is this a problem from supporting it in nix, or does mainline manjaro pinebook pro have these suspend issues as well?

Not sure about the charging problem though. I get issues when I'm debugging suspend, but that's a bit of a special case.

I've found out that I have these shutdown and refuse to charge issues when I try to charge it from the USB-C port. the normal barrel jack still charges regardless, as far as I can tell

[tgunnoe]

Do you all have issues with the audio? I haven't gotten any audio out of it yet. It shows the card and shows that it's indeed providing sound from the applications though pulseaudio, but nothing is heard.

[samueldr]

I would rather you open a new issue, instead of adding on top of a mostly unrelated issue. (This is not an installation issue.)

I have opened #12 for that.