Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0070] Merge nixos-hardware in to nixpkgs (#70) #70

Open
wants to merge 3 commits into
base: master
from

Conversation

@Mic92
Copy link

Mic92 commented Jun 20, 2020

We are currently having a separate repository to providing NixOS profiles for hardware specific extensions at nixos-hardware. This RFC proposes to merge this repository (4000 LOCs/350 commits) and add support for detecting hardware in nixos-generate-config.

Rendered

@Mic92 Mic92 changed the title [RFC 0070] Platform Support Tiers (#70) [RFC 0070] Merge nixos-hardware in to nixpkgs (#70) Jun 20, 2020
@nh2
Copy link

nh2 commented Jun 20, 2020

People just don’t know about nixos-hardware (because it’s not discoverable by grepping through nixpkgs)

Yes, I'm for making it part of nixpkgs, and mentioning this functionality right up in the NixOS manual in the installation section.

@Ericson2314
Copy link
Member

Ericson2314 commented Jun 20, 2020

Yes since day 1 nixos-hardware has been hampered by this: especially given that the hardware compatibility deals with versioning/interface violations---subtle variations in software and hardware that have a far more disastrous consequences than intended---I don't think it's feasible to evolve or version nixos and nixos-hardware separately. A means to synchronize everything (as a single repo provides) is absolutely required.

@nixos-discourse
Copy link

nixos-discourse commented Jun 20, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfc-on-merging-nixos-hardware-into-nixpkgs/7775/1

@Mic92 Mic92 added the status: new label Jun 20, 2020
@lukegb
Copy link

lukegb commented Jun 20, 2020

Yes, please: this would make adopting nixos-module much easier. Some of the hardware I use already has modules in nixos-hardware, but since I didn't know about it I ended up reinventing the wheel myself at first.

Copy link
Member

grahamc left a comment

One problem with the built-in profiles is I find they're good for a while, and then usually I want to override them in some way, sometimes in ways which are difficult to do without copying their code in to my repo. Is there a style guide or best practice we could come up with which makes this easier for users?


2. How to integrate it into hydra/ofborg

We cannot test the workings of the hardware of course,

This comment has been minimized.

Copy link
@grahamc

grahamc Jun 20, 2020

Member

I wonder about validating that they at least evaluate coherently? Understandably we can't actually assert they're sensible.

This comment has been minimized.

Copy link
@Profpatsch

Profpatsch Jun 20, 2020

Member

Evaluation and probably also build of the images.

@Profpatsch
Copy link
Member

Profpatsch commented Jun 20, 2020

One problem with the built-in profiles is I find they're good for a while, and then usually I want to override them in some way, sometimes in ways which are difficult to do without copying their code in to my repo.

They would slowly evolve into more composable modules definitions, as with everything.

@grahamc
Copy link
Member

grahamc commented Jun 20, 2020

I wonder if this would also be a good place to put expressions like NixOS on:

  • Packet
  • AWS
  • Hetzner
  • DigitalOcean
  • Vultr
  • Oracle Cloud

etc.?

@grahamc
Copy link
Member

grahamc commented Jun 20, 2020

This turns out to be a bit of a loaded question, and I didn't mean for it to be. The reason it is a bit loaded is because to generate the expression for Packet, you have to print out a good bit of information unique to each machine. If moving the packet expr into nixos-hardware is appropriate, then it opens a question for me if the generation should be moved in to nixos-generate-config, too.

@Ericson2314
Copy link
Member

Ericson2314 commented Jun 20, 2020

@grahamc Yeah I don't think we need to figure out every architectural decision up front. Combing the repos to me is just signalling "we can't meaningfully version these separately". Exactly how to integrate the two I rather figure out via experiment----we don't even have to integrate anything at all, and we still get the benefits of synchronous development. (For the record, I do think integrating, somehow, with nixos-generate-config would be good.)

@Ericson2314
Copy link
Member

Ericson2314 commented Jun 20, 2020

Also

One problem with the built-in profiles is I find they're good for a while, and then usually I want to override them in some way, sometimes in ways which are difficult to do without copying their code in to my repo.

I have the same problem with nixos-generated-config, already. On one hand, "Thou shalt not commit generated code", on the other hand wantonly re-running nixos-generated-config terrifies me.

Many things we could try. The most complex might be stratifying included/generated code by both "how sure we are about the auto-detection we are making our decision on", and "how essential vs merely recommended is the config item is".

@edolstra
Copy link
Member

edolstra commented Jun 20, 2020

I'm strongly opposed to this. nixos-hardware is intended as a low-barrier, little reviewed dumping ground for configurations. It should not be merged into nixpkgs which is already way too bloated, is a PR bottleneck, and therefore shouldn't accrue even more functions.

@Ma27
Copy link
Member

Ma27 commented Jun 20, 2020

I think that this is actually a good idea, also because it would be possible to generate even more hardware-specific configuration using nixos-generate-config with nixos-hardware expressions as mentioned in the RFC. @edolstra in case we don't accept this, what would you suggest to do for such a feature? (I'm not opposed to e.g. flakes, but from what I've seen it will take some time until those are stabilized).

is a PR bottleneck

Hmm, nixos-hardware doesn't seem to be a high-traffic repository, so I'm not even sure if we would notice an actual increase of PRs/issues.

should not be merged into nixpkgs which is already way too bloated

I know that I'll probably be known as the guy who initially hijacked the discussion from now on, but whenever I read something like this when people want to add e.g. a new feature, I wonder if we should reach at least some basic consensus about the direction nixpkgs should go in the future first.

@bbigras
Copy link

bbigras commented Jun 20, 2020

Would we have some QA for those hardware profiles?

For example, I made one profile for my laptop and I'm afraid to open a PR since I have no idea if I choose the right stuff. tlp vs thermald, sssd sutff like trim or not.

@Ericson2314
Copy link
Member

Ericson2314 commented Jun 20, 2020

little reviewed dumping ground for configurations.

I think it's fine to merge alone as-is, but yes we should treat it as initially dubious, and carefully vet afresh any bit that gets included in nixos-generate-config.

I wonder if we should reach at least some basic consensus about the direction nixpkgs should go in the future first.

Yes, we really need to have that discussion! (but in a dedicated thread not here.)

flakes

I don't think flakes will work because the vagaries of comparability are inherently fast moving. A kernel update can change an interface breaking an obscure out-of-tree driver at any time. The version nightmare is a social problem caused mainly by sloppy hardware manufactures that no technical solution like flakes can fix---I don't fault flakes at all.

@samueldr
Copy link
Member

samueldr commented Jun 20, 2020

This may need considerations for backporting hardware profile changes to "current stable".

The separate repository is good because it generally means the same expressions are used for both stable and unstable branches of Nixpkgs.

Folding this into Nixpkgs, I'm calling it, will make nixos-hardware as good as stale every six months.

While it is useful for older hardware too, I think many users are using it to get around issues with fresh new hardware. If it's getting stale every six months on stable, then it's going to lose most of its attraction.

@samueldr
Copy link
Member

samueldr commented Jun 20, 2020

Another issue, comes with NixOS on ARM which I wanted to write a not-RFC about for the nixos-hardware project.

Right now, NixOS on ARM are guidelines on a wiki, sometimes without actual expressions that are easy to use.

I was interested into folding in SBC expressions into NixOS hardware.

That in itself is not an issue. It's only a couple options. But there is one option that is an issue, and is an issue even outside of ARM SBCs.

Do we want Nixpkgs to start hoarding more and more arbitrarily-sourced forks of the Linux kernel?

Many ARM SBCs end up needing patches on top of mainline, or an outright fork of the kernel to be usable.

This is not an SBC-specific issue, sometimes other class of all-purpose hardware are also better served with patches or forks of the kernel.

Let's say we add support for a Chromebook range that requires patches for something to work right, until it's part of mainline, or the pinebook pro, which requires some patches or fork of mainline. They're not SBCs, might not even be ARM, but add additional kernels that would be part of Nixpkgs.

In my opinion even the raspberry pi kernel fork shouldn't be in Nixpkgs proper. Other single-purpose (as in hardware support) kernel forks shouldn't either.

Though there is a good reason to get them in Nixpkgs, even if I don't like the idea: cached builds! That chromebook user, or even me and my Pinebook would be much better served if the single-purpose forks are built and cached!

  • Is the added Hydra load okay? (probably)
  • Is the added package maintenance workload okay? (probably not)
  • Who updates those kernels? (nobody?)
  • What happens when they get stuck on "bad" versions?

And this is before even considering Mobile NixOS, which is part of Future work. Mobile NixOS, because of vendors, basically has as many kernel forks as there are devices to support. Some of them are abandoned, EOL (but working). Some of them are updated downstream, or downstream~ish (e.g. LineageOS community), but are not bumped in the repository.

@samueldr
Copy link
Member

samueldr commented Jun 20, 2020

One problem with the built-in profiles is I find they're good for a while, and then usually I want to override them in some way, sometimes in ways which are difficult to do without copying their code in to my repo. Is there a style guide or best practice we could come up with which makes this easier for users?

I hope there is a way to automatically mark anything part of <nixpkgs/nixos/hardware/> as a new default level for options, e.g. mkOpinionatedDefault or something of the like, so setting conflicting options in a user's configuration works out of the box, without having to pepper mkOpinionatedDefault all over the <nixpkgs/nixos/hardware/> expressions.

@Ericson2314
Copy link
Member

Ericson2314 commented Jun 20, 2020

@samueldr So just to understand your argument a bit better, you think that back porting changes to nixos/hardware (let's call it) from unstable like we do for other things will not be feasible? Is that because of the shear amount of stuff to backport / rate of change, or because some of them are breaking changes?

@Profpatsch
Copy link
Member

Profpatsch commented Jun 20, 2020

Do we want Nixpkgs to start hoarding more and more arbitrarily-sourced forks of the Linux kernel?

We should make a distinction between “common hardware that people want to use and/or only needs a few defaults” and “support every crazy device under the sun including their kernel blobs”.

For reference, this RFC is about the first kind.

@Mic92
Copy link
Author

Mic92 commented Jun 20, 2020

I'm strongly opposed to this. nixos-hardware is intended as a low-barrier, little reviewed dumping ground for configurations. It should not be merged into nixpkgs which is already way too bloated, is a PR bottleneck, and therefore shouldn't accrue even more functions.

Actually our review standards for nixos-hardware are higher than for nixpkgs. We have branch protection and require at least one review before changes can get merged to it. I don't think nixpkgs is a PR bottleneck either. Only 3 % of all PRs are still open, for Nix it is 15 % and hydra for 16 %. I would say the average time a nixpkgs PR is open is significant lower than most open source projects.

@Mic92
Copy link
Author

Mic92 commented Jun 20, 2020

Do we want Nixpkgs to start hoarding more and more arbitrarily-sourced forks of the Linux kernel?

We should make a distinction between “common hardware that people want to use and/or only needs a few defaults” and “support every crazy device under the sun including their kernel blobs”.

For reference, this RFC is about the first kind.

Yes the cost of maintaining a non-mainline kernel is quite high we should not do it. But we also have not any custom kernel in nixos-hardware right now either.

@Mic92
Copy link
Author

Mic92 commented Jun 20, 2020

I wonder if this would also be a good place to put expressions like NixOS on:

* Packet

* AWS

* Hetzner

* DigitalOcean

* Vultr

* Oracle Cloud

etc.?

We already have Hypervisor specific profiles in nixpkgs (KVM/Xen etc). Most often they are not vendor specific.

@Mic92
Copy link
Author

Mic92 commented Jun 20, 2020

Would we have some QA for those hardware profiles?

For example, I made one profile for my laptop and I'm afraid to open a PR since I have no idea if I choose the right stuff. tlp vs thermald, sssd sutff like trim or not.

In nixos-hardware we often rely on other sources like the archlinux wiki. We also got in contact with the TLP maintainer if necessary.

@bbigras
Copy link

bbigras commented Jun 20, 2020

In nixos-hardware we often rely on other sources like the archlinux wiki. We also got in contact with the TLP maintainer if necessary.

Thank you very much @Mic92. I'll check the arch wiki and create a PR soon for my config.

@Mic92
Copy link
Author

Mic92 commented Jun 20, 2020

One problem with the built-in profiles is I find they're good for a while, and then usually I want to override them in some way, sometimes in ways which are difficult to do without copying their code in to my repo. Is there a style guide or best practice we could come up with which makes this easier for users?

If you have a look at the current code, it is actually quite modularized. Most laptop model profiles only consist of imports of other modules like common/pc/laptop or common/cpu/intel. Some profile even purely consists of imports. Still we encourage those files to be added so that people have an anchor point to get started with. Things like adding intel microcode updates is not something one would think about adding but is important for security updates.

@Mic92
Copy link
Author

Mic92 commented Jun 24, 2020

Ok. We would need two more.

@Mic92
Copy link
Author

Mic92 commented Jun 24, 2020

Tomorrow is the next rfc steering committee meeting. The next one is in two weeks. Only around those events it would be opened up for discussion.

@Mic92
Copy link
Author

Mic92 commented Jun 25, 2020

Before proceeding with my nixos-config-generate prototype I also would like some feedback before investing too much time into it.

@Kloenk
Copy link

Kloenk commented Jun 25, 2020

@Mic92 maybe we can change nixos-generate-config, nixos-rebuild, nixos-install, nixos-enter, nixos-options, nixos-build-vms, nixos-container and nixos-version to be inside a package. This also would make testing of your new nixos-generate-config much easier.

@Ma27
Copy link
Member

Ma27 commented Jun 25, 2020

@Kloenk we had this discussion in the past: NixOS/nixpkgs#48122 (comment)

@Mic92 Mic92 mentioned this pull request Jun 25, 2020
11 of 12 tasks complete
@Mic92
Copy link
Author

Mic92 commented Jun 25, 2020

@Mic92 maybe we can change nixos-generate-config, nixos-rebuild, nixos-install, nixos-enter, nixos-options, nixos-build-vms, nixos-container and nixos-version to be inside a package. This also would make testing of your new nixos-generate-config much easier.

That might be an future perspective, but nothing I would do right now. It's too much work for one person.

Update. I think I misunderstood you. If you just want to test things the following command works:

$ nix-build -E '(import <nixpkgs/nixos> {}).config.system.build.nixos-rebuild'
@nixos-discourse
Copy link

nixos-discourse commented Jun 29, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-weekly-06-nixos-weekly/7896/1

@mexisme
Copy link

mexisme commented Jul 5, 2020

Very late to this party...

I'm generally a huge advocate for monorepo-style development, but one thing nixos-hardware helps is faster-moving updates to what can be a very tricky area for Linux: hardware that requires a lot of effort to reverse engineer for Linux.

I've found it supremely helpful to have the two repos running at rather different release cadences (even though nixos-hardware doesn't actually "release" per se...)
Are there docs for easy ways to overlay a faster-moving set of Nix expressions into the main channel, that doesn't involve a lot of clever Nix and/or creating your own local repo that manually merges the two together?

(I have experience with dealing with stuff like this in larger monorepos, but it may not quite apply cleanly to Nix...)

@Mic92
Copy link
Author

Mic92 commented Jul 6, 2020

I made a pull request to include nixos-hardware in the flake registry: NixOS/flake-registry#3 and convert nixos-hardware to a flake: NixOS/nixos-hardware#176

@Mic92
Copy link
Author

Mic92 commented Jul 6, 2020

We still need two shepherds until this Thursday to move this RFC to the next phase. If you want to know what the responsibilities of a shepherd is, ping Mic92 in #nixos on IRC (freenode)

@edolstra
Copy link
Member

edolstra commented Jul 6, 2020

I'd like to nominate myself as shepherd.

@Kloenk
Copy link

Kloenk commented Jul 6, 2020

I'd like to nominate myself as shepherd too

@mexisme
Copy link

mexisme commented Jul 11, 2020

I made a pull request to include nixos-hardware in the flake registry

What an awesome idea to do that, @Mic92 !!
Has that not solved many of the issues that started this RFC?

@Mic92
Copy link
Author

Mic92 commented Jul 11, 2020

I made a pull request to include nixos-hardware in the flake registry

What an awesome idea to do that, @Mic92 !!
Has that not solved many of the issues that started this RFC?

  1. nixpkgs updates can still break nixos-hardware as there are no evaluation tests. There are different solutions to that. One would be to include nixos-hardware into nixpkgs and the other is to integrate nixos-hardware into the CI.
  2. nixos-generate-config is still not very powerful in detecting hardware. I am working on that however currently there is another project in the pipeline that has priority over this. So I will get back to it after it.
@flokli
Copy link

flokli commented Jul 14, 2020

IIRC, these are NixOS system configuration snippets, which also interact a lot with the rest of the NixOS system. There seems to be the need to keep this tightly coupled. Moving it into a flake doesn't change the fact the code lives in a separate repo, and that pointers need to be manually synchronized. Also, refactorings with nixpkgs NixOS code gets much more complicated, as well as testing.

Having to use an experimental feature just to make some configuration snippets available seems a bit risky and overkill, which is also why I'm in favor of this RFC.

In any case, can the RFC Steering Committee move this RFC forward?

@Mic92
Copy link
Author

Mic92 commented Jul 15, 2020

In any case, can the RFC Steering Committee move this RFC forward?

We will tomorrow. Last week there were not enough committee members present to set the shepherd team.

@Kloenk
Copy link

Kloenk commented Jul 24, 2020

@samueldr, @garbas, @edolstra, @Kloenk, I created a dudle poll, hope one of the time slots are fitting.
doodle: https://dudle.crypto.koeln/poll.php?poll=22afe196
jitsi: https://meet.pbb.lc/nixos-rfc-70

@garbas
Copy link

garbas commented Jul 28, 2020

@Kloenk I can only make it 21:00CEST or later. Is it possible to also add this option to the list?

@Kloenk
Copy link

Kloenk commented Jul 29, 2020

@Kloenk I can only make it 21:00CEST or later. Is it possible to also add this option to the list?

In that time I only have to possible dates in the next 2 weeks. I created a new poll: https://dudle.crypto.koeln/poll.php?poll=11447229

@spacekookie spacekookie mentioned this pull request Jul 30, 2020
12 of 12 tasks complete
@edolstra
Copy link
Member

edolstra commented Jul 31, 2020

@Kloenk Unfortunately it turns out I cannot make it on Tuesday after all :-(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.