Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove cruft from platform descriptions #34274

Open
Ericson2314 opened this issue Jan 25, 2018 · 17 comments
Open

Remove cruft from platform descriptions #34274

Ericson2314 opened this issue Jan 25, 2018 · 17 comments

Comments

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Jan 25, 2018

Simply put, I should be able to do

$ nix-build '<nixpkgs>' --arg crossSystem '{ config = "aarch64-unknown-linux-gnu"; }' -A stdenv

and that should work. I could make it work right now with a dirty white-list and fall-backs, but that's kind of defeats the point.

Currently, the biggest culprits are

  1. Random top-level fields, like withTLS and openssl.system. See https://github.com/NixOS/nixpkgs/blob/master/lib/systems/examples.nix
  2. The platform field, see https://github.com/NixOS/nixpkgs/blob/master/lib/systems/platforms.nix
  3. arch which overlaps with the far more principled parsed,arch. See elaborate https://github.com/NixOS/nixpkgs/blob/master/lib/systems/default.nix#L17.

The general design issue is we need to handle the per-package random crap in a more modular way, without giving up the possibility of the user to override with their own values. My GNU ld emulation picking logic at https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/bintools-wrapper/default.nix#L167-L179 is certainly in a more modular location, but also cannot be nicely overridden.

Checklist

  • #34303: withTLS
  • #34315: bigEndian, openssl.system
  • #34351: linux kernel cleanups
  • #107214 / #110544: A few cleanups:
    • Get rid of nesting items under platforms for no good reason
    • group linux config fields rather than merely prefixing them
    • improve defaulting logic so config is enough in more cases.
  • Combine/rerrange platforms.nix and examples.nix
    • We should only default to generic/"multiplatform" settings; specific machines should be specified by hand.
    • Specific machines shouldn't be partly in platforms.nix and partly in examples.nix, that is just confusing.
  • Update manual to show how indeed the config is usually enough, and remove the reference to this issue.

CC @dtzWill @dezgeg @Infinisil @teto

@teto
Copy link
Contributor

@teto teto commented Jan 26, 2018

I have a PR in the works (should be ready tomorrow) that allows to do

my_kernel = buildLinux {
kernelAutoModules=true, ignoreConfigErrors=true, kernelPreferBuiltin
}

which makes for a much better experience as it looks more like your typical derivation.
I let the parameters kernelAutoModules etc as defaults in the platform sets. Do you suggest I remove these parameters from these platform sets https://github.com/NixOS/nixpkgs/blob/master/lib/systems/platforms.nix#L36 ?

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Jan 26, 2018

@teto I'd say only leave things for kernel headers there, as that actually effects everything (on Linux).

@dezgeg
Copy link
Contributor

@dezgeg dezgeg commented Jan 26, 2018

I agree for refactoring openssl and most of the kernel stuff out, but much of the other stuff seems quite hard to get rid of.

@dezgeg
Copy link
Contributor

@dezgeg dezgeg commented Jan 26, 2018

I made some WIP stuff for the kernel one time: https://github.com/NixOS/nixpkgs/compare/master...dezgeg:kernel-plat-fix?expand=1, but IIRC some of the kernels failed to boot during runtime, still need to debug that one day.

@Ericson2314 Ericson2314 mentioned this issue Jan 26, 2018
0 of 8 tasks complete
Ericson2314 added a commit to obsidiansystems/nixpkgs that referenced this issue Jan 26, 2018
glibc removed the underlying flag in 2011 in
83cd14204559abbb52635006832eaf4d2f42514a [1].

This gets us one step closer to fixing NixOS#34274: the cross stdenv for
aarch64-unknown-linux-gnu at least evals now.

Thanks to @dezgeg for doing all the research for this.

[1]: https://sourceware.org/git/?p=glibc.git;a=commit;h=83cd14204559abbb52635006832eaf4d2f42514a
dezgeg added a commit that referenced this issue Jan 26, 2018
glibc removed the underlying flag in 2011 in
83cd14204559abbb52635006832eaf4d2f42514a [1].

This gets us one step closer to fixing #34274: the cross stdenv for
aarch64-unknown-linux-gnu at least evals now.

Thanks to @dezgeg for doing all the research for this.

[1]: https://sourceware.org/git/?p=glibc.git;a=commit;h=83cd14204559abbb52635006832eaf4d2f42514a
@Ericson2314 Ericson2314 mentioned this issue Jan 27, 2018
0 of 8 tasks complete
@teto teto mentioned this issue Jan 28, 2018
3 of 8 tasks complete
@Ericson2314 Ericson2314 added this to the 18.03 milestone Jan 31, 2018
@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Feb 7, 2018

Per #34645 (comment) @dtzWill and I need to figure out whether to moved libc into parsed, or similar. @dezgeg or anyone else have any opinions? #34444 is also impacted.

@dtzWill
Copy link
Member

@dtzWill dtzWill commented Feb 7, 2018

For what it's worth, I've started using just the "config" in my cross specifications and so far so good!

I don't cross-build kernels as part of my work or testing, and I note that is the remaining TODO item :). Just wanted to chime in that the rest is in a good place!

Minor convenience thing: it's annoying to specify --arg crossSystem '{ config = "x"; }' instead of something like --argstr crossConfig x, especially as the good work on this issue makes anything other than config unnecessary (at least usually). Is there a simple way we could allow the shorter syntax without compromising abstractions?
(similar to how we handle "system" today, minus special nix support)

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Feb 7, 2018

@dtzWill Maybe we can get --argstr crossSystem.config to work for 1.12? Though I feel bad asking for new features for that release, I'm hesitant to add any more sugar as the platform stuff is already far too complicated.

made issue: NixOS/nix#1850

@dtzWill
Copy link
Member

@dtzWill dtzWill commented Feb 8, 2018

@Ericson2314 thanks for the issue, and agreed not priority for nix update. I was just wanting to discuss it to see what you had in mind.

I'll follow that issue, neat idea!

I agree we shouldn't further complicate the platform stuff, haha, it's actually partly why I asked O:).

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented May 15, 2018

I made a bunch of changes so this is now true. All the examples contain at most config, platform, and additional platform-specific flags like useIosPrebuilt that are new and don't overlap. Yay!

@Profpatsch
Copy link
Member

@Profpatsch Profpatsch commented Aug 15, 2019

This is still linked in the manual, should be removed if closed.

@stale
Copy link

@stale stale bot commented Aug 8, 2020

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

@stale stale bot added the 2.status: stale label Aug 8, 2020
@siraben
Copy link
Member

@siraben siraben commented Nov 24, 2020

Still important to me

@stale stale bot removed the 2.status: stale label Nov 24, 2020
@siraben
Copy link
Member

@siraben siraben commented Dec 13, 2020

I can take a look at this once #105364 is merged.

@siraben
Copy link
Member

@siraben siraben commented Jan 10, 2021

@Ericson2314 @Profpatsch Is this still an issue? Can the reference in the manual be deleted?

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Jan 11, 2021

If we can merge #107214, and (better still) rearrange examples.nix and platforms.nix into something more sensible afterwords, then I'd say this is done.

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Jan 11, 2021

Added the checkboxes for those.

@Ericson2314
Copy link
Member Author

@Ericson2314 Ericson2314 commented Jan 23, 2021

@siraben OK, just the final coding step (and manual victor lap) remains. If you want to take a stab at this, that would be much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants