stdenv: add uutils-coreutils stdenv

[SuperSandro2000]

To try this use the pkgsUutils overlay and replace python3 with any package:

nix-build -A pkgsUutils.python3

ToDo:

• merge coreutils: disable tests which fail in sandbox #116241
• wait for 1f831f0 to be in master

Issues preventing us from using it in a stdenv:

• autotools does not work because the died not handle escape codes tr does not handle escape sequences correct uutils/coreutils#1817
• install does not work when installing files from directors install: does not remove path from files uutils/coreutils#1823
• head does not work with null bytes head: missing --zero-terminated uutils/coreutils#1822
• touch cannot parse some date touch -m -h -d 2009-10-10 link is failing with "Unable to parse date" uutils/coreutils#2311
• cp to many files open cp Too many open files (os error 24) uutils/coreutils#2312
• cp main panic cp --reflink=auto panics when copying nested symlinks with restrict permissions uutils/coreutils#2314
• converting absolute symlinks to relative ones Converting absolute to relative symlink errors with No such file or directory uutils/coreutils#2335
• ln multiple --symbolic ln: The argument '--symbolic' was provided more than once, but cannot be used multiple times uutils/coreutils#2359
• missing symlink from test to [ make with MULTICALL is missing symlink from test to [ uutils/coreutils#2420
• tr posix character classes tr does not support posix character class  uutils/coreutils#2421
• cp dangling or nested symlinks cp: copying dangling or nested symlink results in error uutils/coreutils#2422
• cp permission denied cp: permission denied uutils/coreutils#2423
• cut not working in rustc cut: -d= works differently than gnu-cut uutils/coreutils#2424
• Fix installation of [ symlink https://github.com/Fix installation of [ symlink uutils/coreutils#2491
• Create completion directories incase they do not exist Create completion directories incase they do not exist uutils/coreutils#2490

Motivation for this change

• Try it out!
• https://discourse.nixos.org/t/rust-coreutils-replacement/11867

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rust-coreutils-replacement/11867/4

[kvtb]

Is it ...practical?
Rust is funny, but uutils-coreutils is 80MB vs. 1.5MB of coreutils

[SuperSandro2000]

Is it ...practical?
Rust is funny, but uutils-coreutils is 80MB vs. 1.5MB of coreutils

The single binary is much smaller and when rust supports lining to shared find it will be reduced even further. uutils/coreutils#747

[siraben]

Changing the hash (temporary workaround) GNU Hello compiles and runs:

$ nix-build --show-trace -E 'with import ./. { }; hello.override { stdenv = stdenvUutilsCoreutils; }' && ./result/bin/hello
[nix-shell:~/nixpkgs]$ nix-build --show-trace -E 'with import ./. { }; hello.override { stdenv = stdenvUutilsCoreutils; }' && ./result/bin/hello
/nix/store/jxklig35p42bxhp9yzk0gkgz2d5i8vml-hello-2.10
Hello, world!

[siraben]

Checked as of 2021-06-16

• ed failed to build

shrinking RPATHs of ELF executables and libraries in /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17
shrinking /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/bin/ed
gzipping man pages under /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/share/man/
strip is /nix/store/b8y61srrw1m2n87dr5snfaxy0rqgk6qy-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/bin
patching script interpreter paths in /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17
/nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/bin/red: interpreter directive changed from "#! /bin/sh" to "/nix/store/3xqkrab9zqpv91bw1xp8551sjk7qrc34-bash-4.4-p23/bin/sh"
thread 'main' panicked at 'Unable to parse date
Invalid day.', src/uu/touch/src/touch.rs:248:19
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
builder for '/nix/store/yh6a7878qfg3qqi24mk61jaax47y2jpa-ed-1.17.drv' failed with exit code 101
error: build of '/nix/store/yh6a7878qfg3qqi24mk61jaax47y2jpa-ed-1.17.drv' failed

• vim failed to build

these derivations will be built:
  /nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv
building '/nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv'...
unpacking sources
unpacking source archive /nix/store/dvh1x2m7gvp1iv5jjlvrf5ry5g3ay23k-source
cp: error: '/nix/store/dvh1x2m7gvp1iv5jjlvrf5ry5g3ay23k-source/src/normal.c' -> 'source/src/normal.c': Too many open files (os error 24)
do not know how to unpack source archive /nix/store/dvh1x2m7gvp1iv5jjlvrf5ry5g3ay23k-source
builder for '/nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv' failed with exit code 1
error: build of '/nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv' failed

Compiling other packages with the uutils-based stdenv:

• vim (symlink converting fails)
• coreutils-gnu (failed with error: The argument '--symbolic' was provided more than once, but cannot be used multiple times)
• tinycc (failed)
• chez
• pforth
• nawk
• gawk
• guile
• chicken (failed, unable to parse date)
• miranda (cp: '/nix/store/dl6ahdcycmw4yry2zpmslq7w7i2mdlvk-source/miralib/ex/box.m' -> 'source/miralib/ex/box.m': Permission denied (os error 13))
• rustc (error: Found argument '--libdir' which wasn't expected, or isn't valid in this context)
• bash
• dash
• tmux
• screen
• go (failed, unable to parse date)
• htop (cp: '/nix/store/21zp1xfq78bp4rgvdlvd5r2d0rmmp55n-source/README' -> 'source/README': Permission denied (os error 13))
• uutils-coreutils
• fish
• ghc (mkdir: ./utils/ghc-cabal/dist/build/tmp//.: No such file or directory (os error 2))
• openjdk (error: cannot find symbol)
• julia (failed, failing test in LinearAlgebra/triangular)
• tcl (failed, ln: error: No such file or directory (os error 2))
• sbcl (failed, unable to parse date)
• emacs (failed, unable to parse date)
• binutils-unwrapped
• the-powder-toy
• gnumake
• cmake
• ripgrep
• gnupg
• git (test failure)

[siraben]

Unfortunately it will take some digging to figure out exactly what went wrong in the builds that failed, so that upstream has a clear view on what incompatibilities they need to address.

[siraben]

Please rebase and remove patches that were merged. I think that's nearly all the blocking issues needed to be a replacement for coreutils?

[SuperSandro2000]

Please rebase and remove patches that were merged. I think that's nearly all the blocking issues needed to be a replacement for coreutils?

There where still other issues when building rustc which I didn't investigate at the time.

[vcunat]

Nit: copy&paste "Musl".

[sylvestre]

Any chance this pr gets merged? Thanks

[SuperSandro2000]

Any chance this pr gets merged? Thanks

No, because it is not practical and uutils is not fully compatible with gnu coreutils.

[AndersonTorres]

Closing this as sadly abandoned.

[06kellyjac]

Could draft it until uutils is more matute

[SuperSandro2000]

Please re-open. I have no mechanic to not delete this branch from my fork.

[sylvestre]

I think it is now mature enough to replace gnu.

[nyabinary]

0.0.22 released

[siraben]

Can we rebase again?