Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manual's build example doesn't build #2259

Open
pushcx opened this issue Jun 27, 2018 · 12 comments
Open

Manual's build example doesn't build #2259

pushcx opened this issue Jun 27, 2018 · 12 comments

Comments

@pushcx
Copy link

@pushcx pushcx commented Jun 27, 2018

Chapter 13. A Simple Nix Expression (from doc/manual/expressions/simple-expression.xml) introduces building packages with a worked example of building GNU hello. I attempted to use this to learn to build nix packages and was unable to build the package. I don't know where I went wrong, so this bug report is something of a step-by-step. (For context, I'm a longtime professional linux sysadmin and developer who's totally new to nix. I was able to set up a NixOS box from its manual to play around with.)

The chapter is structured around three steps: writing a nix expression (13.1), writing a build script (13.2), and adding the package (13.3).

13.1 doesn't explain where to create default.nix, but from the note "It is customary to place each package in a separate directory" I figured I should create a test directory with this default.nix in it. In 13.2 I also put builder.sh in this directory.

Then instead of the section heading for 13.3 ("Arguments and Variables") matching the third bullet point in the intro, it's kind of a random name. It talks about editing all-packages.nix but the syntax is invalid. As a newbie, I don't have the experience to fill in ... so the example is unusable. Additionally it talks about editing pkgs/top-level/all-packages.nix like that directory already exists, and the only file by that name that find turned up is /nix/store/ilagh0i1vxhp68hd9r73lh0f2mjqbj5x-nixos-18.09pre143771.a8c71037e04/nixos/pkgs/top-level/all-packages.nix. The description of the nix store in 5.3 has a wishy-washy warning against editing files in /nix/store and I don't see any way to regenerate that hash (or maybe I don't need to? The introduction to the hash at the top of chapter 1 says it's only based on dependencies, not contents). Still, it isn't clear at all what I'm supposed to be doing to what file, or why this is required for building hello.

Then section 13.4 starts "You can now try to build Hello." and nix-build -A hello looks like it's assuming I put those files in a directory named hello, though after renaming my directory I got the error error: getting status of '/home/pushcx/default.nix': No such file or directory from outside the directory and from inside I got error: cannot auto-call a function that has an argument without a default value ('stdenv') (with no file name or line-number). The error message is useless because I'm trying to build a package, not auto-call a function, and "auto-call" does not appear in the manual. I don't see anything more to experiment with to try to fix this.

After chatting with infinisil in #nixos, I think the two causes of my confusion are:

  1. The manual does not provide a complete, worked example of building a package.
  2. The manual conflates building a package and contributing publicly to nixpkgs.

He suggested (via the bot, so is this a frequent problem?), "If a Nix file ./foo.nix starts with something like { bar, baz }:, you can build it with nix-build -E '(import <nixpkgs> {}).callPackage ./foo.nix {}'" This doesn't match the description of callPackage in the manual, which only explains callPackage as something you'd write in all-packages.nix rather than a command-line tool. And it doesn't account for writing a builder.sh.

Additionally, it would probably be worthwhile for someone on the team to watch a newbie try to work through the doc without answering any questions, because this wasn't the first issue I had to puzzle out, just the issue I couldn't puzzle out.

@vaibhavsagar
Copy link
Member

@vaibhavsagar vaibhavsagar commented Jun 29, 2018

It is frustrating that the example in the manual doesn't build, especially when only small changes are required to default.nix:

let
  pkgs = import <nixpkgs> {};
in
pkgs.stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = pkgs.fetchurl {
    url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
  perl = pkgs.perl;
}
@pushcx
Copy link
Author

@pushcx pushcx commented Jul 4, 2018

Thanks very much for the improvement, @vaibhavsagar, I can see how this looks more like a complete example. What command are you using to build? Do you have it in a hello directory? Are you running nix-build from within that directory, or a parent? Are you able to run nix-shell or nix-env?

(Asking because I've tried a bunch of variations like nix-build -A hello and nix-build -A . with no luck.)

@pushcx
Copy link
Author

@pushcx pushcx commented Jul 4, 2018

I sort of got it to start a build, but got a strange failure out of configure. I tried to run nix-shell because that seemed like the right tool but it also lacks newbie docs. There's a section over in the "nixpkgs" manual (though I'm not trying to contribute to a public package so I don't understand why it's there) but its description of "phases" don't at all match what's present in the shell and lead to a whole 'nother set of bugs and underdocumented topics. One again, I don't see any way forward with building a package.

I appreciate the folks who've tried to help, but I've reached my limit of time and frustration on this experiment. I'm unsubscribing from this issue; I don't know what your project standards are around closing or keeping issues open, so I'll leave it to a contributor to handle.

@vaibhavsagar
Copy link
Member

@vaibhavsagar vaibhavsagar commented Jul 10, 2018

Oops, that's meant to go in a default.nix file that you then run by executing nix-build default.nix or just nix-build (which looks for a default.nix if no filename is provided).

@cmal
Copy link

@cmal cmal commented Nov 15, 2018

@vaibhavsagar Hi, Thanks for your fix. I am stepping over the errors mentioned by the original author. I am using nix-build default.nix to build GNU Hello, but still errors:


...
...
make[2]: Entering directory '/private/tmp/nix-build-hello-2.1.1.drv-0/hello-2.1.1/src'
source='hello.c' object='hello.o' libtool=no \
depfile='.deps/hello.Po' tmpdepfile='.deps/hello.TPo' \
depmode=gcc3 /nix/store/n9hba031gjky8hpjgx9fnlaxhidyzxbz-bash-4.4-p23/bin/bash ../depcomp \
clang -DLOCALEDIR=\"/nix/store/vh1c9dv71ggh4viyvlvjr4gbvvnv6az8-hello-2.1.1/share/locale\" -DHAVE_CONFIG_H -I. -I. -I.. -I. -I. -I.. -I../intl -I../intl    -g -O2 -c `test -f 'hello.c' || echo './'`hello.c
hello.c:332:17: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security]
        printf (_("hello, world\n"));
                ^~~~~~~~~~~~~~~~~~~
hello.c:47:19: note: expanded from macro '_'
#define _(String) gettext (String)
                  ^~~~~~~~~~~~~~~~
hello.c:332:17: note: treat the string as an argument to avoid this
        printf (_("hello, world\n"));
                ^
                "%s",
hello.c:47:19: note: expanded from macro '_'
#define _(String) gettext (String)
                  ^
hello.c:340:10: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security]
        printf (_("\
                ^~~~
hello.c:47:19: note: expanded from macro '_'
#define _(String) gettext (String)
                  ^~~~~~~~~~~~~~~~
hello.c:340:10: note: treat the string as an argument to avoid this
        printf (_("\
                ^
                "%s",
hello.c:47:19: note: expanded from macro '_'
#define _(String) gettext (String)
                  ^
2 errors generated.
make[2]: *** [Makefile:199: hello.o] Error 1
make[2]: Leaving directory '/private/tmp/nix-build-hello-2.1.1.drv-0/hello-2.1.1/src'
make[1]: *** [Makefile:175: all-recursive] Error 1
make[1]: Leaving directory '/private/tmp/nix-build-hello-2.1.1.drv-0/hello-2.1.1'
make: *** [Makefile:131: all] Error 2
builder for '/nix/store/6h7c2np6z1j29fci1ck721q1x8s533qw-hello-2.1.1.drv' failed with exit code 2
error: build of '/nix/store/6h7c2np6z1j29fci1ck721q1x8s533qw-hello-2.1.1.drv' failed

How can I fix this?

$ uname -a                                                                                                   
Darwin xxx.local 18.2.0 Darwin Kernel Version 18.2.0: Fri Oct  5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64 x86_64
@vaibhavsagar
Copy link
Member

@vaibhavsagar vaibhavsagar commented Nov 15, 2018

These look like macOS-specific issues with clang. I don't think I can help.

@gloaming
Copy link

@gloaming gloaming commented May 3, 2019

Out of curiosity I tried to reproduce this by building with clang on NixOS, but it works fine, even though the same warnings are enabled. I would guess that the compiler has special logic for the use of gettext that doesn't work on Darwin for some reason, maybe.

@cmal Could you try with this line inside the mkDerivation?

hardeningDisable = stdenv.lib.optional stdenv.isDarwin "format";

Relevant links I found:
NixOS/nixpkgs#12895
https://fedoraproject.org/wiki/Format-Security-FAQ

@mawis
Copy link

@mawis mawis commented Jun 6, 2019

I also have problems just at this very basic point of trying to learn how to write packages. I was following the manual with the same problems as the original author of this issue. Indeed it is very frustrating if even the hello world example in the documentation cannot be build.

With the changes in default.nix mentioned above I came a bit futher. (Thanks @vaibhavsagar ). Still it doesn't build successfully. I get the following error message:

[…]
source='getopt1.c' object='getopt1.o' libtool=no \
depfile='.deps/getopt1.Po' tmpdepfile='.deps/getopt1.TPo' \
depmode=gcc3 /nix/store/cinw572b38aln37glr0zb8lxwrgaffl4-bash-4.4-p23/bin/bash ../depcomp \
gcc -DLOCALEDIR=\"/nix/store/qwqg5ah3n4wg29h4m1h65b4sxbydk3v0-hello-2.1.1/share/locale\" -DHAVE_CONFIG_H -I. -I. -I.. -I. -I. -I.. -I../intl -I../intl    -g -O2 -c `test -f 'getopt1.c' || echo './'`getopt1.c
gcc  -g -O2   -o hello  hello.o version.o getopt.o getopt1.o
make[2]: Leaving directory '/build/hello-2.1.1/src'
Making all in man
make[2]: Entering directory '/build/hello-2.1.1/man'
help2man --name="Friendly Greeting Program" ../src/hello >hello.1
/nix/store/cinw572b38aln37glr0zb8lxwrgaffl4-bash-4.4-p23/bin/bash: help2man: command not found
make[2]: *** [Makefile:282: hello.1] Error 127
make[2]: Leaving directory '/build/hello-2.1.1/man'
make[1]: *** [Makefile:175: all-recursive] Error 1
make[1]: Leaving directory '/build/hello-2.1.1'
make: *** [Makefile:131: all] Error 2
builder for '/nix/store/mvg07i5xl1g71bpj7ic9q2r1wrh2565v-hello-2.1.1.drv' failed with exit code 2
error: build of '/nix/store/mvg07i5xl1g71bpj7ic9q2r1wrh2565v-hello-2.1.1.drv' failed

Maybe I should also mention, that I changed the hash value in default.nix to c510e3ad0200517e3a14534e494b37dc0770efd733fc35ce2f445dd49c96a7d5 is this is the sha256sum I get for ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz

I'm trying this on NixOS 19.03.

@mawis
Copy link

@mawis mawis commented Jun 6, 2019

Okay, finally I got it building. I needed to make the help2man package accessible to the build process. So what worked for me is this:

file default.nix:

let
  pkgs = import <nixpkgs> {};
in
pkgs.stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = pkgs.fetchurl {
    url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    sha256 = "c510e3ad0200517e3a14534e494b37dc0770efd733fc35ce2f445dd49c96a7d5";
  };
  perl = pkgs.perl;
  help2man = pkgs.help2man;
}

file builder.sh:

source $stdenv/setup                                                             
                                                                                 
PATH=$perl/bin/:$help2man/bin/:$PATH                                             
                                                                                 
tar xvfz $src                                                                    
cd hello-*                                                                       
./configure --prefix=$out                                                        
make                                                                             
make install

And building this with nix-build in the folder these two files are in.

@burbon
Copy link

@burbon burbon commented Dec 6, 2019

Following documentation further

Discerning readers will note that the buildInputs could just as well have been set in the Nix expression, like this:
buildInputs = [ perl ];
which seems to solve help2man dependency so

let
  pkgs = import <nixpkgs> {};
in
pkgs.stdenv.mkDerivation {
  name = "hello-2.1.1";
  buildInputs = [ pkgs.perl ];
  src = pkgs.fetchurl {
    url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
  };
}

this is just building fine for me using nix-build

@bburdette
Copy link
Contributor

@bburdette bburdette commented Dec 11, 2019

Ok I found a relevant stackoverflow for this. A quote:

And finally, there are two common formats for derivations in default.nix: derivation, and callPackage derivation. You can't nix-build the latter. Almost any package in nixpkgs is written in this style, see hello. But you can
nix-build -E 'with import { }; callPackage ./path/to/default.nix { }'
as a workaround. nix-shell also supports this -E argument.

Sure enough, with the original default.nix and builder.sh in a directory, I can make it work:

bburdette@BB-5520:~/op-code/nix-example$ ls
builder.sh  default.nix
bburdette@BB-5520:~/op-code/nix-example$ nix-build -E 'with import <nixpkgs> { }; callPackage ./default.nix { }'
/nix/store/fkr5ab4lc1k0cn98lp13615pvc8r1nyh-hello-2.1.1
bburdette@BB-5520:~/op-code/nix-example$ ls
builder.sh  default.nix  result
bburdette@BB-5520:~/op-code/nix-example$ ./result/bin/hello 
Hello, world!
@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Oct 28, 2020

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-on-prgmr-and-failing-to-learn-nix-push-cx/9672/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants