nix-shell doesn't work right with nixpkgs' own bash

[timbertson]

bash-4.3$ nix-shell --pure -p nodejs-0_10
[nix-shell:~]$

^ working shell

But if I use nixpkgs' own bash:

bash-4.3$ nix-build --no-out-link -A bash '<nixpkgs>'
/nix/store/xag5ayq906w9zhlxs8wayv4kvpiyqphq-bash-4.3-p42
bash-4.3$ env PATH="/nix/store/xag5ayq906w9zhlxs8wayv4kvpiyqphq-bash-4.3-p42/bin/:$PATH" nix-shell --pure -p nodejs-0_10
bash: shopt: progcomp: invalid shell option name
bash: complete: command not found...

\[\][nix-shell:~]$\[\]

^ weird prompt, no readline support, errors printed

This happens reliably when you run nix-shell from inside nix-shell, since most derivations implicitly rely on bash. I run into it all the time when I run nix-shell from a dir where I'm using direnv to set up project-specific dependencies, since that acts a lot like nix-shell.

I'm not really sure what the right fix is, but I can think of a few options:

• have nix-shell explicitly invoke pkgs.bashInteractive rather than whatever's available on $PATH. I think it should do this by default if possible, but it could also be behind a flag
• have nix-shell invoke the first bash on $PATH which wasn't compiled with --disable-readline (although this may not even be possible)
• have nix-shell accept an explicit path-to-bash argument so that it doesn't have to do either of the above (requires most work from the user though)

[fkz]

I regularly use nix-shell -p bashInteractive ... for that reason.

[timbertson]

That's a useful workaround, but I'd love if nix-shell could figure this out by default, as it's really not obvious where the errors come from (I assumed for a long time that it was something in my .bashrc or something, as I had no idea there even was a bashInteractive variant).

Also, it just doesn't feel very nix-y that nix-shell depends on whichever bash it happens to find on $PATH.

[fkz]

looking at the source code, there's the NIX_BUILD_SHELL environment variable that can be used for this (and grepping the nix source reveals it's only used at this point) (I think this would be your third proposed fix). If it's not set, it defaults to the bash in PATH.
I think I'll set this in my nixos configuration to bashInteractive now.

[fkz]

hm, NIX_BUILD_SHELL isn't used when using --pure. I would consider this a bug.

[fkz]

Thinking a little bit more about it, at least for me the problem most of the time is not nix-shell in (something similiar to) nix-shell but simply bash in nix-shell. My use case is mostly starting IDEs inside nix-shell which in turn start a terminal window. That's of cause a different use case than yours; and here adding bashInteractive seems like the right thing to do. Could be better documented still.

[Ericson2314]

I swear there was another issue in this, more generally whether there should be a custom list of environment variables that are supposed to leak.

In any event I agreee with your last post. I'm this specific case I think it would be cleaner to have nix-shell make the derivation based off the value of NIX_BUILD_SHELL rather than leak the variable itself.

[fkz]

you're probably right, that's similiar to how it's done with PATH, we get the bash from PATH but don't propagate PATH itself; I guess we should do it like that with NIX_BUILD_SHELL too (in the --pure case) (we can't use nix-shell after that anyway because it's not in the PATH anymore and even if it was (nix-shell --pure -p nix anyone) we'd need to set at least NIX_DAEMON...

Custom environment variables sound interesting too, we could do things like propagate X11 or pulseaudio and still using --pure. But that's a little off topic here.

[Ericson2314]

Sure I could see using NIX_BUILD_SHELL and then falling back on PATH---I'd love to fallback on bashInteractive but that's a layer violation. Maybe nixpkgs can wrap nix-shell to define NIX_BUILD_SHELL as ${bashInteractive} if its not defined already.

[timbertson]

I'd be happy if $NIX_BUILD_SHELL could be used with --pure, as in #733, thanks @fkz. I'd be happier if it used bashInteractive by default, but I can see how that might be outside of its responsibilities.

Another alternative would be to always looks for a system bash (/bin/bash, /usr/bin/bash) before falling back to $PATH. It'd be more code, and would be thoroughly surprising for anyone who has their preferred bash elsewhere on $PATH, although that should be very rare. Thoughts?

[fkz]

this is also related to #498 and #545 and NixOS/nixpkgs#4260.

[fkz]

We could also use the user's shell. This currently only works if this is bash.
I don't think it's good to look at specific paths like /usr/bin/bash (not even there under nixos) or /run/current-system/sw/bin/bash or ~/.nix-profile/bin/bash.

I think the clearest solution would be to use the bash from stdenv. We already include $stdenv/setup, so maybe stdenv could also export an interactive bash (maye $bash or $bashInteractive). That way we'd depend on the same bash from the nixpkgs that's available in the package we install; not some other nixpkgs.
An other way would be to wrap nix-shell inside nixpkgs.

[timbertson]

We could also use the user's shell

I don't think that's a good idea - I use fish, and I know that plenty of nix-isms (bash functions, mainly) are not going to work. Nix-shell is tied to bash, although you can get by with other shells if you don't need to use functions.

maybe stdenv could also export an interactive bash (maye $bash or $bashInteractive)

Is that going to cause a new bashInteractive dependency on every new derivation via the stdenv? If so, it sounds like that also removes the point of maintaining separate bash / bashInteractive packages since nobody will depend on "just bash". But maybe I misunderstand how the stdenv works.

[fkz]

I just used this beast expression to get a nice bash inside nix-shell

nix-shell --expr "let nixpkgs = import <nixpkgs> {}; in nixpkgs.lib.overrideDerivation (import ./. {}) (p: { buildInputs = p.buildInputs ++ [ nixpkgs.bashInteractive ]; })"

[kc1212]

Hi, I also use fish so something like nix-shell '<nixpkgs>' -A pan doesn't work for me even when $NIX_PATH is set correctly (but it has spaces instead of colons because of fish). I'm not too familiar with Nix so what's the recommended workaround if I don't want to use bash?

[timbertson]

@kc1212 nix-shell works fine from fish for me. I don't know what error you're getting (or what your NIX_PATH is set to), but I believe you should create a separate github issue with that information - this issue is about running nix-shell from within bash.

[catern]

This issue is the first Google result for "nix bash: complete: command not found", so let me note that if you're seeing such error messages with Nix's bash, you may want to install "bash-interactive" rather than "bash".

[DavidEGrayson]

No one in this thread has suggested that the bash from nixpkgs should have readline support, just like bashInteractive. Wouldn't that be a simple solution to the problems people are having here? The only downsides I see are that builds would take slightly longer since bash has more work to do (i.e. loading the readline shared library) and that mass rebuilds would be more frequent, happening whenever readline changes.

[zimbatm]

Yes that's a good point. Or stop making the non-interactive bash default. As a user I'm expecting to get bash when running nix-env -iA nixos.bash. bashNonInteractive is an optimisation only useful inside of a build context.

[edolstra]

Fixed by c94f3d5.

[DavidEGrayson]

Thanks!

Looking at the code you added, I don't see anything that will add bashInteractive to the PATH, so subshells will still behave incorrectly (e.g. if you run bash or nix-shell inside a nix-shell). Also, that commit introduces more coupling between nix-shell and Nixpkgs than there already was, which makes me a bit sad as someone who likes to write derivations my own way outside of Nixpkgs. I suppose I can just specify NIX_BUILD_SHELL in my derivation or something if that coupling ever causes me any issues though.

I wrote up some of my thoughts on related nix-shell issues here:

NixOS/nixpkgs#27126 (comment)

My recommendations came down to this: make nixpkgs.bash have readline support, and allow derivations to specify a PATH environment variable that will get added to the host environment's PATH in nix-shell.

[Ericson2314]

Also, that commit introduces more coupling between nix-shell and Nixpkgs than there already was

I agree this is unfortunate. But know it was already coupled because nix-shell sources stdenv/setup.hs! IMO long term nix-shell needs to be rethought to properly decouple it: what we were doing before is essentially sweeping the coupling under the rug.

[DavidEGrayson]

Yes, I noticed that one too. nix-shell will check if $stdenv/setup exists and source it if so. It would have been nice if it used some more generic mechanism, like if it checked $builder_setup_script instead of just replaced $stdenv/setup. I worked around it by providing a pretend $stdenv directory.

[Ericson2314]

@DavidEGrayson Right. At the core nix derivation run some command in some environment. We have --command and --run already, but it would be a good foundation to run an arbitrary command in the derivation environment (pure or impure), bypassing bash altogether. nixpkgs-shell would be a layer on top dealing with bash interactive, $stdenv/setup, etc.