Python package build input executables not in PATH in development mode shell

[jluttine]

Issue description

Executables of build inputs aren't available in "development mode" nix shell for a Python package.

Steps to reproduce

I have a directory with the following two files only:

setup.py:

if __name__ == "__main__":
    from setuptools import setup, find_packages
    setup(
        packages = find_packages(),
        name     = "test",
        version  = "0.0.0",
    )

shell.nix:

let
  pkgs = import <nixpkgs> {};
in pkgs.python3Packages.buildPythonPackage rec {
  pname = "test";
  version = "0.0.0";
  src = ./.;
  propagatedBuildInputs = with pkgs.python3Packages; [ jupyterhub pkgs.wget ];
  doCheck = false;
}

I then run nix-shell --pure in this directory. In the nix shell, I have neither jupyterhub executable nor wget executable in PATH.
I wrote shell.nix according to these instructions:

https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/python.section.md#development-mode

The package indeed seems to be installed as pip install -e I suppose, but the executables of the build inputs aren't in PATH in that shell.

Technical details

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.33, NixOS, 19.09pre174426.acbdaa569f4 (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2`
 - channels(root): `"nixos-19.09pre174426.acbdaa569f4"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[jluttine]

I downgraded my NixOS and now it works.

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.33, NixOS, 19.09pre174426.acbdaa569f4 (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2`
 - channels(root): `"nixos-19.09pre174426.acbdaa569f4"`
 - nixpkgs: `/nix/store/6s8pxfslfgjw2ncgnqd6zclrz3wyzh2x-1233c8d9e9bc463899ed6a8cf0232e6bf36475ee.tar.gz`

So I suppose there is some recent change that broke this?

EDIT: Probably that version information is a bit misleading as I downgraded my system by setting the following in configuration.nix:

  nix.nixPath = [
    "nixpkgs=https://github.com/NixOS/nixpkgs/archive/1233c8d9e9bc463899ed6a8cf0232e6bf36475ee.tar.gz"
    "nixos-config=/etc/nixos/configuration.nix"
  ];

If there's a better way to pin system-wide nixpkgs, I'd be happy to hear. This method didn't provide sha256 validation.

[symphorien]

Python packages now have strictDeps = true; so only nativeBuildInputs are in PATH iirc.
This shell.nix seems to work:

let
  pkgs = import <nixpkgs> {};
in pkgs.python3Packages.buildPythonPackage rec {
  pname = "test";
  version = "0.0.0";
  src = ./.;
  nativeBuildInputs = with pkgs; [ python3Packages.jupyterhub wget ];
  propagatedBuildInputs = with pkgs.python3Packages; [ jupyterhub ];
  doCheck = false;
}

[FRidh]

Correct. This seems to be a not-so-nice side-effect.

[CMCDragonkai]

It does make it more complex to use nix-shell as a development tool. It could be better that we have a purpose built nix-dev designed for development and leave nix-shell for simulating a build environment strictly.

[jluttine]

I tried to add jupyterlab or jupyterhub to nativeBuildInputs:

  nativeBuildInputs = with pkgs.python3Packages; [ jupyterlab ];

But I get the following warnings and the nix shell is totally broken:

$ nix-shell
bash: /nix/store/fvv5pvb1j7hvq7p6xk6frh4xi5hfll5s-coreutils-8.30/bin/mktemp: Argument list too long
bash: /nix/store/fvv5pvb1j7hvq7p6xk6frh4xi5hfll5s-coreutils-8.30/bin/mkdir: Argument list too long
bash: /nix/store/nqvqkrkfbl824bsv5zfyjj55h1rvyvbq-python3.7-bootstrapped-pip-18.1/bin/pip: Argument list too long

Weird thing is that I have two machines with the same nixpkgs version but the other one gives these errors and the other one works just fine.. Any ideas what's wrong or is there some other way to add jupyterlab etc to PATH in nix shell?

[jluttine]

Using depsBuildBuild instead of nativeBuildInputs fixed that issue for me.

[CMCDragonkai]

I thought we usually use one of buildInputs, nativeBuildInputs, propagatedBuildInputs or propagatedNativeBuildInputs?

…

On 7 June 2019 16:20:55 WET, Jaakko Luttinen ***@***.***> wrote: Using `depsBuildBuild` instead of `nativeBuildInputs` fixed that issue for me. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #59257 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/propagating-runtime-dependencies-to-python-packages/20788/1