bundlerApp is unable to install/build gem from a git source

[dsander]

Describe the bug

bundlerApp fails to build when the gem configured in the Gemfile is pointing to a git repository.

Gemfile

source 'https://rubygems.org'

gem 'mdl', github: 'markdownlint/markdownlint'

default.nix

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/ff5da29a3ffc3b2a37386f8904b63ca861716d2f.tar.gz) { }
}:

pkgs.bundlerApp
{
  pname = "mdl";
  gemdir = ./.;
  exes = [ "mdl" ];
}

shell.nix

let
  pkgs = import (fetchTarball https://github.com/NixOS/nixpkgs/archive/ff5da29a3ffc3b2a37386f8904b63ca861716d2f.tar.gz) { };
  mdl = import ./default.nix { };
in
pkgs.mkShell {
  buildInputs = [
    mdl
  ];
}

nix-build output

....
patching script interpreter paths in /nix/store/wdr9q2qvn071n2z89kf8z98bb6xrl3bz-ruby2.7.3-chef-utils-17.2.29
checking for references to /build/ in /nix/store/wdr9q2qvn071n2z89kf8z98bb6xrl3bz-ruby2.7.3-chef-utils-17.2.29...
building '/nix/store/5ciz7fakyj3g4ybg8fnnmjmi03wk7jml-ruby2.7.3-tomlrb-2.0.1.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
installing
buildFlags:
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/lib/ruby/gems/2.7.0/bin
  Gem home: /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/lib/ruby/gems/2.7.0
  Plugins dir: /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/lib/ruby/gems/2.7.0/plugins
Successfully installed tomlrb-2.0.1
1 gem installed
/nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/lib/ruby/gems/2.7.0 /build
removed 'cache/tomlrb-2.0.1.gem'
removed directory 'cache'
/build
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1
rewriting symlink /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/nix-support/gem-meta/spec to be relative to /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1
strip is /nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/lib  /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1/bin
patching script interpreter paths in /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1
checking for references to /build/ in /nix/store/bc6lkaqwybwb9vf0mh7d0rd9gxx470ia-ruby2.7.3-tomlrb-2.0.1...
building '/nix/store/hxnl7fnc5qsvilqc5i0vbc31x2n7r7mv-ruby2.7.3-mixlib-shellout-3.2.5.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
installing
buildFlags:
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/lib/ruby/gems/2.7.0/bin
  Gem home: /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/lib/ruby/gems/2.7.0
  Plugins dir: /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/lib/ruby/gems/2.7.0/plugins
Successfully installed mixlib-shellout-3.2.5
1 gem installed
/nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/lib/ruby/gems/2.7.0 /build
removed 'cache/mixlib-shellout-3.2.5.gem'
removed directory 'cache'
/build
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5
rewriting symlink /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/nix-support/gem-meta/spec to be relative to /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5
strip is /nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/lib  /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5/bin
patching script interpreter paths in /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5
checking for references to /build/ in /nix/store/2qhc80c2jx5a0wzz2qj7dmq38ipsd7l7-ruby2.7.3-mixlib-shellout-3.2.5...
building '/nix/store/4qbrhiz8gx29x938kr6nk65mw8dby263-ruby2.7.3-mixlib-config-3.0.9.drv'...
unpacking sources
patching sources
configuring
no configure script, doing nothing
installing
buildFlags:
WARNING:  You build with buildroot.
  Build root: /
  Bin dir: /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/lib/ruby/gems/2.7.0/bin
  Gem home: /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/lib/ruby/gems/2.7.0
  Plugins dir: /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/lib/ruby/gems/2.7.0/plugins
Successfully installed mixlib-config-3.0.9
1 gem installed
/nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/lib/ruby/gems/2.7.0 /build
removed 'cache/mixlib-config-3.0.9.gem'
removed directory 'cache'
/build
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9
rewriting symlink /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/nix-support/gem-meta/spec to be relative to /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9
strip is /nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/lib  /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9/bin
patching script interpreter paths in /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9
checking for references to /build/ in /nix/store/51b49ccz8631fpaajsci8bq58pn7dmmc-ruby2.7.3-mixlib-config-3.0.9...
building '/nix/store/hs14rlg3i666ykxhh2pg8fc7dllpxvh4-ruby2.7.3-mdl-0.11.0.drv'...
unpacking sources
unpacking source archive /nix/store/crsffq7wp4d3cy8a1khjgcn5bzadv3fp-markdownlint-d4722e8
source root is markdownlint-d4722e8
patching sources
configuring
no configure script, doing nothing
building
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
Initialized empty Git repository in /build/markdownlint-d4722e8/.git/
installing
buildFlags:
`/homeless-shelter` is not a directory.
Bundler will use `/build/bundler20210626-24-103kcgb24' as your home directory temporarily.
Using mdl 0.11.0 from https://github.com/markdownlint/markdownlint.git (at d4722e8@d4722e8)
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0
rewriting symlink /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/nix-support/gem-meta/spec to be relative to /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0
strip is /nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/lib  /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/bin
patching script interpreter paths in /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0
/nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/lib/ruby/gems/2.7.0/bundler/gems/markdownlint-d4722e894306/scripts/run_rubocop.sh: interpreter directive changed from "#!/bin/bash" to "/nix/store/kxj6cblcsd1qcbbxlmbswwrn89zcmgd6-bash-4.4-p23/bin/bash"
/nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/lib/ruby/gems/2.7.0/bundler/gems/markdownlint-d4722e894306/bin/mdl: interpreter directive changed from "#!/usr/bin/env ruby" to "/nix/store/asz43r0cbqphhkbax7fdfcn6n35wahy7-ruby-2.7.3/bin/ruby"
/nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/lib/ruby/gems/2.7.0/bundler/gems/markdownlint-d4722e894306/tools/view_markdown.rb: interpreter directive changed from "#!/usr/bin/env ruby" to "/nix/store/asz43r0cbqphhkbax7fdfcn6n35wahy7-ruby-2.7.3/bin/ruby"
/nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/lib/ruby/gems/2.7.0/bundler/gems/markdownlint-d4722e894306/tools/test_location.rb: interpreter directive changed from "#!/usr/bin/env ruby" to "/nix/store/asz43r0cbqphhkbax7fdfcn6n35wahy7-ruby-2.7.3/bin/ruby"
/nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0/lib/ruby/gems/2.7.0/bin/mdl: interpreter directive changed from "#! ruby" to "/nix/store/asz43r0cbqphhkbax7fdfcn6n35wahy7-ruby-2.7.3/bin/ruby"
checking for references to /build/ in /nix/store/9cgi2kdjkfwpyqgxbmv6h5f8wic21bma-ruby2.7.3-mdl-0.11.0...
building '/nix/store/apd5vsbpdrqay9kgnksdmggxgqybj5nv-mdl-0.11.0.drv'...
created 26 symlinks in user environment
building '/nix/store/xg6ldp4bms6hba7iy5k7yimpagii7xk1-mdl-0.11.0.drv'...
find: '/nix/store/qp8dx9s5sak4mvw79dzz1f91a5dn8zac-mdl-0.11.0/lib/ruby/gems/2.7.0/gems/mdl-0.11.0': No such file or directory
error: builder for '/nix/store/xg6ldp4bms6hba7iy5k7yimpagii7xk1-mdl-0.11.0.drv' failed with exit code 1

To Reproduce

Run nix-build with the configuration posted above.

Expected behavior
The package should build like it does when the gem is pointed to a released version.

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.72-microsoft-standard-WSL2, Ubuntu, 20.04.1 LTS (Focal Fossa)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4pre20210601_5985b8b`
 - channels(dominik): `"home-manager, nixpkgs-21.11pre295670.189a1368878"`
 - nixpkgs: `/home/dominik/.nix-defexpr/channels/nixpkgs`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[ymarkus]

Can confirm this, I have the same issue trying to package https://github.com/castwide/readapt

[PuercoPop]

I ran into this issue today. Both in my scenario and the recipe above the issue happens bundlerApp tries to install the man pages of the gem. The directory find is call with doesn't exist. While I don't know the proper way to fix bundlerApp, a work around is to set the installManpages attributes to false. ej.

{ pkgs ? import (fetchTarball https://github.com/NixOS/nixpkgs/archive/ff5da29a3ffc3b2a37386f8904b63ca861716d2f.tar.gz) { }
}:

pkgs.bundlerApp
{
  pname = "mdl";
  gemdir = ./.;
  exes = [ "mdl" ];
  installManpages = false;
}

[Animeshz]

I'm having issue as well, is there any temporal fix atleast? Its annoying, and making my setup not-so-reproducible, I'm bundling puppet and it says:

find: '/nix/store/wq6bbkdak5ir3ycv6d2dfjhp3csl3c9r-puppet-7.26.0/lib/ruby/gems/3.1.0/gems/puppet-7.26.0': No such file or directory

Its actually been created with rev: at /nix/store/21n7lgbhyahwg4agxgwnf0h9i4b4xvxl-ruby3.1.4-puppet-7.26.0/lib/ruby/gems/puppet-5e6faed3498a and haven't been copied over.

[Animeshz]

Seems like you're right @PuercoPop not installing manpages did somehow fix it, but not sure how you figured out...

What I find post trying that is that the problem resides exactly in these lines:

nixpkgs/pkgs/development/ruby-modules/bundler-app/default.nix

Lines 68 to 73 in 278f364

	${lib.optionalString installManpages ''
	for section in {1..9}; do
	mandir="$out/share/man/man$section"
	find -L ${basicEnv}/${ruby.gemPath}/gems/${basicEnv.name} \( -wholename "*/man/*.$section" -o -wholename "*/man/man$section/*.$section" \) -print -execdir mkdir -p $mandir \; -execdir cp '{}' $mandir \;
	done
	''}

[PuercoPop]

@Animeshz I figured out the workaround by reading the error from nix log path to drv. Seeing the call to find being the error and looking for the expression responsible for making the call. Found it was the the installManpages option as you linked above.

However I'm not familiar enough with the process to know what is the proper way to fix this. It seems the reason it fails is because in ${basicEnv}/${ruby.gemPath}/gems/ only has the dependencies of the gem we are building, not the gem itself. Should the man page installation happen in a latter phase? Should it look for the gem man pages in a different place? In what phase it is running?

[Animeshz]

@PuercoPop @ymarkus @dsander

I started a PR for the fix, can you please try building packages from this PR branch?