Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: file 'REPEAT' was not found in the Nix search path #163374

Closed
stackshadow opened this issue Mar 8, 2022 · 23 comments · Fixed by NixOS/nix#6296
Closed

error: file 'REPEAT' was not found in the Nix search path #163374

stackshadow opened this issue Mar 8, 2022 · 23 comments · Fixed by NixOS/nix#6296
Assignees
Labels
0.kind: bug Something is broken

Comments

@stackshadow
Copy link
Contributor

Describe the bug

Can not install an simple expression with nix-env. I always get the error
error: file 'REPEAT' was not found in the Nix search path (add it using $NIX_PATH or -I)

Steps To Reproduce

Steps to reproduce the behavior:

Create two files:

Dockerfile

FROM nixos/nix:latest AS base

RUN nix-channel --update

RUN nix-env -i gnumake gnupg

COPY ./ /usr/src/broken/

RUN cd /usr/src/broken/ && \
    nix-env -iA package -f expression.nix 

expression.nix

{ system ? builtins.currentSystem }:
let
  pkgs = import <nixpkgs> { inherit system; };
  lib = pkgs.lib;
  inherit (pkgs) buildGoModule fetchFromGitHub;
  inherit (lib) sourceByRegex;

in
{
  package = pkgs.writers.writeBash "/bin/some-clean" ''
    echo "Dummy"
  '';
}

And run

docker build .

Expected behavior

The simple script is installed in the env :)

Metadata

 - system: `"aarch64-linux"`
 - host os: `Linux 5.15.25`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.7.0`
 - channels(root): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
@stackshadow stackshadow added the 0.kind: bug Something is broken label Mar 8, 2022
@bonds
Copy link

bonds commented Mar 9, 2022

The full error for me:

scott@Scotts-MacBook-Air ~ % nix-env --install wget
installing 'wget-1.21.2'
error: file 'REPEAT' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at /nix/store/b8d7603nslma9qfn00qh8gmjhgjpyxd7-env-manifest.nix:1:1936:

and that file says:

scott@Scotts-MacBook-Air ~ % cat /nix/store/b8d7603nslma9qfn00qh8gmjhgjpyxd7-env-manifest.nix
[ { meta = { available = true; broken = false; description = "The Glasgow Haskell Compiler"; homepage = "http://haskell.org/ghc"; insecure = false; license = { deprecated = false; free = true; fullName = "BSD 3-clause \"New\" or \"Revised\" License"; redistributable = true; shortName = "bsd3"; spdxId = "BSD-3-Clause"; url = "https://spdx.org/licenses/BSD-3-Clause.html"; }; maintainers = [ { email = "guillaum.bouchard@gmail.com"; github = "guibou"; githubId = 9705357; name = "Guillaume Bouchard"; } { email = "cdep.illabout@gmail.com"; github = "cdepillabout"; githubId = 64804; matrix = "@cdepillabout:matrix.org"; name = "Dennis Gosnell"; } { email = "nix@monoid.al"; github = "expipiplus1"; githubId = 857308; keys = [ { fingerprint = "FC1D 3E4F CBCA 80DF E870  6397 C811 6E3A 0C1C A76A"; longkeyid = "rsa4096/0xC8116E3A0C1CA76A"; } ]; matrix = "@ellie:monoid.al"; name = "Ellie Hermaszewska"; } { email = "malte.brandy@maralorn.de"; github = "maralorn"; githubId = 1651325; matrix = "@maralorn:maralorn.de"; name = "Malte Brandy"; } { email = "sternenseemann@systemli.org"; github = "sternenseemann"; githubId = 3154475; name = "Lukas Epple"; } ]; name = "ghc-8.10.7"; outputsToInstall = [ "out" ]; platforms = [ "aarch64-darwin" "aarch64-linux" "armv7l-linux" "i686-linux" "x86_64-darwin" "x86_64-linux" ]; position = "/nix/store/ccg2bflff3j9kcghnbfw168r4v6h5ci1-nixpkgs/nixpkgs/pkgs/development/compilers/ghc/8.10.7.nix:375"; timeout = 86400; unfree = false; unsupported = false; }; name = "ghc-8.10.7"; out = { outPath = "/nix/store/crfkr9bbd4qa79r40nph08ligdyl33ns-ghc-8.10.7"; }; outPath = "/nix/store/crfkr9bbd4qa79r40nph08ligdyl33ns-ghc-8.10.7"; outputs = [ "out" ]; system = "aarch64-darwin"; type = "derivation"; } { meta = { available = true; broken = false; description = "The command-line interface for Cabal and Hackage"; homepage = "https://hackage.haskell.org/package/cabal-install"; insecure = false; license = <REPEAT>; maintainers = [ { email = "simons@cryp.to"; github = "peti"; githubId = 28323; name = "Peter Simons"; } ]; name = "cabal-install-3.6.2.0"; outputsToInstall = [ "out" ]; platforms = [ "i686-cygwin" "x86_64-cygwin" "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" "i686-freebsd" "x86_64-freebsd" "aarch64-genode" "i686-genode" "x86_64-genode" "x86_64-solaris" "js-ghcjs" "aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7a-linux" "armv7l-linux" "i686-linux" "m68k-linux" "mipsel-linux" "powerpc64-linux" "powerpc64le-linux" "riscv32-linux" "riscv64-linux" "s390-linux" "s390x-linux" "x86_64-linux" "mmix-mmixware" "aarch64-netbsd" "armv6l-netbsd" "armv7a-netbsd" "armv7l-netbsd" "i686-netbsd" "m68k-netbsd" "mipsel-netbsd" "powerpc-netbsd" "riscv32-netbsd" "riscv64-netbsd" "x86_64-netbsd" "aarch64_be-none" "aarch64-none" "arm-none" "armv6l-none" "avr-none" "i686-none" "msp430-none" "or1k-none" "m68k-none" "powerpc-none" "powerpcle-none" "riscv32-none" "riscv64-none" "s390-none" "s390x-none" "vc4-none" "x86_64-none" "i686-openbsd" "x86_64-openbsd" "x86_64-redox" "wasm64-wasi" "wasm32-wasi" "x86_64-windows" "i686-windows" ]; position = "/nix/store/ccg2bflff3j9kcghnbfw168r4v6h5ci1-nixpkgs/nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix:50293"; unfree = false; unsupported = false; }; name = "cabal-install-3.6.2.0"; out = { outPath = "/nix/store/69vidhq89j1vz0l08v4x3zaqsdrqiq5k-cabal-install-3.6.2.0"; }; outPath = "/nix/store/69vidhq89j1vz0l08v4x3zaqsdrqiq5k-cabal-install-3.6.2.0"; outputs = [ "out" ]; system = "aarch64-darwin"; type = "derivation"; } { meta = { available = true; broken = false; description = "A network diagnostics tool"; homepage = "https://www.bitwizard.nl/mtr/"; insecure = false; license = { deprecated = true; free = true; fullName = "GNU General Public License v2.0"; redistributable = true; shortName = "gpl2"; spdxId = "GPL-2.0"; url = "https://spdx.org/licenses/GPL-2.0.html"; }; maintainers = [ { email = "koral@mailoo.org"; github = "k0ral"; githubId = 524268; name = "Koral"; } { email = "orivej@gmx.fr"; github = "orivej"; githubId = 101514; name = "Orivej Desh"; } { email = "7c6f434c@mail.ru"; github = "7c6f434c"; githubId = 1891350; name = "Michael Raskin"; } { email = "mail@glob.in"; github = "globin"; githubId = 1447245; name = "Robin Gloster"; } ]; name = "mtr-0.95"; outputsToInstall = [ "out" ]; platforms = [ "i686-cygwin" "x86_64-cygwin" "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" "i686-freebsd" "x86_64-freebsd" "x86_64-solaris" "aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7a-linux" "armv7l-linux" "i686-linux" "m68k-linux" "mipsel-linux" "powerpc64-linux" "powerpc64le-linux" "riscv32-linux" "riscv64-linux" "s390-linux" "s390x-linux" "x86_64-linux" "aarch64-netbsd" "armv6l-netbsd" "armv7a-netbsd" "armv7l-netbsd" "i686-netbsd" "m68k-netbsd" "mipsel-netbsd" "powerpc-netbsd" "riscv32-netbsd" "riscv64-netbsd" "x86_64-netbsd" "i686-openbsd" "x86_64-openbsd" "x86_64-redox" ]; position = "/nix/store/ccg2bflff3j9kcghnbfw168r4v6h5ci1-nixpkgs/nixpkgs/pkgs/tools/networking/mtr/default.nix:47"; unfree = false; unsupported = false; }; name = "mtr-0.95"; out = { outPath = "/nix/store/f9iya5c9i8fdljsxjnxp2907f89fvxsp-mtr-0.95"; }; outPath = "/nix/store/f9iya5c9i8fdljsxjnxp2907f89fvxsp-mtr-0.95"; outputs = [ "out" ]; system = "aarch64-darwin"; type = "derivation"; } { meta = { available = true; broken = false; changelog = "https://github.com/htop-dev/htop/blob/3.1.2/ChangeLog"; description = "An interactive process viewer for Linux"; homepage = "https://htop.dev"; insecure = false; license = { deprecated = false; free = true; fullName = "GNU General Public License v2.0 only"; redistributable = true; shortName = "gpl2Only"; spdxId = "GPL-2.0-only"; url = "https://spdx.org/licenses/GPL-2.0-only.html"; }; maintainers = [ { email = "rob.vermaas@gmail.com"; github = "rbvermaa"; githubId = 353885; name = "Rob Vermaas"; } { email = "ricky@elrod.me"; github = "relrod"; githubId = 43930; name = "Ricky Elrod"; } ]; name = "htop-3.1.2"; outputsToInstall = [ "out" ]; platforms = [ "i686-cygwin" "x86_64-cygwin" "x86_64-darwin" "i686-darwin" "aarch64-darwin" "armv7a-darwin" "i686-freebsd" "x86_64-freebsd" "aarch64-genode" "i686-genode" "x86_64-genode" "x86_64-solaris" "js-ghcjs" "aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7a-linux" "armv7l-linux" "i686-linux" "m68k-linux" "mipsel-linux" "powerpc64-linux" "powerpc64le-linux" "riscv32-linux" "riscv64-linux" "s390-linux" "s390x-linux" "x86_64-linux" "mmix-mmixware" "aarch64-netbsd" "armv6l-netbsd" "armv7a-netbsd" "armv7l-netbsd" "i686-netbsd" "m68k-netbsd" "mipsel-netbsd" "powerpc-netbsd" "riscv32-netbsd" "riscv64-netbsd" "x86_64-netbsd" "aarch64_be-none" "aarch64-none" "arm-none" "armv6l-none" "avr-none" "i686-none" "msp430-none" "or1k-none" "m68k-none" "powerpc-none" "powerpcle-none" "riscv32-none" "riscv64-none" "s390-none" "s390x-none" "vc4-none" "x86_64-none" "i686-openbsd" "x86_64-openbsd" "x86_64-redox" "wasm64-wasi" "wasm32-wasi" "x86_64-windows" "i686-windows" ]; position = "/nix/store/ccg2bflff3j9kcghnbfw168r4v6h5ci1-nixpkgs/nixpkgs/pkgs/tools/system/htop/default.nix:45"; unfree = false; unsupported = false; }; name = "htop-3.1.2"; out = { outPath = "/nix/store/pyrk5rkdx5aqf06ghv5bz2mp66q1ja8x-htop-3.1.2"; }; outPath = "/nix/store/pyrk5rkdx5aqf06ghv5bz2mp66q1ja8x-htop-3.1.2"; outputs = [ "out" ]; system = "aarch64-darwin"; type = "derivation"; } ]%

I note that REPEAT shows up only once:

license = <REPEAT>

And if I replace <REPEAT> with "REPEAT" in that file it fixes the problem and nix-env --install works again.

@akamaus
Copy link
Contributor

akamaus commented Mar 9, 2022

Got similar problem after executing nix upgrade-nix.
The quick hack proposed by @bonds above worked for me too.

@maksar
Copy link
Contributor

maksar commented Mar 10, 2022

Might be related to this piece of code NixOS/nix@e9c04c3

That is the only thing I found with "" keywords in it.

@Artturin
Copy link
Member

@edolstra

@ppenguin
Copy link
Contributor

@bonds on nixos obviously all files under nix/... are read-only, so how to change the offending <REPEAT>?
(mind you, this occurred after trying to install from a local clone of nixpkgs, and nowhere in the source tree does this word occur).

@Artturin
Copy link
Member

Don't use nix-env

@corngood
Copy link
Contributor

corngood commented Mar 16, 2022

@ppenguin I was able to get around this by adding ~/empty.nix with just "" in it and doing:

nix-env -I REPEAT=$HOME/empty.nix

or

NIX_PATH=$NIX_PATH:REPEAT=$HOME/empty.nix ...

Also worth noting that after I did this and upgraded my user profile and system, I no longer have <REPEAT> in my manifest, and nix-env is back to working like normal. I'm not 100% sure what fixed it.

@bonds
Copy link

bonds commented Mar 16, 2022

@bonds on nixos obviously all files under nix/... are read-only, so how to change the offending ?

Dunno...I just used vi and ignored the warning that the file was read-only. root can do what he wants. ;)

@ppenguin
Copy link
Contributor

@bonds on nixos obviously all files under nix/... are read-only, so how to change the offending ?

Dunno...I just used vi and ignored the warning that the file was read-only. root can do what he wants. ;)

@bonds I take it you're not on nixos, there the nix-store itself is read-only (also for root), but the solution of @corngood works like a charm and is probably a more "idiomatic" workaround.

@bonds
Copy link

bonds commented Mar 17, 2022 via email

@jackpot51
Copy link

As a minimal fix for this, I ran the following and after that nix-env was operational:

env NIX_PATH="REPEAT=/dev/null" nix-env --upgrade

@sternenseemann
Copy link
Member

Let's reopen this until the fix hits nixpkgs master at least.

@stackshadow
Copy link
Contributor Author

Wow, many thanks for your help 💯 really great community help here ❤️ 💟 😍 totally awesome 😄
I will add this to my docker-build-stuff and will test it.

As a minimal fix for this, I ran the following and after that nix-env was operational:

env NIX_PATH="REPEAT=/dev/null" nix-env --upgrade

@mgudemann
Copy link

mgudemann commented Mar 31, 2022

Don't use nix-env

@Artturin what should I use instead?

@eddyb
Copy link
Contributor

eddyb commented Apr 2, 2022

@edolstra Correct me if I'm wrong, but my understanding of NixOS/nix#6296 is that it fixes generating the env-manifest.nix, but if I already have an env-manifest.nix (linked from ~/.nix-profile/manifest.nix) with <REPEAT>s in it, there's no way to recover the data that got hidden with <REPEAT>s?

So anyone who used nix-env in the few weeks in which it was broken, now has to use NIX_PATH="REPEAT=/dev/null" nix-env --upgrade, with no other workaround?

EDIT: also, the workaround worked for me, and I updated over a week ago (22.05pre362644.1ec61dd4167 to be exact) - I believe this means that I already have a fixed Nix, which is why the workaround worked, but there's no reason to keep this issue open unless some <REPEAT> special-casing when reading the manifest is added.

EDIT2: disregard above, that was quite far off from the right understanding, see #163374 (comment).

@sternenseemann
Copy link
Member

The fix is neither released nor in nixpkgs AFAICT.

@eddyb

This comment was marked as outdated.

@sternenseemann
Copy link
Member

The repl has used this notation already prior to this patch which is not part of any Nix release nor applied as a patch in nixpkgs.

@eddyb
Copy link
Contributor

eddyb commented Apr 3, 2022

Alright, I was quite wrong, but also after spending a few hours being confused it makes sense now.

(click to open some ramblings)

So I decided to check the most direct way I can thing of:

$ nix build $(nix show-derivation $(which nix-env) | jq -r 'values[].env.src')
$ grep -n -C2 -R '<REPEAT>' result/
result/src/libexpr/eval.cc-127-    case tAttrs: {
result/src/libexpr/eval.cc-128-        if (!v.attrs->empty() && !seen.insert(v.attrs).second)
result/src/libexpr/eval.cc:129:            str << "<REPEAT>";
result/src/libexpr/eval.cc-130-        else {
result/src/libexpr/eval.cc-131-            str << "{ ";
--
result/src/libexpr/eval.cc-143-    case tListN:
result/src/libexpr/eval.cc-144-        if (v.listSize() && !seen.insert(v.listElems()).second)
result/src/libexpr/eval.cc:145:            str << "<REPEAT>";
result/src/libexpr/eval.cc-146-        else {
result/src/libexpr/eval.cc-147-            str << "[ ";

But the nix-env in question doesn't seem to generate <REPEAT>s into manifest.nix... anymore?
I'm not sure I understand what happened.

After every system upgrade, I run nix-env -u --always (or at least used to, before switching to home-manager the other day), and that succeeded. Presumably that left me with the broken manifest.nix with <REPEAT>s in it.

But why is everything I did since NIX_PATH="REPEAT=/dev/null" nix-env --upgrade not causing the bug again?

Best guess right now is the nix::Values aren't deduplicated in memory, so you can only trigger this with multiple installs/upgrades?

In short, I didn't notice the original repro had two installed packages and apparently the only multi-package operation I ever did was the bulk upgrade (after the system-level nixos-rebuild switch --upgrade).

After the move to home-manager it's a bit annoying to mess with nix-env but I was able to use NIX_PROFILE to demonstrate the problem still being in my nix-env:

$ cd $(mktemp -d)
$ ln -s . profile
$ NIX_PROFILE=$PWD/profile nix-env -iA nixos.gnumake nixos.gnupg
installing 'gnumake-4.3'
installing 'gnupg-2.3.4'
$ grep '<REPEAT>' ./profile/manifest.nix # most of the output replaced with "..." for brevity
... license = <REPEAT>; ...
... maintainers = [ ... <REPEAT> ]; ...
... platforms = <REPEAT>; ...
$ NIX_PROFILE=$PWD/profile nix-env -iA nixos.hello
installing 'hello-2.12'
error: file 'REPEAT' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at /nix/store/a8fqrf8j3sv7fngmx0zzix22508fggrd-env-manifest.nix:1:2758:

@jbg
Copy link
Contributor

jbg commented Apr 16, 2022

I use nix on macOS. I have https://github.com/nixos/nixpkgs/archive/master.tar.gz nixpkgs in ~/.nix-channels.

For a month or so now, I get this error whenever running nix-env -u after nix-channel --update:

error: file 'REPEAT' was not found in the Nix search path (add it using $NIX_PATH or -I)

I know from the above that I can work around it with NIX_PATH="REPEAT=/dev/null", which does work, but it's not clear from the above comments what I'm supposed to do to fix it permanently?

@animaldaydream
Copy link

I use nix on macOS. I have https://github.com/nixos/nixpkgs/archive/master.tar.gz nixpkgs in ~/.nix-channels.

For a month or so now, I get this error whenever running nix-env -u after nix-channel --update:

error: file 'REPEAT' was not found in the Nix search path (add it using $NIX_PATH or -I)

I know from the above that I can work around it with NIX_PATH="REPEAT=/dev/null", which does work, but it's not clear from the above comments what I'm supposed to do to fix it permanently?

Upgrading in a root shell fixed it for me. I also use the unstable channel and am on Fedora 35 however.

I reinstalled Nix and rebooted twice just to fix this issue. I was pretty sure I had messed up something, and obviously I had the freshest packages on my user profile, so I don't understand what I did right to fix it.

@iFreilicht
Copy link
Contributor

@jbg what version of nix are you running? I had the same issue and was able to fix it by running a stable version of nix-env once:

Problem example

$ nix-env -iA nixpkgs.nixpkgs-fmt
replacing old 'nixpkgs-fmt-1.2.0'
installing 'nixpkgs-fmt-1.2.0'
error: file 'REPEAT' was not found in the Nix search path (add it using $NIX_PATH or -I)

       at /nix/store/7sh64rs37jggd00zlxxx1afav1yn6f60-env-manifest.nix:1:4991:

Solution

Install stable version of nix:

$ nix-channel --add https://nixos.org/channels/nixpkgs-21.11-darwin stable
$ nix-channel --update
unpacking channels...
$ nix-channel --list
nixpkgs https://nixos.org/channels/nixpkgs-unstable
stable https://nixos.org/channels/nixpkgs-21.11-darwin
$ env NIX_PATH="REPEAT=/dev/null" nix-env -iA stable.nix   
installing 'nix-2.3.16'
# output omitted ...
building '/nix/store/75vfjxsy1clpm53y7v68pgf0b81lsypv-user-environment.drv'...
$ nix-env --version
nix-env (Nix) 2.7.0
$ ~/.nix-profile/bin/nix-env --version
nix-env (Nix) 2.3.16

Install a single derivation with the stable version of nixpkgs. After that, the old version works as well.

$ ~/.nix-profile/bin/nix-env -iA nixpkgs.nixpkgs-fmt
replacing old 'nixpkgs-fmt-1.2.0'
installing 'nixpkgs-fmt-1.2.0'
$ nix-env -iA nixpkgs.nixpkgs-fmt
replacing old 'nixpkgs-fmt-1.2.0'
installing 'nixpkgs-fmt-1.2.0'

Afterwards, you can even nix-env -e nix to remove the stable version and it still works. I'm not sure why.

@eddyb
Copy link
Contributor

eddyb commented Apr 19, 2022

Afterwards, you can even nix-env -e nix to remove the stable version and it still works. I'm not sure why.

The bug is only triggered when installing/upgrading multiple packages, which share some attributes (the sharing comes from evaluating <nixpkgs> once in the same nix-env execution, there's no deduplication with what's already in the profile manifest).
(that was the source of my confusion in #163374 (comment) as well)

So you only need the workaround once (to "repair" the profile manifest), then either:

  1. upgrade to a version of Nix without the bug
  2. don't install/upgrade multiple packages at once
    • if you must, you can use the workaround again afterwards

yayayayaka pushed a commit to yayayayaka/nixpkgs that referenced this issue May 1, 2022
limeytexan pushed a commit to flox/nixpkgs that referenced this issue May 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken
Projects
None yet
Development

Successfully merging a pull request may close this issue.