ENOENT: no such file or directory, copyfile '/project/.pnpm-store/... -> '/project/node_modules/...

[o-az]

pnpm version: 7.18.2

Code to reproduce the issue: (running vscode devcontainer)

# .devcontainer/Dockerfile

ARG VARIANT="bullseye"
FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:0-${VARIANT}

# [Optional] Install additional OS packages.
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
  && apt-get --yes install --no-install-recommends apt-utils dialog 2>&1 \
  #
  # Verify git, process tools, lsb-release (common in install instructions for CLIs) installed
  && apt-get --yes install curl git iproute2 procps lsb-release \
  #
  # install required packages
  && apt-get --yes install sudo python3 && ln -sf /usr/bin/python3 /usr/bin/python

# [Choice] Node.js version: none, lts/*, 16, 14, 12, 10
ARG NODE_VERSION="lts/*"
RUN if [ "${NODE_VERSION}" != "none" ]; then umask 0002 \
  && sh -c ". /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION}" 2>&1; fi

RUN sh -c ". /usr/local/share/nvm/nvm.sh && nvm alias default ${NODE_VERSION} \
  && npm install --global pnpm && pnpm setup && source /root/.bashrc" 2>&1 \
  && sh -c "pnpm add --global turbo" 2>&1

// .devcontainer/devcontainer.json
{
  "name": "myapp",
  "build": {
    "dockerfile": "Dockerfile",
    "args": {
      "devcontainercli": "true",
      "VARIANT": "bullseye",
      "NODE_VERSION": "18.12.1"
    }
  },
  "runArgs": [
    "--cap-add=SYS_PTRACE",
    "--security-opt",
    "seccomp=unconfined"
  ],
  "postCreateCommand": "node --version && pnpm --version",
  "features": {
    "git": "latest",
    "github-cli": "latest",
    "docker-in-docker": "latest"
  },
  "settings": {
    "typescript.tsdk": "node_modules/typescript/lib"
  }
}

Expected behavior:

It should install packages with no errors.

Actual behavior:

$ pnpm install

Scope: all 12 workspace projects
Downloading registry.npmjs.org/typescript/4.9.4: 11.6 MB/11.6 MB, done
Packages: +816
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /workspaces/myapp/.pnpm-store/v3
  Virtual store is at:             node_modules/.pnpm
 ENOENT  ENOENT: no such file or directory, copyfile '/workspaces/myapp/.pnpm-store/v3/files/4e/e1c88f8c3f4e4cd34cb6c00339bf9d6d036ff4ade3af49e871cc8966b84c729d8b75492acc6413c9a664ac00a57958223ac13c4229da8c62ebe6a53e4f783f' -> '/workspaces/myapp/node_modules/.pnpm/camel-case@4.1.2/node_modules/_tmp_12988_94ced9782f6a201d347acf8754b6bc09/dist/index.spec.d.ts'

pnpm: ENOENT: no such file or directory, copyfile '/workspaces/myapp/.pnpm-store/v3/files/4e/e1c88f8c3f4e4cd34cb6c00339bf9d6d036ff4ade3af49e871cc8966b84c729d8b75492acc6413c9a664ac00a57958223ac13c4229da8c62ebe6a53e4f783f' -> '/workspaces/myapp/node_modules/.pnpm/camel-case@4.1.2/node_modules/_tmp_12988_94ced9782f6a201d347acf8754b6bc09/dist/index.spec.d.ts'
Progress: resolved 863, reused 0, downloaded 813, added 460
Downloading registry.npmjs.org/turbo-linux-arm64/1.6.3: 1.62 MB/5.69 MB

Additional information:

• node -v prints: v18.12.1
• Windows, macOS, or Linux?: I'm on M1 Mac and devcontainer running ubuntu 11

$ cat /etc/*release && node --version && pnpm --version && pnpm store path

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
v18.12.1
7.18.2
/workspaces/myapp/.pnpm-store/v3

[rdss-dheuskel]

that happens on mac, only working fix is to move the store directory outside of the volume mount, or dont use hardlinks.

after that error happens, pnpm store status should give the exact same error as above. removing the store then does nothing.

either use mutagen, if you need the volume mount (i imagine for indexing in vscode that would be neat) or just ditch pnpm (or docker for that matter)

i had some hope when virtiofs went out of beta for docker for desktop, but its still happening here as well. bug was reported multiple times already, they cant fix it.

only difference now is, that until now there existed a file with the same name, but with strange modes (sorry cant remember) set and .exec appended to the name.

this is no longer the case, when you check now, the file exists, so might be a race condition somewhere.

"99 got problems multithreading but aint one"

oh, and another solution is to just run pnpm install until it exits 0 :P
that works but still is a joke.

[o-az]

The store was getting created on the same directory level as the project's node_modules.

Running pnpm config set store-dir /root/.local/share/pnpm/store to change the store directory solved it for me.

# ...
RUN npm install --global pnpm@latest \
  && SHELL=bash pnpm setup \
  && source /root/.bashrc \
  && pnpm config set store-dir /root/.local/share/pnpm/store

[amlcodes]

pnpm in devcontainers is completely unsuable for me, even with @o-az suggestion

[rdss-dheuskel]

the suggestion is basically to use copy, because /root is on a different file system, so no hardlinking. pnpm is just broken

[amlcodes]

the suggestion is basically to use copy, because /root is on a different file system, so no hardlinking. pnpm is just broken

Appreciate the help @rdss-dheuskel! I'm not a Docker guru at all, I'm sure you're more knowledgable, but COPY would bring over potentially GB's worth of data right? Especially in a monorepo. Is there anything officially referencing this issue and the incompatibility of pnpm with docker/devcontainers? Prisma's repo uses pnpm in their devcontainer albeit I believe they are using a custom image with pnpm preinstalled. Definitely a giant obstacle for me atm.

[o-az]

I dunno about COPY. @amlcodes what does your devcontainer.json and Dockerfile look like? I'll try to help

[lalexdotcom]

I also have this kind of messages, only when using VirtioFS on mac Docker. As it's already been told, it looks like a race condition: if running several times, the error message is different each time (even if each time is for a "clean" base, just checked out from repository)...
I tried to switch to osxfs or gRPC FUSE and everything works fine. So I don't know if this kind of issue is due to pnpm or docker (specifically VirtioFS)...

gRPS FUSE with virtualisation framework seem to deliver decent performances, so as it's important for me to work in an isolated env (vscode devcontainer) I'll stick to that waiting for this issue to get resolved (by pnpm or docker)....

We should find a way for @zkochan to get his hands on an M1/M2 computer for him to be able to handle this kind of "mac-only" bug...

[andersk]

I found a very quick reproduction recipe using Docker Desktop for Mac with the file sharing implementation set to gRPC FUSE or VirtioFS but not osxfs (Legacy). Run this from an empty directory:

docker run --rm -v "$(pwd):/vol" -w /vol -t node:alpine corepack pnpm install \
    https://codeload.github.com/postcss/postcss-media-minmax/tar.gz/refs/tags/5.0.0

This fails with an ENOENT error in 89 out of 100 runs with gRPC FUSE, 68 out of 100 runs with VirtioFS, and 0 out of 100 runs with osxfs. I opened docker/for-mac#6787.

[o-az]

This is just an update, not complaining cause I found a short term solution:

I'm using latest pnpm and this issue still present.

[zanminkian]

that happens on mac, only working fix is to move the store directory outside of the volume mount, or dont use hardlinks.

As @rdss-dheuskel says, the reproduced steps are:

• Run docker run -it -v $HOME/projects:/root/projects ubuntu on mac
• And then cd /root/projects && git clone xxxx in the container to clone a simple node project using pnpm.
• Run cd /root/projects/xxx && pnpm i will fail. And print error message like: ENOENT: no such file or directory, copyfile 'xx' -> 'xx'

I hope it can help you to reproduce it 😃 @zkochan

[zkochan]

I wasn't able to reproduce it

root@23853f8e64e4:~/projects/graph-sequencer# pnpm -v
8.4.0
root@23853f8e64e4:~/projects/graph-sequencer# pnpm install
Lockfile is up to date, resolution step is skipped
Packages: +278
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /root/projects/.pnpm-store/v3
  Virtual store is at:             node_modules/.pnpm
Progress: resolved 278, reused 0, downloaded 278, added 278, done

devDependencies:
+ jest 29.5.0

Done in 13s

the suggestion is basically to use copy, because /root is on a different file system, so no hardlinking. pnpm is just broken

That shouldn't be a problem as pnpm automatically falls back to copy if hard link creation fails.

This one does fail but not always:

docker run --rm -v "$(pwd):/vol" -w /vol -t node:alpine corepack pnpm install \
    https://codeload.github.com/postcss/postcss-media-minmax/tar.gz/refs/tags/5.0.0

Also, looks like if I connect to the container and run cp on the failed file, it works.

[zanminkian]

Hi @zkochan. Appreciate your effort on this. Please try steps below to reproduce it. Thanks!

• rm -rf ~/projects && mkdir ~/projects (necessary)
• docker run -it --rm -v $HOME/projects:/root/projects --entrypoint zsh zengmingjian/web-ide
• cd /root/projects
• git clone https://github.com/antfu/eslint-config.git
• cd eslint-config
• pnpm -v
• pnpm i

[zanminkian]

This fails with an ENOENT error in 89 out of 100 runs with gRPC FUSE, 68 out of 100 runs with VirtioFS, and 0 out of 100 runs with osxfs. I opened docker/for-mac#6787.

Inspired by @andersk, setting container file sharing to osxfs implementation solves this issue for me.

[lalexdotcom]

Inspired by @andersk, setting container file sharing to osxfs implementation solves this issue for me.

The thing is that osxfs has bad performance compared to grpc, and even worse compared to virtiofs...

[zanminkian]

The thing is that osxfs has bad performance compared to grpc, and even worse compared to virtiofs...

Yes that gRPC FUSE also solves this issue for me and have a better performance than osxfs.

[innpyt]

Orbstack(Docker Desktop alternative) has only VirtioFS, any working solution for it?

[mnlbox]

I can't install the package below on devContainer node.js 18 and the latest version of PNPM on MacOS because of the same error message. Any plan to solve this issue? It reported around 1 year ago. 😞
https://github.com/uNetworking/uWebSockets.js

[collimarco]

This issue seems related:

novuhq/novu#4238

Can you please fix this?

[darkbasic]

I'm not using docker though I get the same issue on Arch Linux.

[colemickens]

I'm hitting this, on ZFS, on a fast NVME. It's easy to repro, I can delete the pnpm cache (which is stored under ~/.local/share instead of ~/.cache, unfortunate), and then serially run pnpm install in two worktrees for a project and get this error.

[darkbasic]

Damn that's it then, zfs 2.2 on a PCIe4 NVME here.

[Nightvision53]

I'm having the same issue with devcontainers. Is there any workaround for this or we just can't use pnpm with containers ?

[FedericoBiccheddu]

@zkochan I'd like to try to fix this issue. I have no idea on how navigate the codebase; can you share some tips on how/where I can look?

Adding to the issue: I noticed that the problem arise only when installing multiple packages. Installing few of them the installation process succeed.

[joyqi]

Orbstack(Docker Desktop alternative) has only VirtioFS, any working solution for it?

Just use pnpm 8.6.* instead.

[samzuercher]

On latest node.js, npm and pnpm i get a similar error: ENOENT: no such file or directory, rename '\\mac\Home\Downloads\folder\node_modules\.pnpm\nuxt@3.10.3_sass@1.71.1_vite@5.1.4\node_modules\@nuxt\devalue' -> '\\mac\Home\Downloads\folder\node_modules\.pnpm\nuxt@3.10.3_sass@1.71.1_vite@5.1.4\node_modules\@nuxt\.ignored_devalue'

This is on a clean repository, first time install. Already tried to prune, remove cache, reinstall.. It works fine with npm, but not at all with pnpm.

On a Macbook Pro M3 in Windows on Parallels.

To me it either is an issue with the installation (https://stackoverflow.com/questions/77297035/pnpm-install-error-enoent-no-such-file-or-directory) or with mapped directories / the file system.

[almereyda]

Here we were running pnpm in a dev container based on Node 20. There, to avoid permission errors with the process running unprivileged, we had to put the PNPM store into the home directory of the node user instead of root.

We run this command, after PNPM has been installed globally with NPM under the unprivileged node user:

pnpm config set store-dir /home/node/.local/share/pnpm/store