Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EBUSY on npm install because of rewrite of mounted package.json #28

Open
denengineer opened this issue Jul 18, 2018 · 17 comments
Open

EBUSY on npm install because of rewrite of mounted package.json #28

denengineer opened this issue Jul 18, 2018 · 17 comments

Comments

@denengineer
Copy link

denengineer commented Jul 18, 2018

Hi! Thanks for your work of gathering all this stuff in one place!

But how do you managed to make npm install work with direct package.json mounts? I get this errors every time:

 resource busy or locked, rename '/opt/package.json.3249071875' -> '/opt/package.json'

because npm tries to rewrite file from scratch rather than make atomic writes.

@BretFisher
Copy link
Owner

BretFisher commented Jul 18, 2018

You'll need to describe your setup. Host, OS and docker versions, etc.

@denengineer
Copy link
Author

denengineer commented Jul 19, 2018

Windows 10, Docker is 18.03.1-ce and npm v6.1.0 on top of node alpine image.

To reproduce this I think you will need to just mount package.json

volumes:
      - ./src/package.json:/opt/package.json

and make npm install.

My main thought that this is related to docker mechanism of mounting single files and npm mechanism of replacing the whole package.json instead of modifying it. So I assume this is will appear on any version of docker since file mounting enabled and any version of npm.

Maybe it's related to host and on Linux docker volumes work another way, because npm works in the same way everywhere if I remember right.

@BretFisher
Copy link
Owner

BretFisher commented Jul 24, 2018

Did you look around for any other windows users having this issue?

Are you using Docker for Windows (Hyper-V) or Docker Toolbox (Virtualbox)?

@denengineer
Copy link
Author

denengineer commented Jul 24, 2018

Yep, they have same problems and just mounting folders, not files. I'm on Hyper-V. So you are using this conf on Linux and do not have any problems or you not have problems with Windows too?

@BretFisher
Copy link
Owner

BretFisher commented Jul 24, 2018

I'm on macOS and no problems last I checked. I knew it would be an issue on Linux, but figured the SMB sharing on Windows would work around the issue like it does on Mac. Sucks that npm replaces the file.

@denengineer
Copy link
Author

denengineer commented Jul 28, 2018

Hmm, can you please describe what do you mean by SMB sharing in this case? Even some link with the direction where to dig would be awesome :)

@BretFisher
Copy link
Owner

BretFisher commented Jul 29, 2018

SMB ("windows file sharing") is the protocol that Windows and the Linux VM running Docker use to share your host code when bind-mounting.

@denengineer
Copy link
Author

denengineer commented Jul 29, 2018

Hm, seems like in Win it doesn't solve the problem. Maybe it will be great to add comments in compose file for this Windows case?

@adams-family
Copy link

adams-family commented Sep 7, 2019

I'm can reproduce the same problem on Debian Linux 9 however there is a trick, please see below:

Steps to reproduce:

  1. Git clone this repository

  2. docker-compose up

  3. Break it with Ctrl-C one it's up

  4. Execute the following:

docker-compose run node npm install moment --save

Result:

npm WARN checkPermissions Missing write access to /opt/node_app/app/node_modules

  1. Execute the following:

docker-compose run node sh -c "cd /opt/node_app; npm install moment --save"

Result:

npm WARN saveError EBUSY: resource busy or locked, rename '/opt/node_app/package.json.3920881860' -> '/opt/node_app/package.json'
npm WARN saveError EBUSY: resource busy or locked, rename '/opt/node_app/package-lock.json.3269793734' -> '/opt/node_app/package-lock.json'

@d3ce1t
Copy link

d3ce1t commented Sep 20, 2019

I have the same problem. I'm mounting package.json from host to a docker container. I think the problem is npm install creates a modified copy of package.json named package.json.SOMETHING. And when it comes the time to save it as package.json, it is just renaming the file instead of copying.

@adams-family
Copy link

adams-family commented Sep 22, 2019

@d3ce1t Yes I'm exactly on the same opinion. Until there is an option in npm to overwrite package json (instead of creating a copy and renaming it) I believe that this approach won't be working anymore :(

@frenzymind
Copy link

frenzymind commented Dec 25, 2019

Meet the same issue.
I also find this answer: https://stackoverflow.com/questions/48960320/locked-package-json-files-in-docker-container-using-docker-compose/58880425#58880425
I try play with links as described, but I don't get success on it.
May be some one else try to do it ? May be I do something wrong.

@andrey-hohlov
Copy link

andrey-hohlov commented Jan 10, 2020

Unanswered issue in npm repository npm/npm#16615

UPD: to make it works need to mount folder with package.json file, not directly package.json

@andreafspeziale
Copy link

andreafspeziale commented Feb 11, 2020

I created the following example to highlight what I can't make it work:

I don't have any local node_modules. I'm running everything through Docker.

As soon as I bootstrap the project with $ docker-compose up everything looks great.
If I try to change the Hello World! sentence in index.js nodemon reloads the server as expected.

But as soon as I want to add a dependency like lodash while developing and while my container is up and running the package.json is not updated.

As explained in this repo (node-docker-good-defaults) I'm using $ docker-compose exec -w /opt/node_app node npm install --save lodash to install the new dependency and reflect the changes on my host project. Unfortunately it isn't working.

If I'm not able to develop locally without having NodeJS installed in the host machine I don't really see the point of using in the docker-compose (at least for development) a service which serves my app.

It makes sense for staging and production since it will use the packaged app image and run everything without mounting anything and hot-reloading anything.

Any hints? Am I missing something also in my assumptions? (@BretFisher moved here the discussion)

@andreafspeziale
Copy link

andreafspeziale commented Feb 11, 2020

I have updated the upper project code in order to develop with NO NodeJS installed on host machine. It should work. But in the Bret setup the EBUSY problem still exists.

@BretFisher
Copy link
Owner

BretFisher commented Feb 11, 2020

This repo is just one example of using docker-compose. In my Docker for Node.js course, I cover two Solutions for designing the compose file for local development and students have thought of several other Solutions along the way.

I also talk about my two "Solutions" at DockerCon. Those slides start here in my talk.

This repo is more aligned with Solution 2 below, but you might consider Solution 1 if the package.json bind-mount issue is a blocker for you.

Solution 1:

  • Don't move node_modules up a directory. Leave everything in the standard single dir and do a single bind-mount in the compose file.
  • Before doing a docker-compose up the first time, we'll need to do a docker-compose run <app> npm install to see the node_modules that are bind-mounted to the host. This is the main disadvantage of this Solution.
  • This also means on macOS/Windows that you can't "dual develop" by sometimes using native host Node.js and sometimes using Docker-based Node.js because the binaries in node_modules (if you have them) would be incompatible. This is a disadvantage if you don't always use docker to run this code.

Solution 2:

  • Similar to this repo's setup, you would move node_modules and package*.json up a level in the file path, so they can be separated from host node_modules. This adds complexity to the setup but also provides the flexibility of developing with a different node_modules on the host than what is in the container.

@petrprikryl
Copy link

petrprikryl commented Aug 3, 2020

Isn't there way using yarn's --modules-folder? Assuming switch from NPM to yarn.
https://stackoverflow.com/a/53998408/1763888
https://classic.yarnpkg.com/en/docs/cli/install#toc-yarn-install-modules-folder

Then you shouldn't need bind mounts on package*.json because all yarn commands should be feasible from project directory directly, shouldn't?

Disclaimer: I haven't tested it yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants