Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

npm install from github fails with access permissions when using cache #263

Closed
ringods opened this issue Aug 13, 2016 · 6 comments

Comments

Projects
None yet
5 participants
@ringods
Copy link

commented Aug 13, 2016

This case is very Wercker focussed, but the Wercker CLI is thin Docker API client. I suspect that the root cause is somewhere in the Docker Mac file system layer.

Wercker CLI is a Docker API client to build and test your code using containers. You no longer need your dev tools on your host machine. The Wercker CLI downloads the Docker image, sets up the file systems and networking and starts the container. Using the wercker dev option, you can spawn a shell in the container and work from there.

Expected behavior

The npm-install step is a short shell script running some npm commands. It first configures the NPM cache when use-cache: true. It then runs npm install to download the required NPM packages specified in package.json. In the package.json, you can specify package names and versions from npmjs.com or you can specify Github locations. Here is the expected outcome when running npm install with different package locations, and running it both under Docker Linux and Mac:

Platform Pkg from npmjs.com Pgk from Github
Linux
Mac

Actual behavior

Here is the actual outcome:

Platform Pkg from npmjs.com Pgk from Github
Linux
Mac

As you can see, when running it with Docker on Linux, it works even when using packages from Github. With Docker Mac 1.12.0-beta22 (build 11222), using packages from Github fails with file system permission errors. Installing npm packages from npmjs.com was fixed via #76. I first thought this was another case of the same issue but @dsheets asked me to file a separate issue for this.

Here is the console output:

$ wercker dev
--> Executing pipeline
open /Users/ringods/Projects/releasequeue/rq-website/.wercker/builds: no such file or directory
--> Running step: setup environment
Pulling from ringods/hugo-asset-build: 2016-08-03
Digest: sha256:f4a2a0492eb420692e84a40cd76727098210eab47094139a1715e3443ca057df
Status: Image is up to date for ringods/hugo-asset-build:2016-08-03
--> Copying source to container
--> Running step: wercker-init
--> Running step: npm-install
Using wercker cache
Creating $WERCKER_CACHE_DIR/wercker/npm
Configuring npm to use wercker cache
Starting npm install, try: 1
npm ERR! setPermissions Failed to change git repository ownership under npm cache for /pipeline/cache/wercker/npm/_git-remotes/git-https-github-com-gulpjs-gulp-git-4-0-4b46db44
npm ERR! Linux 4.4.16-moby
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! node v6.3.1
npm ERR! npm  v3.10.5
npm ERR! path /pipeline/cache/wercker/npm/_git-remotes/git-https-github-com-gulpjs-gulp-git-4-0-4b46db44/objects/pack/pack-182d819c91725e4f3eb09be74fb84ef746d69142.idx
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall chown

npm ERR! Error: EACCES: permission denied, chown '/pipeline/cache/wercker/npm/_git-remotes/git-https-github-com-gulpjs-gulp-git-4-0-4b46db44/objects/pack/pack-182d819c91725e4f3eb09be74fb84ef746d69142.idx'
npm ERR!     at Error (native)
npm ERR!  { Error: EACCES: permission denied, chown '/pipeline/cache/wercker/npm/_git-remotes/git-https-github-com-gulpjs-gulp-git-4-0-4b46db44/objects/pack/pack-182d819c91725e4f3eb09be74fb84ef746d69142.idx'
npm ERR!     at Error (native)
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'chown',
npm ERR!   path: '/pipeline/cache/wercker/npm/_git-remotes/git-https-github-com-gulpjs-gulp-git-4-0-4b46db44/objects/pack/pack-182d819c91725e4f3eb09be74fb84ef746d69142.idx',
npm ERR!   parent: 'rq-website' }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

npm ERR! Please include the following file with any support request:
npm ERR!     /pipeline/source/npm-debug.log
Clearing npm cache
npm ERR! Linux 4.4.16-moby
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "cache" "clear"
npm ERR! node v6.3.1
npm ERR! npm  v3.10.5
npm ERR! path /pipeline/cache
npm ERR! code EBUSY
npm ERR! errno -16
npm ERR! syscall rmdir

npm ERR! EBUSY: resource busy or locked, rmdir '/pipeline/cache'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /pipeline/source/npm-debug.log

Information

  • Diagnostic ID: Docker Mac is stuck in "Diagnosing" but doesn't return a diagnostic ID.
  • Docker Mac 1.12.0-beta22 (build 11222)
  • OSX 10.11.6

Steps to reproduce the behavior

The Wercker CLI uses a wercker.yml file to read all about the expected setup. To reproduce this case, paste the following snippet in a wercker.yml file:

dev:
  box:
    id: ringods/hugo-asset-build:2016-08-03
  steps:
    - wercker/npm-install@1.1.4:
        use-cache: true
    - internal/shell

and use the following NPM package.json file in the same directory:

{
  "name": "rq-website",
  "devDependencies": {
    "gulp": "git+https://github.com/gulpjs/gulp.git#4.0",
    "gulp-ccr-clean": "^0.1.1",
    "gulp-chef": "^0.1.4",
    "string": "^3.3.1"
  },
  "scripts": {
    "clean": "gulp clean",
    "build": "gulp",
    "tasks": "gulp --tasks"
  }
}

What this basically does when running wercker dev is spinning up the container from the image specified in the box tag, mounting the local file system in the container, running the npm-install step (wrapper around a shell script) and spawning a shell in the container.

@dsheets

This comment has been minimized.

Copy link
Contributor

commented Aug 15, 2016

There are potentially multiple issues here. The primary issue is the same issue is #117 -- wercker is attempting to change the ownership of a file that has permissions 444 (read-only). I've updated our internal ticket and increased its priority. Thank you for your detailed report. I will leave this issue open until I can confirm that this specific wercker use case functions correctly.

@dzonekl

This comment has been minimized.

Copy link

commented Aug 23, 2016

I have the same problem, #117 is closed however, so what is the solution?
Thanks Christophe

@dsheets

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2016

The fix is under development. The workaround is to only change ownership of files and directories that you have read and write access to. We are aware that this is not always possible and have expedited the fix. Sorry for the inconvenience. 😕

@dzonekl

This comment has been minimized.

Copy link

commented Aug 24, 2016

Hi, Not sure I understand the work around, in our case it's NPM which creates a (.npm) directory on a mounted volume, when getting js packages. so npm changes the ownership of the files.

The permissions on this mounted volume is my own login on the Mac. So should a specific docker user, be the owner or be in the group? I am not aware of a 'docker' user anyways...

@Morriz

This comment has been minimized.

Copy link

commented Sep 7, 2016

I had the same problem, and in the container's npm install script just did a

chown -R root /root/.npm
chmod -R 777 /root/.npm

which solved the issue...npm is now allowed to do whatever on it's cache

@dsheets

This comment has been minimized.

Copy link
Contributor

commented Oct 26, 2016

This should be fixed in the just-released Beta 29. Sorry for the delay in resolving the issue -- we had a couple of false starts on the design but now we use an ACL entry to make xattrs (containing ownership metadata) behave like inode metadata in most circumstances. Please give it a try and let us know how it works (or doesn't) for you. I'm going to close this issue but if you find related problems, please feel free to re-open. If you find un-related problems, please open a new issue. :-)

Thanks for using Docker for Mac!

@dsheets dsheets closed this Oct 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.