New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permission errors #31

Closed
treeder opened this Issue Sep 9, 2015 · 31 comments

Comments

Projects
None yet
8 participants
@treeder
Copy link

treeder commented Sep 9, 2015

Probably needs a chmod after generation since docker will make everything owned by root. I did a rm -rf on _site and tried this again to ensure this is an issue (similar to this: https://github.com/treeder/go/blob/master/go.sh#L45).

          Source: /srv/jekyll
       Destination: /srv/jekyll/_site
      Generating... 
              Lunr: Skipping search indexing at user request
jekyll 2.4.0 | Error:  Permission denied @ dir_s_mkdir - /srv/jekyll/_site
Moving Gemfile.docker back to Gemfile
I hope you did not have a Gemfile actually called that.

@treeder treeder changed the title Permission errors on 2nd run Permission errors Sep 9, 2015

@treeder

This comment has been minimized.

Copy link
Author

treeder commented Sep 9, 2015

Happens every run actually.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

I appreciate the link to your code but we already chmod files and setuid/gid. Provide the command you are using because I am unable to replicate this.

@treeder

This comment has been minimized.

Copy link
Author

treeder commented Sep 9, 2015

just running this:

 docker run --rm --label=jekyll --volume=$(pwd):/srv/jekyll -t -p 4000:4000 jekyll/jekyll:pages jekyll s
@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

I'm inclined to believe this is a problem with one of your plugins or something and not a direct bug of ours, if they don't pick up the SETUID/GID because they run independently (for or otherwise) there is nothing we can do about that and nothing we will do about that (because really there is nothing we can do about that, anything we do restricts your flexibility,) but you can work around that yourself by either removing the jekyll s and letting /usr/local/bin/default take over or doing what /usr/local/bin/default does:

docker run --rm --label=jekyll --volume=$(pwd):/srv/jekyll -t -p 4000:4000 \
  jekyll/jekyll:pages chpst -u jekyll:jekyll jekyll s

@envygeeks envygeeks added wontfix and removed pending feedback labels Sep 9, 2015

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

Actually now that I think about it, there is something I can do about it, but I'm gonna flag it as ultra-low priority... We can rewrap jekyll with chpst through jekyll to ensure that plugins that do stuff don't have problems and only do it if we detect that the GID/UID = 0 but since it's easy to work around if you have plugins that fork/spawn or otherwise I won't put it above other stuff.

@envygeeks envygeeks closed this in 8ac08bb Sep 9, 2015

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

You should be able to pull the new image and see if this fixes your problem @treeder, if it doesn't then I'm gonna need more information (a repo if you can provide it) so I can debug it from top to bottom but hopefully me wrapping Jekyll into chpst back into the main jekyll will fix it.

@treeder

This comment has been minimized.

Copy link
Author

treeder commented Sep 9, 2015

Same thing after pull. Running it on this: https://github.com/iron-io/docs

I remove the gem "therubyracer" then bundle update, then run it. When I have therubyracer in there it tries to build gems on run which is too slow.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

Oh, you are running gem commands on your own, that's something that is out of our scope and that's why we have gem runners, there is nothing we can do about that and wrapping every command to guard against what you do is way out of our scope IMO.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

Wrap the commands into chpst -u jekyll:jekyll or do sudo -u jekyll both of which we readily make available for just these sorts of situations.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

BTW, if you read the docs you will notice that we provide $BUNDLE_CACHE which tells envygeeks/alpine to cache your gems into vendor/bundle so that things aren't slow after the first run, also, the RubyRacer is known to be broken on Alpine (that's why we provide nodejs/iojs on the image) see cowboyd/therubyracer#378 for info on the RubyRacer alpine issue.

@treeder

This comment has been minimized.

Copy link
Author

treeder commented Sep 9, 2015

Are you saying I shouldn't have a Gemfile or Gemfile.lock or something? Confused.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

I'm saying if you bundle install manually that's a scope you created, you need to chpst and chown yourself or use our environment variables and installers, I know it sounds rude of me to say that but we cannot be the keepers of what you do on the image.

@parkr

This comment has been minimized.

Copy link

parkr commented Sep 9, 2015

@envygeeks Does Jekyll have the capability to make _site?

@treeder

This comment has been minimized.

Copy link
Author

treeder commented Sep 9, 2015

To be clear, I'm not doing bundle install inside the container, I'm doing a bundle update without Docker just to update the Gemfile.lock. I'm deleting the container after every run.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 9, 2015

Yes, it does in-fact make it's own _site folder as the user jekyll but if you go out of our context (aka run bundle install, use something that forks (previous to our re-wrapping -- now we wrap jekyll into chpst back into Jekyll to ensure that forks gain our user id through the system) or run commands yourself on that folder it can lose that permission and there is nothing we can really do about that because that's the nature of the game with Docker -- in a non-literal sense because there is something we can do about it honestly.

By default we run bundle install for you multiple different ways through the parent image:
https://github.com/jekyll/docker-jekyll/blob/master/copy/etc/startup1.d/01-gem-depends#L23
https://github.com/envygeeks/docker/blob/master/vendor/common/usr/local/share/docker/helpers.d/ruby#L209

And through the parent image (envygeeks/alpine) we also allow you to cache by sending --env='BUNDLE_CACHE=true' and even override our cache folder or add extra arguments onto bundler with BUNDLE_ARGS='--your-arg' in this context through Jekyll it will retain it's permissions.

Now, I will concede that we can chown when it first boots but I'm a bit hesitant to do so because I don't know what I could break in the users folder, but tbh, they are already told that we run everything as UID=1000 anyways so half their files will be that effective UID to begin with. So I'm willing to consider that if it might fix your problem but again, I don't know how it would break a users stuff and I don't know that we should care.

@envygeeks envygeeks reopened this Sep 9, 2015

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 10, 2015

I thought about it long and hard last light and we are going to chown in startup.d for you.

@alberto56

This comment has been minimized.

Copy link

alberto56 commented Sep 10, 2015

Hi, I recently started getting the following error:

Permission denied @ rb_sysopen 

I'm wondering if it's related to this issue.

I have two CircleCI builds of the exact same commit here,

  • the first from one month ago uses v. 91c3c267a129 of jekyll/jekyll and is working fine.
  • the second from today uses v. 5f8dc0edd59c of jekyll/jekyll and is failing with jekyll 2.5.3 | Error: Permission denied @ rb_sysopen - /srv/jekyll/_site/Dcycle-Dockerfile-jekyll.
  • The code is here.
  • I can't reproduce this locally (on my CoreOS VM), only on CircleCI.
@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 10, 2015

Is there any way you can provide all of the CircleCI output (minus sensitive data of course.) It seems odd this would be happening but I need logs and commands to see exactly where things are going wrong.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 10, 2015

Oh, I just noticed it is avaialble. I'm dumb.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Sep 10, 2015

@alberto56 file another ticket so I can close that one in a unique commit, you're issue is unrelated to this one, this (yours) is related to an upper level Docker problem it looks like, I don't know if Docker would consider that a bug but we'll fix it from our end.

@alberto56

This comment has been minimized.

Copy link

alberto56 commented Sep 10, 2015

@envygeeks thanks for your swift response, it is much appreciated. I opened another issue at #32 as requested.

Cheers,

Albert.

@envygeeks envygeeks closed this in 52b776a Sep 10, 2015

@FredrikWendt

This comment has been minimized.

Copy link

FredrikWendt commented Dec 2, 2015

Adding a comment for latecomers: I got bit by this too, and it took me some time to find "we run as UID=1000". I'm doing all the work with Go.cd and the build agents have some unknown UID when they run. As suggested on https://github.com/jekyll/docker/wiki/Deploying-with-Jekyll-Docker using jekyll/jekyll:builder works for this scenario.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Dec 2, 2015

WAT? Please clarify everything you said because that image is _no different_ than the other ones.

@FredrikWendt

This comment has been minimized.

Copy link

FredrikWendt commented Dec 2, 2015

Hmm, my bad. Reverted my local work to pull :latest and that works (as it should, and it did pull a new image so I must've tricked myself into thinking I was running the latest :latest, while I was in fact not ...).

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Dec 2, 2015

@FredrikWendt you probably had an image that was pre: https://github.com/jekyll/docker/blob/master/copy/etc/startup1.d/01-preserve-uid which is not linked to this commit but was technically spawned because of this problem and a later problem (if I remember right.)

@agoransson

This comment has been minimized.

Copy link

agoransson commented Apr 13, 2016

When I run the following command locally it works like a charm

docker run --name="myjekyllsite" --label="jekyll" -i --rm -p 61616:80 -p 4000:4000 -v /home/data/myjekyllsite/src:/srv/jekyll jekyll/jekyll

however when I run it on a server it fails with the following error

Configuration file: /srv/jekyll/_config.yml
            Source: /srv/jekyll
       Destination: /srv/jekyll/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
jekyll 3.1.2 | Error:  Permission denied @ dir_s_mkdir - /srv/jekyll/_site

Any thoughts?

@FredrikWendt

This comment has been minimized.

Copy link

FredrikWendt commented Apr 13, 2016

The user, running jekyll inside the container, most likely doesn't have write access to the host OS' /home/data/myjekyllsite/src directory. You either need to match UID (and perhaps GID) of the host OS directory, or change the UID/GID of the user running the jekyll process inside the container.

@envygeeks

This comment has been minimized.

Copy link
Owner

envygeeks commented Apr 13, 2016

Yup, it could also be different permissions in sub-folders too. Since we only match the UID of /srv/jekyll if any sub-folder has a different UID it could still lead to a permission error since we expect consistent UID/GID and tbh, there isn't really anything we can do about that other than enforcing it (which users have backlashed against before so we do not do it.)

@agoransson

This comment has been minimized.

Copy link

agoransson commented Apr 13, 2016

The easiest thing I just tried now was to create the missing _site folder which kind of "solved" the problem right now. I tried sharing user id as per the manual (thanks for the hint) but I couldn't quite get it to work. I'll keep working on that since I'm positive that is the correct way to address the issue.

envygeeks added a commit that referenced this issue Apr 22, 2016

envygeeks added a commit that referenced this issue Apr 22, 2016

@flq

This comment has been minimized.

Copy link

flq commented Aug 14, 2016

I would like to add that on Windows 10 with Docker 1.12.0-beta21 (up to date)
running
docker run --rm --label=jekyll --volume=C:/...path_to_my_site/realfiction:/srv/jekyll -t -p 4000:4000 jekyll/jekyll:pages chpst -u jekyll:jekyll jekyll s

gives me

jekyll 3.1.6 | Error: Permission denied @ dir_s_mkdir - /srv/jekyll/_site

trying to connect to port 4000 gives me connection refused

@ZzAntares

This comment has been minimized.

Copy link

ZzAntares commented Oct 25, 2016

I encountered this problem just today, I just have to chown -R jekyll:jekyll /srv/jekyll since when entering docker I was creating files as root without noticing it.

In the end is better right after entering to the docker container to work as the jekyll user doing su jekyll to avoid this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment