Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/var/lib/nginx/ Permission Denied #4

Closed
johnjelinek opened this issue May 14, 2014 · 20 comments
Closed

/var/lib/nginx/ Permission Denied #4

johnjelinek opened this issue May 14, 2014 · 20 comments

Comments

@johnjelinek
Copy link
Contributor

@johnjelinek johnjelinek commented May 14, 2014

I am seeing a lot of these permissions errors with proxied sites. Can you add permissions to /var/lib/nginx/ as seen here: http://nishal-tech.blogspot.com/2013/06/nginx-13-permission-denied-while.html otherwise, I have to turn buffering off in my site.conf.

2014/05/14 17:50:52 [crit] 7#0: *111 open() "/var/lib/nginx/proxy/4/00/0000000004" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/application.css?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/application.css?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:50:52 [crit] 7#0: *44 open() "/var/lib/nginx/proxy/5/00/0000000005" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/jquery.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/jquery.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:50:53 [crit] 7#0: *47 open() "/var/lib/nginx/proxy/6/00/0000000006" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/backbone/plugins/emoticons.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/backbone/plugins/emoticons.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:50:56 [crit] 7#0: *51 open() "/var/lib/nginx/proxy/7/00/0000000007" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/application.css?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/application.css?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:50:56 [crit] 6#0: *259 open() "/var/lib/nginx/proxy/8/00/0000000008" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/application.css?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/application.css?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:50:56 [crit] 6#0: *260 open() "/var/lib/nginx/proxy/9/00/0000000009" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/jquery.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/jquery.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:50:57 [crit] 6#0: *275 open() "/var/lib/nginx/proxy/0/01/0000000010" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/backbone/plugins/emoticons.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/backbone/plugins/emoticons.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:51:09 [crit] 6#0: *331 open() "/var/lib/nginx/proxy/1/01/0000000011" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/application.css?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/application.css?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:51:10 [crit] 6#0: *257 open() "/var/lib/nginx/proxy/2/01/0000000012" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/jquery.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/jquery.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
2014/05/14 17:51:11 [crit] 6#0: *258 open() "/var/lib/nginx/proxy/3/01/0000000013" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/backbone/plugins/emoticons.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/backbone/plugins/emoticons.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
@pilwon pilwon closed this in 6bfe874 May 14, 2014
@johnjelinek

This comment has been minimized.

Copy link
Contributor Author

@johnjelinek johnjelinek commented May 14, 2014

@pilwon I am still getting errors:

2014/05/14 20:53:24 [crit] 10#0: *19 open() "/var/lib/nginx/proxy/3/00/0000000003" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/backbone/plugins/emoticons.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/backbone/plugins/emoticons.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"

It appears that the permissions are appropriate though:

$ cat /var/log/nginx/kandan_vnext_error.log
2014/05/14 20:59:20 [crit] 11#0: *214 open() "/var/lib/nginx/proxy/6/00/0000000006" failed (13: Permission denied) while reading upstream, client: 10.9.4.52, server: , request: "GET /assets/backbone/plugins/emoticons.js?body=1 HTTP/1.1", upstream: "http://172.17.0.4:3000/assets/backbone/plugins/emoticons.js?body=1", host: "kandan:8080", referrer: "http://kandan:8080/users/sign_in"
[ root@cc668cb100dc:/var/lib/nginx ]$ ls -l
total 20K
drwx------ 2 www-data root 4.0K May 14 20:53 body/
drwx------ 2 www-data root 4.0K May 14 20:53 fastcgi/
drwx------ 2 www-data root 4.0K May 14 20:53 proxy/
drwx------ 2 www-data root 4.0K May 14 20:53 scgi/
drwx------ 2 www-data root 4.0K May 14 20:53 uwsgi/

Any thoughts? Perhaps the temp buffering dirs still being created as root. There's nothing in /var/lib/nginx/proxy/ when I check.

@pilwon

This comment has been minimized.

Copy link
Member

@pilwon pilwon commented May 14, 2014

@johnjelinek Check the permission of /var/lib/nginx/proxy/6/00/0000000006

@johnjelinek

This comment has been minimized.

Copy link
Contributor Author

@johnjelinek johnjelinek commented May 14, 2014

It doesn't exist. The response is buffered to that directory for a very short period ... so it's hard to see anything that shows up here.

[ root@b5be22863378:/var/lib/nginx/proxy ]$ ls -l
total 0
[ root@b5be22863378:/var/lib/nginx/proxy ]$ cd 6
bash: cd: 6: No such file or directory
@johnjelinek

This comment has been minimized.

Copy link
Contributor Author

@johnjelinek johnjelinek commented May 14, 2014

I am also getting this when I try to upload files.

2014/05/14 21:14:30 [crit] 12#0: *189 open() "/var/lib/nginx/body/0000000003" failed (13: Permission denied), client: 10.9.4.52, server: , request: "POST /channels/1/attachments.json HTTP/1.1", host: "kandan:8080", referrer: "http://kandan:8080/"
[ root@afd05def8f17:/var/lib/nginx ]$ ls -l -R
.:
total 20K
drwx------ 2 www-data root 4.0K May 14 21:10 body/
drwx------ 2 www-data root 4.0K May 14 21:10 fastcgi/
drwx------ 2 www-data root 4.0K May 14 21:10 proxy/
drwx------ 2 www-data root 4.0K May 14 21:10 scgi/
drwx------ 2 www-data root 4.0K May 14 21:10 uwsgi/

./body:
total 0

./fastcgi:
total 0

./proxy:
total 0

./scgi:
total 0

./uwsgi:
total 0
@Palver

This comment has been minimized.

Copy link

@Palver Palver commented Jun 2, 2014

How about the OS? Ubuntu has Apparmor which can prevent nginx accessing files that are not defined in nginx' apparmor profile file. Check the syslog for lines that both contain nginx and DENIED. Just a hint.

@iragsdale

This comment has been minimized.

Copy link

@iragsdale iragsdale commented Aug 26, 2014

I am having this same issue. I mounted a system directory there too, still failing.

@mascor

This comment has been minimized.

Copy link

@mascor mascor commented May 15, 2015

SOLVED:
chown -R www-data:www-data /var/lib/nginx

@adrian7

This comment has been minimized.

Copy link

@adrian7 adrian7 commented Jan 8, 2016

I am having the same issue, although I did chown -R www-data:www-data /var/lib/nginx .
Also seems to trigger only when uploading files larger then 4kb.

@carlalexander

This comment has been minimized.

Copy link

@carlalexander carlalexander commented Jan 24, 2016

I'm also running into the same issue as @adrian7 (works for files under 4kb). It's not consistent though. It only happens when I switch the user id of www-data to workaround a known boot2docker issue.

@adrian7

This comment has been minimized.

Copy link

@adrian7 adrian7 commented Jan 25, 2016

@carlalexander After some reading it's not an issue with docker, but with nginx and maybe because in my case it's running under supervisord. I just added client_body_temp_path /var/www/tmp 1 2; in sites-available/default.conf and solved the issue.

More on this here https://wincent.com/wiki/Fixing_nginx_client_body_temp_permission_denied_errors

@adrian7

This comment has been minimized.

Copy link

@adrian7 adrian7 commented Jan 25, 2016

BTW /var/www/ is a shared volume

@carlalexander

This comment has been minimized.

Copy link

@carlalexander carlalexander commented Jan 25, 2016

Thanks @adrian7, will look at that!

@carlalexander

This comment has been minimized.

Copy link

@carlalexander carlalexander commented Jan 25, 2016

I used /tmp instead of /var/www/tmp. Worked like a charm. Thanks!

@Ocramius

This comment has been minimized.

Copy link

@Ocramius Ocramius commented Apr 13, 2016

Note: if you don't deal with uploaded files, you can just set a higher client_body_buffer_size.

Setting a high client_body_buffer_size also reduces file I/O, which is kinda good if you don't have a fast filesystem (virtual environments):

To sum it up, you need to just customize this bit of your config:

http {
    # ...

    client_body_temp_path /tmp 1 2;
    client_body_buffer_size 256k;
    client_body_in_file_only off;

    # ...
}
@dcwangmit01

This comment has been minimized.

Copy link

@dcwangmit01 dcwangmit01 commented Jun 7, 2017

In some versions of nginx docker images the nginx runs as user "nginx" instead of "www-data". This is why the chown -R www-data:www-data /var/lib/nginx worked for some people and not others. Changing the location to /tmp works because /tmp is writable for all users. The buffer_size fix might only work if you have small files.

Cameri added a commit to Cameri/docker-revive-adserver that referenced this issue Jul 17, 2018
@sacdallago

This comment has been minimized.

Copy link

@sacdallago sacdallago commented Sep 4, 2018

For anyone landing here having this problem in uwsgi or similar: if you use an include like include /etc/nginx/uwsgi_params;, you need to change the temp file location for that specific handler, i.e.:

# Override temp file locations
client_body_temp_path /var/lib/nginx-tmp/client_body;
proxy_temp_path /var/lib/nginx-tmp/proxy;
fastcgi_temp_path /var/lib/nginx-tmp/fastcgi 1 2;
uwsgi_temp_path /var/lib/nginx-tmp/uwsgi;
scgi_temp_path /var/lib/nginx-tmp/scgi;
@kunal097

This comment has been minimized.

Copy link

@kunal097 kunal097 commented Sep 25, 2018

@johnjelinek is your issue resolved?
I'm getting exactly same issue.
Please help!!

@kunal097

This comment has been minimized.

Copy link

@kunal097 kunal097 commented Sep 25, 2018

error

@6a

This comment has been minimized.

Copy link

@6a 6a commented Nov 22, 2018

For anyone landing here having this problem in uwsgi or similar: if you use an include like include /etc/nginx/uwsgi_params;, you need to change the temp file location for that specific handler, i.e.:

# Override temp file locations
client_body_temp_path /var/lib/nginx-tmp/client_body;
proxy_temp_path /var/lib/nginx-tmp/proxy;
fastcgi_temp_path /var/lib/nginx-tmp/fastcgi 1 2;
uwsgi_temp_path /var/lib/nginx-tmp/uwsgi;
scgi_temp_path /var/lib/nginx-tmp/scgi;

This fix worked! Thank you.

@SunLn

This comment has been minimized.

Copy link

@SunLn SunLn commented Feb 25, 2019

I am in macOs Mojave. Solved by

ps aux | grep "nginx: worker process"
sudo chown -R your_current_nginx_user:admin /usr/local/var/run/nginx/proxy_temp/
sudo nginx -s reload
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.