PHP-FPM stderr logs cause nginx to return 502 errors to client #878
Comments
|
Instead of |
|
I am facing a similar issue, once too many notices come out of a request. We get a 502 from nginx. I tried changing the following things: (in /usr/local/etc/php-fpm.d/docker.conf) To To To However, it did not solve the issue. |
|
Switching I question why this is overridden by default? It seems this decision should not be forced on the user. nginx is very commonly used with php-fpm, and does not support this limit by default (only up to |
|
Thanks, I've seen this, but this is poor reasoning, in my opinion. In fact, he admits it wasn't changed in PHP due to backwards compatibility. I think it should keep the PHP default and you can choose to override if you want, not the other way around. At the very least it should be documented that this was bumped and you will need to take that into account when pairing with nginx, because literally everyone using nginx will run into this problem at some point. |
|
@tomsisk I ran into this point today after updating 7.0 to 7.4 php_fpm based images :) Seems not optimal to me to change |
|
I use PHP-FPM 7.4 Alpine 3.12 image in conjunction with I tried to fix |
|
@stepanselyuk, from some quick research, most users suggest that it may be because there is We try to keep github issues focused on bugs in the image and not general deployment debugging. It likely seems not to be a problem with the image so I would suggest trying to post questions like this in the Docker Community Forums, the Docker Community Slack, or Stack Overflow. |
|
Hi @yosifkit, thank you. Seems I found the underlying issue: I write logs and cache to /dev/shm (in local dev-containers) and forgot to change this logic. So after exactly ~4 hours the size of the /dev/shm reached 64M and that caused the issue. |
|
I'm having same problem here After trying different things seems that outputting logs to so I added currently to my dockerfile My Docker version on Ubuntu 20.04 |
|
Hello everyone 👋 TL;DR: set Long explanation: I know I'm late to the party, but this also means I've the advantage of having more google results available and I'm quite confident I found the real fix for what was reported by OP:
It was suggested that the It is true, in case the fastcgi stderr channel contains data and is too big, nginx will act on this, this is the source code thereover where this happens in nginx https://github.com/nginx/nginx/blob/c9c3f2f005c43fd885bd78f86e1044261e639bea/src/http/modules/ngx_http_fastcgi_module.c#L1746-L1762 ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
"FastCGI sent in stderr: \"%*s\"", p - msg, msg);
flcf = ngx_http_get_module_loc_conf(r, ngx_http_fastcgi_module);
if (flcf->catch_stderr) {
pattern = flcf->catch_stderr->elts;
for (i = 0; i < flcf->catch_stderr->nelts; i++) {
if (ngx_strnstr(msg, (char *) pattern[i].data,
p - msg)
!= NULL)
{
return NGX_HTTP_UPSTREAM_INVALID_HEADER;
}
}
}nginx has a setting to control this behaviour and it's called
So theoretically this behaviour can also be mitigated by configuring nginx to not act that way on fastcgi errors being too large. But the real answer is from this stackoverflow question:
And this is IMHO the correct fix: set
What happens is:
That's why people also see the PHP errors (warnings, notices, etc.) twice:
Disabling does the desired thing:
In my case I was processing a GraphQL call which triggered an exception; the exception was caught inside the framework correctly and turned into a proper GraphQL JSON response. But the exception was also logged deliberately and what happen then is:
Personally (though I'm not an expert on this topic), since we're talking about the "official docker PHP" images here, I'd say it would make very much sense to either:
HTH and have a nice day! |
|
/proc/1/fd/2 work with php-fpm 8.2 |
|
@mfn Thank you very much! I've spent a lot of time looking for a solution. Your answer helped solve the problem |
|
My two cents (not that they matter): I still think changing It comes down to changing another default could potentially create other problems we don't know about yet, just like this one. It shouldn't be assumed that someone is working with specific technologies, it should just work like if I installed PHP on my PC. All that said, a note on this issue, even just linking here I think is crucial. I know I spent many hours trying to figure this out years ago, and there are still people doing the same thing. |
|
I've finally implemented #878 (comment) over in #1360, but I don't plan to let it merge until at least the new year (I've had a tab open to this issue for over a year now, intending to implement something 😩). |
added important comment docker-library/php#878
Mistake from RSS-Bridge#3500 Wrong file extension: should have been `.ini` and not `.conf` otherwise it has no effect. See docker-library/php#1360 and docker-library/php#878 (comment)
Mistake from #3500 Wrong file extension: should have been `.ini` and not `.conf` otherwise it has no effect. See docker-library/php#1360 and docker-library/php#878 (comment)
I have php-fpm running on docker with the following php.ini:
Everything worked and nginx was proxying perfectly. The problem with this is logs are never shown in the container's stdout, so I changed it to:
error_log = /proc/self/fd/2
which meant that I could see the logs with docker logs -f, but I started seeing these errors on nginx:
So some plugin in my wordpress (and I know which one it is), is using a deprecated function, but this error is non critical, its actually a notice. The problem is that with this setup, I keep getting 502 Bad Gateway errors when a client requests a page. I actually also see that php-fpm is returning them via the fast cgi protocol, when I connect do it directly.
It all starts working when I set log_errors = Off. So I read a bit and realized the fast-cgi protocol can return a FCGI_STDERR packet to its client, which includes error logs, so I decided not to go down that rabbit hole and try to cheat by doing this:
ln -sf /dev/stderr /var/log/php/error.logto make php-fpm think it was logging to a file and still have the logs show up in docker logs. No such luch, 502 only disappear is I log to /var/log/php/error.log without any connection to stdout/stderr or if I disable logging altogether.I don't know if its my poor understanding or if this is a bug, but there should be a way to log to stdout/stderr without returning errors to nginx.
Can anyone help?
The text was updated successfully, but these errors were encountered: