[Bug] Cold starts cause FCGI: attempt to connect to Unix domain socket /var/run/php/php-fpm.sock (localhost:8000) failed

[pentium10]

Issue description

At cold starts, for a brief time, we get 503 errors.

[Mon Jan 09 19:35:42.278107 2023] [proxy:error] [pid 1131:tid 68609648887360] (2)No such file or directory: AH02454: FCGI: attempt to connect to Unix domain socket /var/run/php/php-fpm.sock (localhost:8000) failed
[Mon Jan 09 19:35:42.278549 2023] [proxy_fcgi:error] [pid 1131:tid 68609648887360] [client 169.254.1.1:63202] AH01079: failed to make connection to backend: httpd-UDS
169.254.1.1 - - [09/Jan/2023:19:35:42 +0000] "GET /i.php HTTP/1.1" 503 473 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"
2023-01-09 21:35:42.287 EET
GET 503 1,016 B 4.9s Chrome 108 https://example.org/i.php

Environment

• Operating System Version:
• Docker Version:

Steps to reproduce the issue

create a i.php phpinfo() file to host container

1. deploy | run
2. constantly hit F5
3. when the first few requests are handled it returns the above error

What is expected?

503 errors should not happen before proper booting of environment

My suspicion is that Apache is up sooner than FPM PHP, and this causes the errors.

Additional details / screenshots

FROM shinsenter/phpfpm-apache:php8.2
ENV PHP_OPEN_BASEDIR /
RUN echo 'error_log = /var/log/php_error_log' >> /etc/php/8.2/fpm/php.ini \
    && echo 'ignore_repeated_errors = on' >> /etc/php/8.2/fpm/php.ini \
    && sed -i 's/80/8080/g' /etc/apache2/sites-available/000-default.conf /etc/apache2/ports.conf \
    && echo "ServerSignature Off" >> /etc/apache2/apache2.conf 

COPY . /var/www/html

EXPOSE 8080

[shinsenter]

@pentium10
You are correct that in some contexts php-fpm will start after Apache, causing a 503 error.
I think that returning a 503 error code is an acceptable result in the above case.

There are a few other possible ways to handle this situation:

1. Fallback 503 errors into a default HTML content with a different error code.
2. Wait and start Apache after php-fpm has finished booting (when there is a user access while Apache has not been started, the server will not respond).

I look forward to receiving some other suitable solutions from you.

[pentium10]

We implemented these containers to use in a serverless environment, we spent 2 days swapping our existing containers for this one. After we put in production bummer, nothing works.

So these containers ARE NOT SUITABLE in serverless environments at all:

• on Cloud Run
• on Kubernetes
• on Fargate etc...

We were trying to see if these containers are faster for a cold start than others. Our local metrics looked promising, but with the small delay in FPM starting in production, they never worked.

So this "sometimes" or "acceptable" result to return 503, is not going to work in serverless environment, because the orchestrator automatically will classify the instance as probe-with-error, and try to start a new instance. Leading to huge number of containers started, that all fail. Why? Because the orchestrator can establish network but it's faulty error code returns.

Any mitigation for a HTTP error code, is not acceptable.

The solution needs to work well for the above situation.
Open the container's listen PORT, when every dependency is up.

[shinsenter]

@pentium10
I apologize for any confusion caused, you are correct.
I will find a better approach for this issue.

Please let me know if there is anything else I can help you with or if you have any idea to make these containers work with serverless environments.

[shinsenter]

@pentium10
I have improved the behavior of the web service inside docker containers.
Please check the latest docker images at https://docker.shin.company/

If you have any questions or concerns about the source code or anything else, please let me know and I'll be happy to help.

[pentium10]

Not sure what's going on, but cold start now has a huge latency >15 seconds.

2023-01-12 17:18:54.831 EET
169.254.1.1 - - [12/Jan/2023:15:18:54 +0000] "GET /i.php HTTP/1.1" 200 112606 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023-01-12 17:18:54.843 EET
GET200110.38 KB17.2 sChrome 109 i.php

2023-01-12 17:18:57.281 EET
169.254.1.1 - - [12/Jan/2023:15:18:57 +0000] "GET /i.php HTTP/1.1" 200 112606 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
2023-01-12 17:18:57.294 EET
GET200110.38 KB15.2 sChrome 109 i.php

[pentium10]

We are using Cloud Run from Google cloud platform to run these in serverless environments.
I've created a test repo to try it out yourself.

https://github.com/pentium10/shinsenterPhpFpm

If you setup the gcloud utility on your terminal.
You can use

gcloud builds submit

[shinsenter]

@pentium10
The latency has been improved. Please take a look.
I deployed using your setup with the latest shinsenter/php:8.2-fpm-apache image.
The Apache server was up within 2 seconds.

[17-Jan-2023 04:13:16] NOTICE: fpm is running, pid 1125
[17-Jan-2023 04:13:16] NOTICE: ready to handle connections
[17-Jan-2023 04:13:16] NOTICE: systemd monitor interval set to 10000ms
Found /var/run/php/php-fpm.sock!
[Tue Jan 17 04:13:16.312710 2023] [mpm_event:notice] [pid 1138:tid 139786159699840] AH00489: Apache/2.4.54 (Ubuntu) OpenSSL/3.0.2 configured -- resuming normal operations
[Tue Jan 17 04:13:16.313657 2023] [core:notice] [pid 1138:tid 139786159699840] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'