When I request my site sometimes I get an error "application did not send a complete response"

[c80609a]

• Centos-7-x86_64
• Rails 4.2.0
• nginx/1.8.0
• Phusion Passenger version 5.0.10 (stand alone)
• Gemfile
• nginx.conf and included site.conf
• passenger system metrics screenshot
• I start the app with command:
  passenger start --daemonize --port 4040 --user webmaster -e production
• log/passenger.4040.log contains:

[ 2015-09-11 07:29:24.1809 30625/7ff3f8e0b780 age/Wat/Main.cpp:1293 ]: Starting PassengerAgent watchdog...
[ 2015-09-11 07:29:24.1849 30628/7fed14983780 age/Hel/Main.cpp:957 ]: Starting PassengerAgent server...
[ 2015-09-11 07:29:24.1850 30628/7fed14983780 age/Hel/Main.cpp:234 ]: PassengerAgent server running in multi-application mode.
[ 2015-09-11 07:29:24.1865 30628/7fed14983780 age/Hel/Main.cpp:707 ]: PassengerAgent server online, PID 30628
[ 2015-09-11 07:29:24.1911 30635/7f992b2bb780 age/Log/Main.cpp:430 ]: Starting PassengerAgent logger...
[ 2015-09-11 07:29:24.1914 30635/7f992b2bb780 age/Log/Main.cpp:311 ]: PassengerAgent logger online, PID 30635
App 30684 stdout:

• I request my site by browser and got 'application did not send a complete response', and passenger log contains:

[ 2015-09-11 07:29:24.1809 30625/7ff3f8e0b780 age/Wat/Main.cpp:1293 ]: Starting PassengerAgent watchdog...
[ 2015-09-11 07:29:24.1849 30628/7fed14983780 age/Hel/Main.cpp:957 ]: Starting PassengerAgent server...
[ 2015-09-11 07:29:24.1850 30628/7fed14983780 age/Hel/Main.cpp:234 ]: PassengerAgent server running in multi-application mode.
[ 2015-09-11 07:29:24.1865 30628/7fed14983780 age/Hel/Main.cpp:707 ]: PassengerAgent server online, PID 30628
[ 2015-09-11 07:29:24.1911 30635/7f992b2bb780 age/Log/Main.cpp:430 ]: Starting PassengerAgent logger...
[ 2015-09-11 07:29:24.1914 30635/7f992b2bb780 age/Log/Main.cpp:311 ]: PassengerAgent logger online, PID 30635
App 30684 stdout:
App 30702 stdout:
[ 2015-09-11 07:29:37.0117 30628/7fed14981700 App/Poo/AnalyticsCollection.cpp:104 ]: Process (pid=30702, group=/home/webmaster/git/bitbucket/s40/public (production)) no longer exists! Detaching it from the pool.
[ 2015-09-11 07:29:37.0118 30628/7fed14981700 age/Hel/Main.cpp:794 ]: Disconnecting long-running connections for process 30702, application /home/webmaster/git/bitbucket/s40/public (production)
App 30763 stdout:
[ 2015-09-11 07:29:42.0115 30628/7fed14981700 App/Poo/AnalyticsCollection.cpp:104 ]: Process (pid=30763, group=/home/webmaster/git/bitbucket/s40/public (production)) no longer exists! Detaching it from the pool.
[ 2015-09-11 07:29:42.0116 30628/7fed14981700 age/Hel/Main.cpp:794 ]: Disconnecting long-running connections for process 30763, application /home/webmaster/git/bitbucket/s40/public (production)
App 30784 stdout:
[ 2015-09-11 07:30:00.7814 30628/7fed11d36700 age/Hel/Req/Utils.cpp:95 ]: [Client 2-1] Sending 502 response: application did not send a complete response
[ 2015-09-11 07:30:02.0138 30628/7fed14981700 App/Poo/AnalyticsCollection.cpp:104 ]: Process (pid=30784, group=/home/webmaster/git/bitbucket/s40/public (production)) no longer exists! Detaching it from the pool.
[ 2015-09-11 07:30:02.0139 30628/7fed14981700 age/Hel/Main.cpp:794 ]: Disconnecting long-running connections for process 30784, application /home/webmaster/git/bitbucket/s40/public (production)
App 30849 stdout:
[ 2015-09-11 07:31:12.7510 30628/7fed12737700 age/Hel/Req/Utils.cpp:95 ]: [Client 1-2] Sending 502 response: application did not send a complete response
[ 2015-09-11 07:31:17.0139 30628/7fed14981700 App/Poo/AnalyticsCollection.cpp:104 ]: Process (pid=30849, group=/home/webmaster/git/bitbucket/s40/public (production)) no longer exists! Detaching it from the pool.
[ 2015-09-11 07:31:17.0140 30628/7fed14981700 age/Hel/Main.cpp:794 ]: Disconnecting long-running connections for process 30849, application /home/webmaster/git/bitbucket/s40/public (production)
App 31080 stdout:

• I make several requests pressing f5 and got the site content but passenger log contains:

[ 2015-09-11 07:46:10.0147 1829/7f53084ac700 Spa/SmartSpawner.h:726 ]: An error occurred while spawning a process: An error occurred while starting the application. Unable to connect to the preloader's socket: Cannot connect to Unix socket '/tmp/passenger.imtn5ZQ/apps.s/preloader.1qgg6bb': Connection refused (errno=111)
[ 2015-09-11 07:46:10.0147 1829/7f53084ac700 Spa/SmartSpawner.h:727 ]: The application preloader seems to have crashed, restarting it and trying again...
App 3601 stdout:
App 3626 stdout:
[ 2015-09-11 07:46:20.0119 1829/7f530d208700 App/Poo/AnalyticsCollection.cpp:104 ]: Process (pid=3626, group=/home/webmaster/git/bitbucket/s40/public (production)) no longer exists! Detaching it from the pool.
[ 2015-09-11 07:46:20.0120 1829/7f530d208700 age/Hel/Main.cpp:794 ]: Disconnecting long-running connections for process 3626, application /home/webmaster/git/bitbucket/s40/public (production)
[ 2015-09-11 07:46:20.0127 1829/7f53083be700 App/Implementation.cpp:303 ]: Could not spawn process for application /home/webmaster/git/bitbucket/s40: An error occurred while starting the application. Unable to connect to the preloader's socket: Cannot connect to Unix socket '/tmp/passenger.imtn5ZQ/apps.s/preloader.19b7u6x': Connection refused (errno=111)
  Error ID: c04dac7e
  Error details saved to: /tmp/passenger-error-fJ3lJ6.html
  Message from application: An error occurred while starting the application. Unable to connect to the preloader&apos;s socket: Cannot connect to Unix socket &apos;/tmp/passenger.imtn5ZQ/apps.s/preloader.19b7u6x&apos;: Connection refused (errno=111)<br>
<h2>Raw process output:</h2>
(empty)

[ 2015-09-11 07:46:20.0128 1829/7f53083be700 Spa/SmartSpawner.h:726 ]: An error occurred while spawning a process: An error occurred while starting the application. Unable to connect to the preloader's socket: Cannot connect to Unix socket '/tmp/passenger.imtn5ZQ/apps.s/preloader.19b7u6x': Connection refused (errno=111)
[ 2015-09-11 07:46:20.0128 1829/7f53083be700 Spa/SmartSpawner.h:727 ]: The application preloader seems to have crashed, restarting it and trying again...
App 3675 stdout:
App 3701 stdout:

[FooBarWidget]

This happens when your application exits prematurely. To understand what this means, consider that Passenger works by sitting between the client and the app. Passenger acts like a reverse proxy, so it forwards the request to your app, then processes the response that the app sends.

client <-----> Passenger <-----> app

If, after Passenger has sent the request, the app crashes or otherwise exits before sending a response, then you will see "application did not send a complete response".

So the question is actually: why does the application exit? Unfortunately I do not know, and neither does Passenger. Passenger only starts your app and expects your app to respond to requests as normal. Maybe there is a bug in the app, or the app encountered some sort of fatal error. Normally the app will print an error message when that happens, but Passenger did not encounter any such messages, or it would have printed them.

So the best thing I can recommend you to do is that you insert debugging statements inside your app and find out what makes it exit.

[c80609a]

Thanks, FooBarWidget!

[c80609a]

Info for future researchers: there was a "out of memory" error. My app was deployed on a small OpenVZ VPS hosting and without swap (vswap).

[ghost]

How did you figure out if it's an "out of memory" error?

free -t -m

         total       used       free     shared    buffers     cached

Mem: 1500 1184 315 17 58 624
-/+ buffers/cache: 501 998
Swap: 2303 369 1934
Total: 3804 1553 2250