DAMN ! worker 1 (pid: 108) died, killed by signal 9 :( trying respawn ...

[taschik]

Hi,

I am running uwsgi 2.0.17 in combination with a nginx infront and a Django 2.0.3 behind it in a docker container. 1,6% of my request are failing due to uwsgi dying for some reasons. I can see it in the logfiles to look like this:

DAMN ! worker 1 (pid: 108) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 1 (new pid: 149)

It happens on random URI's, so no specific views causing it. This is how my uwsgi.ini looks like:

[uwsgi]
# this config will be loaded if nothing specific is specified
# load base config from below
ini = :base

# %d is the dir this configuration file is in
socket = %dapp.sock
master = true
processes = 4

# pot. fix for prematurely closed upstream error
limit-as = 512

[dev]
ini = :base
# socket (uwsgi) is not the same as http, nor http-socket
socket = :8001

[local]
ini = :base
http = :8000

[base]
# chdir to the folder of this config file, plus app/website
chdir = %dserver
# load the module from wsgi.py, it is a python path from 
# the directory above.
module=server.wsgi:application
# allow anyone to connect to the socket. This is very permissive
chmod-socket=666

Any idea on how to trace the reason for dying or fixing the issue in general is highly appreciated.

[lhypj]

maybe your cache is out of memory or request last too long time (mainly caused by io)
i think you would get more infomation from you uwsgi.log

[koorukuroo]

Refer #1792

[colidyre]

I just want to mention that I had a similar error. I used threads in my app and the same error came up. So if someone uses threads, this might be a solution:

[uwsgi]
...
enable-threads = true

See also UWSGI docs:

By default the Python plugin does not initialize the GIL. This means your app-generated threads will not run. If you need threads, remember to enable them with enable-threads.

BTW: This article is also nice to read.

[AlexandreNeuk]

I had the same problem, for me changing the uwsgi.ini file, changing the value of the reload-on-rss setting from 2048 to 4048, and harakiri to 600 solved the problem.

[arviCV]

I almost tried every single thing explained here. Still exactly same error occurred due to uwsgi server. This is specifically for a particular flask endpoint whenever I deployed to k83 cluster and worked perfectly in dev machine.
Surprisingly, requesting more resource fixed the issue.

resources:
  limits:
    memory: 1Gi
  requests:
    memory: 512Mi

Hope this helps if you are also deploying the containers to a k8s cluster.

[mike927]

Changing number of processes from 4 to 1 solved problem for me what points that it has high memory demand.