New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition graceful restart while fork master #1492

Open
gmoryes opened this Issue Mar 22, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@gmoryes

gmoryes commented Mar 22, 2017

Hello, I'm using fork master ("f") scheme of restart, writing here: http://zarnovican.github.io/2016/02/15/uwsgi-graceful-reload/#re-exec-master-r_1

Here my yaml file:

uwsgi:
  socket: 0.0.0.0:2809
  psgi: /path/to/file.psgi
  master:  true
#  processes: 4
  coroae: 1000
  need-app:  true
  log-reopen:  yes 
  worker-reload-mercy: 10800
  vacuum: false                                     # see below for explanation
  master-fifo: /path/to/new/instance.fifo
  master-fifo: /path/to/running/instance.fifo


  if-exists: /path/to/running/instance.fifo
  hook-accepting1-once: writefifo:/path/to/running/instance.fifo q
  endif: 
#race condition point
  hook-accepting1-once: writefifo:/path/to/new/instance.fifo 1P

After #race_condition_point there is 2 ways:

  1. the old master receive 'q' and delete and create fifo-file, after that new master delete and create fifo-file and change master-fifo to 1
  2. the new master delete and create fifo file, change master-fifo to 1 and after this old master delete and create fifo file (because we send 'q' him)

so the good case is 1)
but if the 2) happened, so the old process continue listen fifo file, and after it's death nobody listen fifo file => so we can't do "echo f > /path/to/running/instance.fifo", so to restart service we must kill process and start it again

Is the usage of this yaml is deprecated or it is bug of uwsgi?

@messense

This comment has been minimized.

Show comment
Hide comment
@messense

messense Aug 22, 2017

Same issue.

messense commented Aug 22, 2017

Same issue.

@gmoryes

This comment has been minimized.

Show comment
Hide comment
@gmoryes

gmoryes Aug 23, 2017

meanwhile there is some solution:

uwsgi:
  socket: 0.0.0.0:2809
  psgi: /path/to/file.psgi
  master:  true
  coroae: 1000
  need-app:  true
  log-reopen:  yes 
  worker-reload-mercy: 10800
  vacuum: false                                     # see below for explanation
  master-fifo: /path/to/new/instance.fifo
  master-fifo: /path/to/running/instance.fifo


  if-exists: /path/to/running/instance.fifo
  hook-accepting1-once: writefifo:/path/to/running/instance.fifo q
  endif: 
#race condition point
  hook-accepting1-once: exec:sleep 3 # <-- sleep and wait while old master will exactly dies
  hook-accepting1-once: writefifo:/path/to/new/instance.fifo 1P

gmoryes commented Aug 23, 2017

meanwhile there is some solution:

uwsgi:
  socket: 0.0.0.0:2809
  psgi: /path/to/file.psgi
  master:  true
  coroae: 1000
  need-app:  true
  log-reopen:  yes 
  worker-reload-mercy: 10800
  vacuum: false                                     # see below for explanation
  master-fifo: /path/to/new/instance.fifo
  master-fifo: /path/to/running/instance.fifo


  if-exists: /path/to/running/instance.fifo
  hook-accepting1-once: writefifo:/path/to/running/instance.fifo q
  endif: 
#race condition point
  hook-accepting1-once: exec:sleep 3 # <-- sleep and wait while old master will exactly dies
  hook-accepting1-once: writefifo:/path/to/new/instance.fifo 1P
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment