Support graceful shutdown and hot-deploying #57
Conversation
tcnksm
force-pushed
the
feature/hot-deploying
branch
from
b25fccb
to
c29350e
Compare
tcnksm
force-pushed
the
feature/hot-deploying
branch
from
c29350e
to
ad29d20
Compare
For hot deploy, support `lestrrat/go-server-starter`. Graceful shutdown is done by Go1.8 new feature.
tcnksm
force-pushed
the
feature/hot-deploying
branch
from
ad29d20
to
f286655
Compare
| @@ -25,6 +25,7 @@ type SectionCore struct { | |||
| QueueNum int64 `toml:"queues"` | |||
| NotificationMax int64 `toml:"notification_max"` | |||
| PusherMax int64 `toml:"pusher_max"` | |||
| ShutdownTimeout int64 `toml:"shutdown_timeout"` | |||
There was a problem hiding this comment.
Document this in CONFIGURATION.md.
There was a problem hiding this comment.
And add example config to conf/gaurun.toml.
cmd/gaurun/gaurun.go
Outdated
| gaurun.LogError.Info("wait until queue is empty") | ||
| flushWaitInterval := 1 * time.Second | ||
| for len(gaurun.QueueNotification) > 0 { | ||
| time.Sleep(flushWaitInterval) |
There was a problem hiding this comment.
Could you improve this mechanism by channel-blocking? This implementation intend to wait 1sec in most cases.
There was a problem hiding this comment.
I can not get comment
There was a problem hiding this comment.
Add global wait group !
gaurun/worker.go
Outdated
| @@ -51,6 +60,9 @@ Retry: | |||
|
|
|||
| func pushAsync(pusher func(req RequestGaurunNotification) error, req RequestGaurunNotification, retryMax int, pusherCount *int64) { | |||
| Retry: | |||
| PusherWg.Add(1) | |||
| defer PusherWg.Done() | |||
|
|
|||
There was a problem hiding this comment.
This code is wrong. As pushAsync() is called asynchronously, These functions should be called before calling pushAsync().
FYI
The comment around below will help you.
https://github.com/mercari/gaurun/blob/feature/hot-deploying/gaurun/worker.go#L108
cmd/gaurun/gaurun.go
Outdated
| } | ||
|
|
||
| gaurun.LogError.Info(fmt.Sprintf("wait until queue is empty. Current queue len: %d", queue)) | ||
| time.Sleep(3 * time.Second) |
There was a problem hiding this comment.
Might be 1 * time.Second?
| } | ||
|
|
||
| // Start a goroutine to log number of job queue. | ||
| go func() { |
There was a problem hiding this comment.
This annonymous function has a little deep nest. How about making named function?
There was a problem hiding this comment.
Ummm, I think it's enough (not so deep nest).
For hot deploy, support
lestrrat/go-server-starter. Graceful shutdown is done by Go1.8 new feature.Graceful shutdown is kicked by
server-starterby sendingSIGTERM. If it receivesSIGTERM, first it shutdownnet/httpserver. After shutdown, then it will wait until queue will be empty.Tests are done by same way as this but using
samples/client.go.Not yet done queue flushing test ... (any suggestion how to do this ?)