-
-
Notifications
You must be signed in to change notification settings - Fork 6.1k
fix(graceful): prevent WaitGroup panic on Ctrl+C shutdown #35578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…repeated shutdowns Fixes go-gitea#35559 Signed-off-by: ita004 <shfahmd001@gmail.com>
I don't think I did twice Ctrl+C in #35559 |
yes, I also reproduced the panic with a single Ctrl+C.. the shutdown flow triggers multiple paths that wait on the same WaitGroup, which causes the reuse panic.... this fix ensures wg.Wait() is only called once, so shutdown completes cleanly without the panic. |
This PR's change is just one more patch to the existing messy code (from Restore Graceful Restarting & Socket Activation (#7274)). The root problem is that the "WaitGroup" is abused. Although @zeripath ever complained that I rewrote his code, but for this case, I think we still need to rewrite the code with a correct design to fix the root problem, just like other complete fixes: logger (#24726), queue (#24505), diff (#19958), etc. It shouldn't add more hacky and fragile patches like #11219. |
thanks for the detailed feedback, that makes sense... |
FYI: there are some related fixes like "Refactor graceful manager, fix misused WaitGroup #29738". |
… reject new work during shutdown
// Shutdown the waitgroup to prevent new connections | ||
// and wait for existing connections to finish | ||
srv.wg.Shutdown() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the current design would work or it can be improved.
doHammer
means that it will stop the server immediately without waiting for any running tasks (for example: existing user request connection).
The graceful
package's design overall looks problematic and it seems unfixable to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for clarifying, i understand what you mean now... that makes sense: doHammer should remain immediate and not wait for any running connections, just forcefully stop the server....
i'll keep this note in mind, and if there’s a future plan to refactor or redesign the graceful manager to address the deeper issues, i'd be happy to participate or help with that whenever it’s appropriate...
What
Guard the
wg.Wait()
call usingsync.Once
to avoid"sync: WaitGroup is reused before previous Wait has returned"
panic during shutdown.Why
Multiple shutdown paths could cause
Wait()
to be invoked more than once,which led to a negative WaitGroup counter and panic when pressing Ctrl+C.
How tested
make backend
)./gitea web
and triggered Ctrl+C — server shuts down cleanly (no panic)Fixes #35559