Traefik return 500 internal error - no 500 on backend #4481
Comments
juliens
added
kind/bug/possible
|
Hi, any update on this? |
|
Add some debug info. It's likely go http keepalive use problem or server setting problem |
|
reason is python api server not set to support http keepalive, traefik default use http keepalive. after set python api server support keepalive, problem fixed. below code will disable keepalive force for debug and testdiff --git a/vendor/github.com/vulcand/oxy/forward/fwd.go b/vendor/github.com/vulcand/oxy/forward/fwd.go
index b8500489..34c90c87 100644
--- a/vendor/github.com/vulcand/oxy/forward/fwd.go
+++ b/vendor/github.com/vulcand/oxy/forward/fwd.go
@@ -534,6 +534,12 @@ func (f *httpForwarder) serveHTTP(w http.ResponseWriter, inReq *http.Request, ct
ModifyResponse: f.modifyResponse,
BufferPool: f.bufferPool,
}
+ if ert, ok := f.roundTripper.(ErrorHandlingRoundTripper); ok {
+ if rt, ok := ert.RoundTripper.(*http.Transport); ok {
+ rt.DisableKeepAlives = true
+ inReq.Close = true
+ }
+ }
if f.log.GetLevel() >= log.DebugLevel {
pw := [`utils.NewProxyWriter(w) |
|
We use uwsgi options --http=:80 and --http-keepalive=1，some random 500 disappear. Random 500 response shows when the uwsgi options is --http-socket=:80 |
|
We stumbled across this as well: https://community.traefik.io/t/observing-5xx-requests-with-content-size-header-0-but-requestcontentsize-0/10584 Would be nice if traefik did not blindly assume keep-alive support. |
|
Here is a link to the scenario https://github.com/languitar/traefik-uwsgi-issue by @languitar that should help to reproduce that issue. |
|
CC @Jeinhaus |
|
Hello, Thanks all for your interest in Traefik! I tried to reproduce the issue and I came to some conclusions. First, the status of keepalive of the webserver is not related: I tried with another web server, with and without keepalive enabled, it worked fine both ways. Tests with custom serverSecond, It seems to be related to having a body in the request (why have a body in your GET request in the reproducible case ?). Tests with custom server & uwsgiThird, you can mitigate Tests with `maxidleconnsperhost=-1`Note that I used vegeta for all test cases( When digging through Traefik's logs, we can find a few errors that explain the
As for the
These WDYT? |
|
Hm, maybe the reproduction case I created also triggers further errors. But in our production setup the sporadic 500 codes have definitely disappeared after enabling keepalive in uwsgi. My suspicion was that traefik doesn't care for the keep alive header and always assumes support. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Hi! I'm Træfiker 🤖 the bot in charge of tidying up the issues. I have to close this one because of its lack of activity 😞 Feel free to re-open it or join our Community Forum. |
Do you want to request a feature or report a bug?
Bug
What did you do?
I use Traefik helm inside GKE,
Google LB -> Traefik -> Uwsgi -> Flask
When having high pressure I see Traefik logs:
500 Internal Server ErrorAt the same time my backend shows this:
[uwsgi-body-read] Error reading 162 bytes. Content-Length: 162 consumed: 0 left: 162 message: Client closed connectionMight be related to:
#615
OR
#3163
What did you expect to see?
200? Server timeout? not 500
What did you see instead?
500 internal server error, even the downstream backend didn't show anything
Output of
traefik version: (What version of Traefik are you using?)im also running using this:
my uwsgi config:
If applicable, please paste the log output in DEBUG level (
--logLevel=DEBUGswitch)I'm afraid to use the debug level log, as it will spam my logging system, any chance to increase logging only on errors?
The text was updated successfully, but these errors were encountered: