New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Haproxy 2.0 stuck thread #181
Comments
|
There is nothing abnormal nor unusual in your configuration which could likely explain this. In order to get a core dumped, you will need either to disable the user/group settings or to add the "set-dumpable" directive in the global section. Also, depending on how core dumps are collected on your machine, you may need to disable the chroot or to configure a service to be able to capture them. You can check /proc/sys/kernel/core_pattern to figure if a particular service is involved or if a relative or absolute path is mentioned. |
|
Thank you for your quick response. We've managed to get a coredump/.crash passing it through gdb yields the following: I hope this is usable output. |
|
Hi, Thanks a lot ! It's been very useful, and we can now reproduce it, so we should be enough to fix it soon. |
…by a delimiter When client-side or server-side cookies are parsed, HAProxy enters in an infinite loop if a Cookie/Set-Cookie header value starts by a delimiter (a colon or a semicolon). Depending on the operating system, the service may become degraded, unresponsive, or may trigger haproxy's watchdog causing a service stop or automatic restart. To fix this bug, in the loop parsing the attributes, we must be sure to always skip delimiters once the first attribute-value pair was parsed, empty or not. The credit for the fix goes to Olivier. CVE-2019-14241 was assigned to this bug. This patch fixes the Github issue #181. This patch must be backported to 2.0 and 1.9. However, the patch will have to be adapted.
|
A fix for this was issued in 1.9.9 and 2.0.3. Please upgrade to any of these versions as this bug can be triggered on purpose, making your server potentially vulnerable to an attack. |
|
Closing the issue now. |
|
The CVE indicates that this affects versions 1.4 through 1.9.8 and 2.0.0 through 2.0.2. The home page does say that 1.4 is unmaintained, but will this fix be backported to 1.5, 1.6, 1.7, and 1.8? |
|
The CVE says crap unfortunately, as almost always for CVEs. I took a great care at explaining that only 2.0.0 to 2.0.2 and 1.9.0 to 1.9.8 were vulnerable, and they translated this to "all haproxy up to 2.0.2" then somehow reformulated it as 1.4 to 1.9.8. I responded to the CVE form to get this mess fixed and as expected, there was neither response nor fix. It's confusing for users. I really don't know why I stlil try to get CVEs assigned, I think I never ever saw one done correctly whatever the project that had them... |
|
Thanks for the clarification @wtarreau. |
|
Looks like it still not fixed. Haproxy=2.0.3: backtrace: |
|
Ah that's bad, it's a clear AB/BA locking issue :-( server_warmup() calls pendconn_grab_from_px() under the server's lock, but this one then takes the proxy's lock. In parallel, process_srv_queue() does exactly the opposite. I'm checking if we can just safely switch the two locks in process_srv_queue() to validate the impacts. Thank you! |
|
Let's create a new bug for this one which is completely different or we'll totally mess up backports. |
…by a delimiter When client-side or server-side cookies are parsed, HAProxy enters in an infinite loop if a Cookie/Set-Cookie header value starts by a delimiter (a colon or a semicolon). Depending on the operating system, the service may become degraded, unresponsive, or may trigger haproxy's watchdog causing a service stop or automatic restart. To fix this bug, in the loop parsing the attributes, we must be sure to always skip delimiters once the first attribute-value pair was parsed, empty or not. The credit for the fix goes to Olivier. CVE-2019-14241 was assigned to this bug. This patch fixes the Github issue haproxy#181. This patch must be backported to 2.0 and 1.9. However, the patch will have to be adapted. (cherry picked from commit f0f4238) Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
…by a delimiter When client-side or server-side cookies are parsed, HAProxy enters in an infinite loop if a Cookie/Set-Cookie header value starts by a delimiter (a colon or a semicolon). Depending on the operating system, the service may become degraded, unresponsive, or may trigger haproxy's watchdog causing a service stop or automatic restart. To fix this bug, in the loop parsing the attributes, we must be sure to always skip delimiters once the first attribute-value pair was parsed, empty or not. The credit for the fix goes to Olivier. CVE-2019-14241 was assigned to this bug. This patch fixes the Github issue haproxy#181. This patch must be backported to 2.0 and 1.9. However, the patch will have to be adapted. (cherry picked from commit f0f4238) Signed-off-by: Christopher Faulet <cfaulet@haproxy.com> (cherry picked from commit fc7f52e) Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
Output of
haproxy -vvanduname -aWhat's the configuration?
This is part of the configuration, it contains numerous frontends and backends similar to the one below.
Steps to reproduce the behavior
Actual behavior
HAProxy reloads, with the following logged to syslog:
On our fallback HAProxy instances (running 1.8.14 and earlier) we have not seen this behaviour.
The text was updated successfully, but these errors were encountered: