-
Notifications
You must be signed in to change notification settings - Fork 994
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TSAN] pthread_atfork callbacks are causing deadlocks. #1116
Comments
For me it seems to be very similar to #788, but I'm not sure. I.e. we killed the binary during data race report. |
I think I see a potential problem in the stack trace. In the fork interceptor tsan locks report_mtx:
It was assumed that fork does not call any instrumented user code. But in this case fork calls folly::detail::(anonymous namespace)::AtForkList::child() which is instrumented and triggers a race, which tries to lock report_mtx again, which deadlocks. There may be several potential solutions. |
EDIT: Ignore this comment. Incorrectly read the code. |
I am seeing this issue as well. @dvyukov do you have any ideas on how to proceed? Is there any data I could provide to help? |
Maybe this llvm/llvm-project@be41a98 fixes it? Looks similar. |
Disable a few tests under CONFIG_TSAN, which run into a known TSan issue that results in a hang. google/sanitizers#1116 The disabled tests under TSan include all the qtests as well as the test-char, test-qga, and test-qdev-global-props. Signed-off-by: Robert Foley <robert.foley@linaro.org> Message-Id: <20200605173422.1490-14-robert.foley@linaro.org>
Disable a few tests under CONFIG_TSAN, which run into a known TSan issue that results in a hang. google/sanitizers#1116 The disabled tests under TSan include all the qtests as well as the test-char, test-qga, and test-qdev-global-props. Signed-off-by: Robert Foley <robert.foley@linaro.org>
Disable a few tests under CONFIG_TSAN, which run into a known TSan issue that results in a hang. google/sanitizers#1116 The disabled tests under TSan include all the qtests as well as the test-char, test-qga, and test-qdev-global-props. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Emilio G. Cota <cota@braap.org>
Disable a few tests under CONFIG_TSAN, which run into a known TSan issue that results in a hang. google/sanitizers#1116 The disabled tests under TSan include all the qtests as well as the test-char, test-qga, and test-qdev-global-props. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Emilio G. Cota <cota@braap.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200609200738.445-14-robert.foley@linaro.org>
Disable a few tests under CONFIG_TSAN, which run into a known TSan issue that results in a hang. google/sanitizers#1116 The disabled tests under TSan include all the qtests as well as the test-char, test-qga, and test-qdev-global-props. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Emilio G. Cota <cota@braap.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200609200738.445-14-robert.foley@linaro.org> Message-Id: <20200612190237.30436-17-alex.bennee@linaro.org>
Disable a few tests under CONFIG_TSAN, which run into a known TSan issue that results in a hang. google/sanitizers#1116 The disabled tests under TSan include all the qtests as well as the test-char, test-qga, and test-qdev-global-props. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Emilio G. Cota <cota@braap.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200609200738.445-14-robert.foley@linaro.org> Message-Id: <20200612190237.30436-17-alex.bennee@linaro.org>
google/sanitizers#1116 ref:a150ea15f1fcb6baf3d4ad282698e73b55bbb587
Summary: Because of google/sanitizers#1116, any races detected during pthread_atfork deadlock the process. For forks issued from folly we can use an instrumented fork version to minimize the amount of code running via pthread_atfork. Reviewed By: yfeldblum Differential Revision: D23281093 fbshipit-source-id: fd4d3bf06b7992ff314f631ed899854a8b3f6c4b
Summary: Because of google/sanitizers#1116, any races detected during pthread_atfork deadlock the process. For forks issued from folly we can use an instrumented fork version to minimize the amount of code running via pthread_atfork. Reviewed By: yfeldblum Differential Revision: D23281093 fbshipit-source-id: fd4d3bf06b7992ff314f631ed899854a8b3f6c4b
google/sanitizers#1116 ref:a150ea15f1fcb6baf3d4ad282698e73b55bbb587
google/sanitizers#1116 ref:a150ea15f1fcb6baf3d4ad282698e73b55bbb587
Some context, we have an alarm which kills the binary after 240s with std::exit(1), but it seems that the binary is being stuck during this kill.
It looks very suspicious that we were stuck in trying to acquire some lock in ReportRace, maybe it is even a reason why we hit a timeout, see trace:
PS: Compiled with clang 8.0.
The text was updated successfully, but these errors were encountered: