Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bitcoin-qt hangs in futex() / __cxa_guard_acquire() #14359

Closed
mbuesch opened this issue Sep 30, 2018 · 15 comments

Comments

@mbuesch
Copy link

commented Sep 30, 2018

When I run bitcoin-qt it does not properly start up and hangs in futex():

The last calls shown in strace are:

futex(0x7f3f7bfd77e4, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7f3f7d812c60, FUTEX_WAIT, 65792, NULL

gdb shows:

(gdb) bt
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x00007ffff5e26d2f in __cxa_guard_acquire () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x00007ffff7e06dcc in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5
#3  0x00007ffff7e0b706 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5
#4  0x00007ffff7e2af21 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5
#5  0x00007ffff7e32080 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5
#6  0x00007ffff7df1c67 in QSslCertificate::QSslCertificate(QByteArray const&, QSsl::EncodingFormat) ()                                                                                        
   from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5
#7  0x00007ffff7e07fe6 in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5
#8  0x00007ffff7dfb09d in QSslConfiguration::defaultConfiguration() () from /usr/lib/x86_64-linux-gnu/libQt5Network.so.5                                                                      
#9  0x000055555564841f in main () at qt/bitcoin.cpp:579
#10 0x00007ffff5a49b17 in __libc_start_main (main=0x555555648380 <main>, argc=1, argv=0x7fffffffdcc8, init=<optimized out>, fini=<optimized out>,                                             
    rtld_fini=<optimized out>, stack_end=0x7fffffffdcb8) at ../csu/libc-start.c:310
#11 0x0000555555657dca in _start () at /usr/include/c++/8/ext/new_allocator.h:116

I understand that this might also be a problem in Qt rather than bitcoin-qt.
But other Qt applications do not show this problem. So I'd be happy to get any hint to debug this further.

Thanks!

@mbuesch

This comment has been minimized.

Copy link
Author

commented Sep 30, 2018

This works around the issue:

diff --git a/src/qt/bitcoin.cpp b/src/qt/bitcoin.cpp
index 1e950e268..5cdcc52c5 100644
--- a/src/qt/bitcoin.cpp
+++ b/src/qt/bitcoin.cpp
@@ -573,7 +573,7 @@ int main(int argc, char *argv[])
 #ifdef Q_OS_MAC
     QApplication::setAttribute(Qt::AA_DontShowIconsInMenus);
 #endif
-#if QT_VERSION >= 0x050500
+#if QT_VERSION >= 0x050500 && 0
     // Because of the POODLE attack it is recommended to disable SSLv3 (https://disablessl3.com/),
     // so set SSL protocols to TLS1.0+.
     QSslConfiguration sslconf = QSslConfiguration::defaultConfiguration();

With this patch bitcoin-qt does not deadlock anymore and starts properly.

I'm wondering what the correct fix would be.
Any help is appreciated.
Thanks.

@fanquake fanquake added the GUI label Sep 30, 2018

@fanquake

This comment has been minimized.

Copy link
Member

commented Sep 30, 2018

@mbuesch What OS is this running on? Are you using Qt from depends or installed via a package manager (if so what version)?

Possibly related to #14273.

@real-or-random

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2018

I'm pretty sure that this is exactly #14273. I use Arch Linux and have the same issue, i.e., this is the same OS, same symptoms, same line of code as #14273.

I haven't had time to verify the workaround, though.

@mbuesch

This comment has been minimized.

Copy link
Author

commented Oct 1, 2018

Yes, this is indeed a duplicate of #14273.
I am running Debian Sid with libqt5network5:amd64 5.11.1+dfsg-9

I can also confirm that the sslLibraryVersionString() call from #14273 fixes the problem.
Can we have that change to bitcoin-core?
I understand that this is not a bitcoin-core bug, but rather a Qt bug. But the workaround seems simple enough. It might not be an option for people to update Qt easily.

diff --git a/src/qt/bitcoin.cpp b/src/qt/bitcoin.cpp
index 1e950e268..38e412fcd 100644
--- a/src/qt/bitcoin.cpp
+++ b/src/qt/bitcoin.cpp
@@ -576,6 +576,7 @@ int main(int argc, char *argv[])
 #if QT_VERSION >= 0x050500
     // Because of the POODLE attack it is recommended to disable SSLv3 (https://disablessl3.com/),
     // so set SSL protocols to TLS1.0+.
+    QSslSocket::sslLibraryVersionString(); // Call to sslLibraryVersionString is a workaround for a Qt bug. See bitcoin issue #14273 / #14359
     QSslConfiguration sslconf = QSslConfiguration::defaultConfiguration();
     sslconf.setProtocol(QSsl::TlsV1_0OrLater);
     QSslConfiguration::setDefaultConfiguration(sslconf);

If that change is acceptable, I'll send a pull request.

Thanks for your comments.

@MarcoFalke

This comment has been minimized.

Copy link
Member

commented Oct 1, 2018

I am not sure if this is already filed against qt 5.11, but we should wait for a fix upstream or at least file it against upstream, if not done already.

@zquestz

This comment has been minimized.

Copy link

commented Oct 3, 2018

@mbuesch thanks a bunch for the patch. Worked for me. =)

@mbuesch

This comment has been minimized.

Copy link
Author

commented Oct 3, 2018

@MarcoFalke I did not find an upstream report/fix, yet. But I can surely report this to Qt.
But I think the workaround should be applied to bitcoin-core regardless of that ASAP.
We are not really in control of what Qt versions are being used. And people will use that buggy version for some time to come. This bug has already been around for a couple of weeks, so I don't really see this buggy version to go away tomorrow.

This is a very non-intrusive workaround that I think we should have it in bitcoin-core to get stuff working again. This completely renders bitcoin-qt unusable, after all, if that Qt version is installed on the user's system.

@MarcoFalke

This comment has been minimized.

Copy link
Member

commented Oct 4, 2018

It would still be nice to know why this workaround works or if there is a more suitable workaround.

@real-or-random

This comment has been minimized.

Copy link
Contributor

commented Oct 4, 2018

Since this is a concurrency issue, the workaround could be spurious (e.g., just change the timing to avoid the deadlock). At least we don't know without talking to Qt. But yes, I think we should apply the workaround because it seems to help and it's really non-intrusive.

@real-or-random

This comment has been minimized.

Copy link
Contributor

commented Oct 4, 2018

It turns out that #6384 which introduced the code to disable SSLv3 was redundant because this is the default in Qt anyway due to new default flag SecureProtocols introduced in qt/qtbase@3fd2d9e, which landed in Qt 5.4.

I guess #6384 was the result of a misunderstanding: @laanwj had opened https://bugreports.qt.io/browse/QTBUG-43168 to ask for a flag to disable SSLv3 two days before Qt 5.4 was tagged, so maybe he missed the new SecureProtocols default in Qt 5.4. But the Qt developers still picked up his suggestions to include more flags for specific SSL/TLS protocol versions and implemented it in Qt 5.5. Then @laanwj created #6384.

At this time, OpenSSL was the only TLS backend in Qt. Now there are more backend, and I checked that they all disable SSLv3 in the default configuration with SecureProtocols.

So I think the simplest way to get rid of this issue here is to remove the code disabling SSLv3, because Qt 5.4 enforces this anyway without any configuration tweaks.

@mbuesch

This comment has been minimized.

Copy link
Author

commented Oct 5, 2018

I created a Qt issue:
https://bugreports.qt.io/browse/QTBUG-70956

@real-or-random

This comment has been minimized.

Copy link
Contributor

commented Oct 5, 2018

Hm, I suspect that is in fact our issue. We create an OpenSSL singleton in util.cpp:
The bug disappears when I remove the call to OPENSSL_no_config() here:

OPENSSL_no_config();
.

That means that there is probably a race condition in general and we should delay the Qt stuff until the singleton is created.

@mbuesch

This comment has been minimized.

Copy link
Author

commented Oct 5, 2018

Please also see the full backtrace with debugging symbols I posted here: https://bugreports.qt.io/secure/attachment/77027/full-bt.txt
This looks like a recursive locking of Qt's internal global data structure.

@mbuesch

This comment has been minimized.

Copy link
Author

commented Oct 5, 2018

@real-or-random Thanks, that was the key. I created a minimal example here: https://bugreports.qt.io/secure/attachment/77028/77028_qttest.tar.gz

@MarcoFalke MarcoFalke added the Upstream label Oct 5, 2018

@real-or-random

This comment has been minimized.

Copy link
Contributor

commented Oct 8, 2018

Upstream has responded and essentially postponed the issue. They note that this happens only with OpenSSL 1.1.1, not with 1.1.0.

real-or-random added a commit to real-or-random/bitcoin that referenced this issue Oct 9, 2018

qt: Revert "Force TLS1.0+ for SSL connections"
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956

laanwj added a commit that referenced this issue Oct 16, 2018

Merge #14403: qt: Revert "Force TLS1.0+ for SSL connections"
7d173c4 qt: Revert "Force TLS1.0+ for SSL connections" (Tim Ruffing)

Pull request description:

  This reverts commit 15e26a6, whose
  purpose was to tweak the Qt configuration to force TLS, i.e., to
  disable SSLv3, in Qt versions >= 5.5. However, the default behavior
  of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
  tweak is redundant.

  With Qt 5.11.2, the configuration tweak is not only redundant but in
  fact provokes a deadlock due to a bug in Qt 5.11.2. Since the deadlock
  occurs at the early startup stage of bitcoin-qt, it renders bitcoin-qt
  entirely non-functional when compiled against Qt 5.11.2 (and maybe
  other Qt versions).

  Fixes #14359.

  [1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84

Tree-SHA512: 9dd86557b8d265dfa56592924778a736590f2e6a0b2acf77d4f9f4200206a9edaa79b144b0085ea59ac0cc1bc66d9740402fd02f9298ff74c8d6f526f3f725d6

laanwj added a commit to laanwj/bitcoin that referenced this issue Nov 6, 2018

qt: Revert "Force TLS1.0+ for SSL connections"
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956

Github-Pull: bitcoin#14403
Rebased-From: 7d173c4
Tree-SHA512: 71a34b13202c834c5ca73bcb9b70efff26c34e1aac3b954f098620b62c2be53a8e319929c4764a5b5cc5d0dd163ff70f4eb3a4f1f608363b7d23d1b16b25ddc7

jfhk added a commit to jfhk/bitcoin that referenced this issue Nov 14, 2018

qt: Revert "Force TLS1.0+ for SSL connections"
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956

HashUnlimited pushed a commit to HashUnlimited/chaincoin that referenced this issue Nov 26, 2018

qt: Revert "Force TLS1.0+ for SSL connections"
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956

lateminer added a commit to lateminer/bitcoin that referenced this issue Nov 26, 2018

qt: Revert "Force TLS1.0+ for SSL connections"
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956

AmirAbrams added a commit to AmirAbrams/bitcoin that referenced this issue Jun 18, 2019

QT: Revert "Force TLS1.0+ for SSL connections"
See: bitcoin@0242b5a
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956

UdjinM6 added a commit to dashpay/dash that referenced this issue Jun 18, 2019

QT: Revert "Force TLS1.0+ for SSL connections" (#2985)
See: bitcoin@0242b5a
This reverts commit 15e26a6, whose
purpose was to tweak the Qt configuration to force TLS, i.e., to
disable SSLv3, in Qt versions >= 5.5. However, the default behavior
of Qt >= 5.4 is to disable SSLv3 anyway [1], so the configuration
tweak is redundant.

With Qt 5.11.2, the configuration tweak is not only redundant but in
fact provokes a deadlock (bitcoin#14359) due to Qt 5.11.2 being incompatible
with OpenSSL 1.1.1 [2]. Since the deadlock occurs at the early startup
stage of bitcoin-qt, it renders bitcoin-qt entirely non-functional
when compiled against OpenSSL 1.1.1 and Qt 5.11.2 (and possible future
combinations of OpenSSL and Qt versions).

This commit fixes bitcoin#14359 by removing the redundant code.

[1] https://code.qt.io/cgit/qt/qtbase.git/commit/?id=3fd2d9eff8c1f948306ee5fbfe364ccded1c4b84
[2] https://bugreports.qt.io/browse/QTBUG-70956
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.