New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gitian: add statically built variant of bitcoind/bitcoin-cli for linux build #3914

Merged
merged 1 commit into from Apr 8, 2014

Conversation

Projects
None yet
5 participants
@laanwj
Member

laanwj commented Mar 20, 2014

See #3803 and #3781.

This increases compatibility with older linux distributions.

  2943584 Mar 20 11:43 32/bitcoin-cli
  4376728 Mar 20 11:43 32/bitcoin-cli.static
  7748512 Mar 20 11:43 32/bitcoind
  9140856 Mar 20 11:43 32/bitcoind.static
 12164000 Mar 20 11:43 32/bitcoin-qt
  9740608 Mar 20 11:43 32/test_bitcoin
  8420256 Mar 20 11:43 32/test_bitcoin-qt
  3103728 Mar 20 11:53 64/bitcoin-cli
  4355112 Mar 20 11:53 64/bitcoin-cli.static
  7773680 Mar 20 11:53 64/bitcoind
  8890088 Mar 20 11:53 64/bitcoind.static
 12226032 Mar 20 11:53 64/bitcoin-qt
  9860528 Mar 20 11:53 64/test_bitcoin
  8457776 Mar 20 11:53 64/test_bitcoin-qt

A warning shown during build is:

 warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

Note I have not explicitly disabled hardening/-pie in this build. This makes that the 32-bit version still has a dependency on dynamic libc.so even though it is statically linked:

$ ldd 32/bitcoind.static
    statically linked
$ readelf -a 32/bitcoind.static |grep interpreter
      [Requesting program interpreter: /usr/lib/libc.so.1]

Not sure how much of an issue this is on true 32-bit systems. It needs to be tested. This is not the case for the 64-bit build:

$ ldd 64/bitcoind.static                                                       
    not a dynamic executable
$ readelf -a 64/bitcoind.static |grep interpreter
(nothing)

Edit: v2: -pie now explicitly disabled for 32-bit build as it was unusable otherwise.

Tested on:

  • Debian 7.2 64-bit: OK (tested by me)
  • Ubuntu 10.04.4 LTS 64-bit: OK (tested by me)
  • Ubuntu 10.04.4 LTS 32-bit: OK (tested by me, v2)
  • CentOS 6.5 64bit: OK (tested by alexeft on forums and anton000, v2)
  • CentOS 6.5 32bit: OK (tested by anton000, v2)
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 20, 2014

Member

OK, whereas the 64-bit ones seem to work fine, the 32-bit static executables produced this way are absolutely broken. I have added a commit to disable -pie for 32-bit static executables, and updated the above downoad.

Member

laanwj commented Mar 20, 2014

OK, whereas the 64-bit ones seem to work fine, the 32-bit static executables produced this way are absolutely broken. I have added a commit to disable -pie for 32-bit static executables, and updated the above downoad.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 20, 2014

Member

I'm still very hesitant on this approach. I'd be much more comfortable with picking a minimum required glibc version and wrapping symbols for newer versions.

@laanwj What's the minimum glibc you've got on any of those systems? I'll see if I can work up a POC for the sake of discussion.

Member

theuni commented Mar 20, 2014

I'm still very hesitant on this approach. I'd be much more comfortable with picking a minimum required glibc version and wrapping symbols for newer versions.

@laanwj What's the minimum glibc you've got on any of those systems? I'll see if I can work up a POC for the sake of discussion.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 20, 2014

Member

Glibc 2.11 for Ubuntu 12.10, 2.12 for RH.
But there were also problems with libc++ symbols...

Member

laanwj commented Mar 20, 2014

Glibc 2.11 for Ubuntu 12.10, 2.12 for RH.
But there were also problems with libc++ symbols...

@anton000

This comment has been minimized.

Show comment
Hide comment
@anton000

anton000 Mar 20, 2014

+1 thank you @laanwj for this!
Currently running fine as daemon on CentOS 6.5 64bit :)
Will try to fire up my 32bit centos box to test

edit: works fine on fully updated CentOS 6.5 32bit

+1 thank you @laanwj for this!
Currently running fine as daemon on CentOS 6.5 64bit :)
Will try to fire up my 32bit centos box to test

edit: works fine on fully updated CentOS 6.5 32bit

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 21, 2014

Member

Thanks for testing @anton000

Member

laanwj commented Mar 21, 2014

Thanks for testing @anton000

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Mar 25, 2014

When will this be public? Need it to run 0.9 on debian.

ghost commented Mar 25, 2014

When will this be public? Need it to run 0.9 on debian.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 25, 2014

Member

When it's ready.

In the meantime you can use and help testing the executables here or build your own.

Member

laanwj commented Mar 25, 2014

When it's ready.

In the meantime you can use and help testing the executables here or build your own.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Mar 25, 2014

already did - runs perfectly on debian wheezy 64 bit.

ghost commented Mar 25, 2014

already did - runs perfectly on debian wheezy 64 bit.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 25, 2014

Member

@laanwj May as well go ahead and use this, since it fixes real-world problems in the short-term. I'll keep looking into options for future releases... nothing I've come up with so far has been any more appealing.

Member

theuni commented Mar 25, 2014

@laanwj May as well go ahead and use this, since it fixes real-world problems in the short-term. I'll keep looking into options for future releases... nothing I've come up with so far has been any more appealing.

@BitcoinPullTester

This comment has been minimized.

Show comment
Hide comment
@BitcoinPullTester

BitcoinPullTester Mar 26, 2014

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/ddcd1afc5fdd148cd56f257b40a12f70841bd1b3 for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

Automatic sanity-testing: PASSED, see http://jenkins.bluematt.me/pull-tester/ddcd1afc5fdd148cd56f257b40a12f70841bd1b3 for binaries and test log.
This test script verifies pulls every time they are updated. It, however, dies sometimes and fails to test properly. If you are waiting on a test, please check timestamps to verify that the test.log is moving at http://jenkins.bluematt.me/pull-tester/current/
Contact BlueMatt on freenode if something looks broken.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 26, 2014

Member

Squashed into one commit, removed special-case for 32/64 bit (-static -pie is invalid on all architectures).

Member

laanwj commented Mar 26, 2014

Squashed into one commit, removed special-case for 32/64 bit (-static -pie is invalid on all architectures).

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 27, 2014

Member

@laanwj Please see https://github.com/theuni/bitcoin/tree/libc-compat . That should fix for 2.13. If the approach works as intended, I can move forward with the other symbols as well.

Member

theuni commented Mar 27, 2014

@laanwj Please see https://github.com/theuni/bitcoin/tree/libc-compat . That should fix for 2.13. If the approach works as intended, I can move forward with the other symbols as well.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 28, 2014

Member

Ok, hold on that. glibc behavior changed at some point, so some of my assumptions were wrong on the above. Looking again.

Member

theuni commented Mar 28, 2014

Ok, hold on that. glibc behavior changed at some point, so some of my assumptions were wrong on the above. Looking again.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 28, 2014

Member

Ok, fresh new idea is pushed up to https://github.com/theuni/bitcoin/tree/libc-compat

To use: ./configure --with-glibc-back-compat or --with-glibc-back-compat=X where X is glibc version 2.X.

The idea is to eventually be able to only enable the work-arounds needed for a specific version. For now, there's only one enabled for 2.15.

Member

theuni commented Mar 28, 2014

Ok, fresh new idea is pushed up to https://github.com/theuni/bitcoin/tree/libc-compat

To use: ./configure --with-glibc-back-compat or --with-glibc-back-compat=X where X is glibc version 2.X.

The idea is to eventually be able to only enable the work-arounds needed for a specific version. For now, there's only one enabled for 2.15.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 28, 2014

Member

@theuni so I can point my gitian build at that tree and generate testing executables (to try on the various platforms)? Just have to make sure I pass --with-glibc-back-compat ?

Member

laanwj commented Mar 28, 2014

@theuni so I can point my gitian build at that tree and generate testing executables (to try on the various platforms)? Just have to make sure I pass --with-glibc-back-compat ?

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 28, 2014

Member

@laanwj Yes. Though, the only work-around included for now is for glibc 2.15, so it won't help on anything older than that. If the approach works and seems reasonable, I can begin adding more.

Member

theuni commented Mar 28, 2014

@laanwj Yes. Though, the only work-around included for now is for glibc 2.15, so it won't help on anything older than that. If the approach works and seems reasonable, I can begin adding more.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Mar 29, 2014

Member

@laanwj Hold on that last one as well. I finally hit on a clean way of handling this, and the memcpy for 2.14 is resolved with it, so now the only concern is libstdc++.

If you're ok with waiting a week or so, I'll quit spamming here and just submit a PR that includes gitian descriptors once it's nice and tidy.

Member

theuni commented Mar 29, 2014

@laanwj Hold on that last one as well. I finally hit on a clean way of handling this, and the memcpy for 2.14 is resolved with it, so now the only concern is libstdc++.

If you're ok with waiting a week or so, I'll quit spamming here and just submit a PR that includes gitian descriptors once it's nice and tidy.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Mar 29, 2014

Member

By all means keep 'spamming' to keep us informed as you're working on this, it's nice to know what people are up to.

Member

laanwj commented Mar 29, 2014

By all means keep 'spamming' to keep us informed as you're working on this, it's nice to know what people are up to.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 8, 2014

Member

@theuni Because of the SSL heartbeat vulnerability we'd like to spin a 0.9.1 as soon as possible. It would be useful to have a portable executable in it, so I'd like to pull this now. This solution has been extensively tested. Your symbol-filtering based solution can be used for next release, however it needs testing first.

Member

laanwj commented Apr 8, 2014

@theuni Because of the SSL heartbeat vulnerability we'd like to spin a 0.9.1 as soon as possible. It would be useful to have a portable executable in it, so I'd like to pull this now. This solution has been extensively tested. Your symbol-filtering based solution can be used for next release, however it needs testing first.

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Apr 8, 2014

Member

@laanwj Huh? The heartbleed vuln doesn't affect Bitcoin Core afaik?

Member

luke-jr commented Apr 8, 2014

@laanwj Huh? The heartbleed vuln doesn't affect Bitcoin Core afaik?

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Apr 8, 2014

Member

@luke-jr The vulnerability does not affect the bitcoin protocol but may affect auxilary usage of TLS/HTTPS in RPC SSL and payment request fetching.

Member

laanwj commented Apr 8, 2014

@luke-jr The vulnerability does not affect the bitcoin protocol but may affect auxilary usage of TLS/HTTPS in RPC SSL and payment request fetching.

@luke-jr

This comment has been minimized.

Show comment
Hide comment
@luke-jr

luke-jr Apr 8, 2014

Member

Fetching too? I thought it was just server-side :(

Member

luke-jr commented Apr 8, 2014

Fetching too? I thought it was just server-side :(

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Apr 8, 2014

I think it would be good to release a 0.9.1 with the SSL bug patched.

ghost commented Apr 8, 2014

I think it would be good to release a 0.9.1 with the SSL bug patched.

laanwj added a commit that referenced this pull request Apr 8, 2014

Merge pull request #3914
ddcd1af gitian: add statically built variant of bitcoind/bitcoin-cli (Wladimir J. van der Laan)

@laanwj laanwj merged commit ddcd1af into bitcoin:master Apr 8, 2014

@laanwj laanwj deleted the laanwj:2014_03_statically_built_bitcoind branch Apr 9, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment