New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add armhf/aarch64 gitian builds #8188

Merged
merged 2 commits into from Jun 13, 2016

Conversation

Projects
None yet
4 participants
@theuni
Member

theuni commented Jun 10, 2016

As discussed in yesterday's IRC meeting.

Please see the individual commit messages for more detailed info and caveats. Also note:

  • I haven't tested the resulting bins. I'd prefer to get ACKs from testers on native hardware before merge.
  • Not yet tested for determinism, though I have no reason to believe it should cause problems.
@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli
Member

jonasschnelli commented Jun 10, 2016

build: add armhf/aarch64 gitian builds
- create a script to handle split debug. This will also eventually need to check
  targets, and use dsymutil for osx.
- update config.guess/config.sub for bdb for aarch64.
- temporarily disable symbol checks for arm/aarch64
- quit renaming to linux32/linux64 and use the host directly

This also adds a hack to work around an Ubuntu bug in the gcc-multilib package:
https://bugs.launchpad.net/ubuntu/+source/gcc-defaults-armhf-cross/+bug/1347820

The problem is that gcc-multilib conflicts with the aarch toolchain.
gcc-multilib installs a symlink that points
/usr/include/asm -> /usr/include/x86_64-linux-gnu/asm.

Without this link, gcc -m32 can't find asm/errno.h (and others), since
/usr/include/x86_64-linux-gnu isn't in its default include path. But
/usr/include/i386-linux-gnu is (though it doesn't exist on disk).

So work around the problem by linking
/usr/include/i386-linux-gnu/asm -> /usr/include/x86_64-linux-gnu/asm.

The symlink fix is actually quite reasonable, but echoing the password into
sudo is nasty, and should probably be addressed in gitian itself. It makes more
sense to enable passwordless sudo for the build user by default.
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 10, 2016

Member

Going to test this.

Member

laanwj commented Jun 10, 2016

Going to test this.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 10, 2016

Member

Sorry, looks like something is going wrong here locally. Probably a descriptor/tree mismatch.

Member

laanwj commented Jun 10, 2016

Sorry, looks like something is going wrong here locally. Probably a descriptor/tree mismatch.

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Jun 10, 2016

Member

Hmm, I don't see how this is avoiding building Qt? (I haven't tried, just looking at the code changes)

Member

sipa commented Jun 10, 2016

Hmm, I don't see how this is avoiding building Qt? (I haven't tried, just looking at the code changes)

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 10, 2016

Member

ACK 9d25362

682f88fc641256963247e7af3f9a4884f283966fcb5c231f722f689d89a2cee6  bitcoin-0.12.99-aarch64-linux-gnu-debug.tar.gz
b9ffbbc180381c0f258ce73b03907a083787fd9ff60858c43d6cf0f44eed5329  bitcoin-0.12.99-aarch64-linux-gnu.tar.gz
b68184aaca65cf3e35f5404933cd76e64e050b964a1a9f5b2ffe4c9e04a9f0c4  bitcoin-0.12.99-arm-linux-gnueabihf-debug.tar.gz
c1e2fce0759ad9b960541ebe46ffcf82e0a68de2c1f38aedc2750044b45c3d72  bitcoin-0.12.99-arm-linux-gnueabihf.tar.gz
859d36d60686b1df9349c200bb28d6fd05dfa718fce80e1d3130ba1a3fade01e  bitcoin-0.12.99-i686-pc-linux-gnu-debug.tar.gz
3b0d20fdac61ba321eba25a8171bf0a6c859fedf3d85f62bae03949898229822  bitcoin-0.12.99-i686-pc-linux-gnu.tar.gz
7ec01ff5322a0efabe3230c612a9110501ec1da4604c2518b82d104928b249a8  bitcoin-0.12.99-x86_64-linux-gnu-debug.tar.gz
ae713f6b25a5e35362d1717bcb6f0721a9683f445c71cd0b403b7729ab6dfa5d  bitcoin-0.12.99-x86_64-linux-gnu.tar.gz
  • It correctly built Xorg and Qt and other GUI dependencies only for x64, not the ARM archs
  • ARM tarballs contain the expected files:
bitcoin-0.12.99/
bitcoin-0.12.99/bin/
bitcoin-0.12.99/bin/bitcoin-cli
bitcoin-0.12.99/bin/bitcoind
bitcoin-0.12.99/bin/bitcoin-tx
bitcoin-0.12.99/bin/test_bitcoin
bitcoin-0.12.99/include/
bitcoin-0.12.99/include/bitcoinconsensus.h
bitcoin-0.12.99/lib/
bitcoin-0.12.99/lib/libbitcoinconsensus.so
bitcoin-0.12.99/lib/libbitcoinconsensus.so.0
bitcoin-0.12.99/lib/libbitcoinconsensus.so.0.0.0
  • 32-bit ARM Debian 8.5 (i.MX6 Quad/DualLite)
    • test_bitcoin passes
    • bitcoind was able to run it on a node, where it veriefied blocks, connected to peers correctly and stayed up to date with the chain
    • bitcoind succesfully shuts down, no crashes
  • 64-bit ARM Ubuntu 16.04 (ODROID-C2)
    • test_bitcoin passes
    • bitcoind was able to create its data files and do a testnet sync up to block ~70000 (will keep this running)
    • bitcoind succesfully shuts down, no crashes
Member

laanwj commented Jun 10, 2016

ACK 9d25362

682f88fc641256963247e7af3f9a4884f283966fcb5c231f722f689d89a2cee6  bitcoin-0.12.99-aarch64-linux-gnu-debug.tar.gz
b9ffbbc180381c0f258ce73b03907a083787fd9ff60858c43d6cf0f44eed5329  bitcoin-0.12.99-aarch64-linux-gnu.tar.gz
b68184aaca65cf3e35f5404933cd76e64e050b964a1a9f5b2ffe4c9e04a9f0c4  bitcoin-0.12.99-arm-linux-gnueabihf-debug.tar.gz
c1e2fce0759ad9b960541ebe46ffcf82e0a68de2c1f38aedc2750044b45c3d72  bitcoin-0.12.99-arm-linux-gnueabihf.tar.gz
859d36d60686b1df9349c200bb28d6fd05dfa718fce80e1d3130ba1a3fade01e  bitcoin-0.12.99-i686-pc-linux-gnu-debug.tar.gz
3b0d20fdac61ba321eba25a8171bf0a6c859fedf3d85f62bae03949898229822  bitcoin-0.12.99-i686-pc-linux-gnu.tar.gz
7ec01ff5322a0efabe3230c612a9110501ec1da4604c2518b82d104928b249a8  bitcoin-0.12.99-x86_64-linux-gnu-debug.tar.gz
ae713f6b25a5e35362d1717bcb6f0721a9683f445c71cd0b403b7729ab6dfa5d  bitcoin-0.12.99-x86_64-linux-gnu.tar.gz
  • It correctly built Xorg and Qt and other GUI dependencies only for x64, not the ARM archs
  • ARM tarballs contain the expected files:
bitcoin-0.12.99/
bitcoin-0.12.99/bin/
bitcoin-0.12.99/bin/bitcoin-cli
bitcoin-0.12.99/bin/bitcoind
bitcoin-0.12.99/bin/bitcoin-tx
bitcoin-0.12.99/bin/test_bitcoin
bitcoin-0.12.99/include/
bitcoin-0.12.99/include/bitcoinconsensus.h
bitcoin-0.12.99/lib/
bitcoin-0.12.99/lib/libbitcoinconsensus.so
bitcoin-0.12.99/lib/libbitcoinconsensus.so.0
bitcoin-0.12.99/lib/libbitcoinconsensus.so.0.0.0
  • 32-bit ARM Debian 8.5 (i.MX6 Quad/DualLite)
    • test_bitcoin passes
    • bitcoind was able to run it on a node, where it veriefied blocks, connected to peers correctly and stayed up to date with the chain
    • bitcoind succesfully shuts down, no crashes
  • 64-bit ARM Ubuntu 16.04 (ODROID-C2)
    • test_bitcoin passes
    • bitcoind was able to create its data files and do a testnet sync up to block ~70000 (will keep this running)
    • bitcoind succesfully shuts down, no crashes
@jonasschnelli

This comment has been minimized.

Show comment
Hide comment
@jonasschnelli

jonasschnelli Jun 10, 2016

Member

Tested ACK 9d25362 (AARCH only).
Gitian built successful.
Got different hashes then @laanwj, but I did a merge build:

0614d92 Merge branch 'arm-bins' of https://github.com/theuni/bitcoin
9d25362 build: add armhf/aarch64 gitian builds
980e7eb depends: only build qt on linux for x86_64/x86
67db011 Merge #8177: developer notes: updates for C++11
ac8d041 qt: translations update
fde0ac4 Merge #8133: build: Finish up out-of-tree changes
    c68ea35758ed49abf31e97a516b705ca2d8002699e5b8c82370f39b7d5dec7ce  bitcoin-0.12.99-aarch64-linux-gnu-debug.tar.gz
    5b14e326dff5ccf118df8ca6efadbc6ba6bf7042dd82b81a3c27274722c414eb  bitcoin-0.12.99-aarch64-linux-gnu.tar.gz
    1101faad27ea7f39e876b4274f8601468bf2c534b8bb4038ca783af4cccdcc0b  bitcoin-0.12.99-arm-linux-gnueabihf-debug.tar.gz
    bab4c0213d2a76e72517bab413eebf421ab7dcf67ff245a0e99f25f8cb81e593  bitcoin-0.12.99-arm-linux-gnueabihf.tar.gz
    454b5ab2a3edaad347d9a6c12cfadc921fc89be79922695cdb3a240d23ce4be1  bitcoin-0.12.99-i686-pc-linux-gnu-debug.tar.gz
    bb74034acd026637531e87199f9b902fdef9420767c6eb96ad0878d2c492d465  bitcoin-0.12.99-i686-pc-linux-gnu.tar.gz
    733f39ad756212d5b321b439a66328e4d005956cf03c4252c5ff383b21583d32  bitcoin-0.12.99-x86_64-linux-gnu-debug.tar.gz
    2145f811a8a5892087948a5f827f791a199da8d0a5e631aff8148ed5c4b163c6  bitcoin-0.12.99-x86_64-linux-gnu.tar.gz
    bdd273ce240272abeff7980895c75bb25456446ac6ec2cdc8f2e36d7f4c3eb7a  src/bitcoin-0.12.99.tar.gz
- in_manifest: |-
    b07e1e71c64e656612820ea76a377730c4e1c79cd29d432051fd99efe29a8f07  bitcoin-linux-0.13-desc.yml
    git:0614d929ddc55be702d910ff3f3f4481db0a6192 bitcoin

https://bitcoin.jonasschnelli.ch/pulls/8188/bitcoin-linux-0.13-build.assert

Successfully ran test_bitcoin and bitcoind on my Pine64 2GB.

ubuntu@pine64:~$ cat /proc/cpuinfo 
Processor   : AArch64 Processor rev 4 (aarch64)
processor   : 0
processor   : 1
processor   : 2
processor   : 3
Features    : fp asimd aes pmull sha1 sha2 crc32 
CPU implementer : 0x41
CPU architecture: AArch64
CPU variant : 0x0
CPU part    : 0xd03
CPU revision    : 4
ubuntu@pine64:~$ ./test_bitcoin 
Running 204 test cases...

*** No errors detected
ubuntu@pine64:~$ ./bitcoind --dbcache=300 --datadir=/disk2/fullnode/ --prune=550 --printtoconsole
2016-06-10 15:21:26 
2016-06-10 15:21:26 Bitcoin version v0.12.99.0-0614d92
2016-06-10 15:21:26 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
---snip
2016-06-10 15:22:57 init message: Verifying blocks...
2016-06-10 15:22:57 Verifying last 288 blocks at level 3
2016-06-10 15:22:57 [0%]...[10%]...[20%]...[30%]...[40%]...[50%]...[60%]...[70%]...[80%]...[90%]...[DONE].
2016-06-10 15:24:19 No coin database inconsistencies in last 42 blocks (43555 transactions)
2016-06-10 15:24:19  block index          171850ms
2016-06-10 15:24:19 init message: Loading wallet...
2016-06-10 15:24:19 nFileVersion = 129900
2016-06-10 15:24:19 Keys: 101 plaintext, 0 encrypted, 101 w/ metadata, 101 total
2016-06-10 15:24:20  wallet                  742ms
2016-06-10 15:24:20 Unsetting NODE_NETWORK on prune mode
2016-06-10 15:24:20 init message: Pruning blockstore...
2016-06-10 15:24:20 mapBlockIndex.size() = 415682
2016-06-10 15:24:20 nBestHeight = 414857
2016-06-10 15:24:20 setKeyPool.size() = 100
2016-06-10 15:24:20 mapWallet.size() = 0
2016-06-10 15:24:20 mapAddressBook.size() = 1
2016-06-10 15:24:20 init message: Loading addresses...
2016-06-10 15:24:20 torcontrol thread start
2016-06-10 15:24:21 Loaded 57944 addresses from peers.dat  1023ms
2016-06-10 15:24:21 init message: Loading banlist...
2016-06-10 15:24:21 dnsseed thread start
2016-06-10 15:24:21 net thread start
2016-06-10 15:24:21 msghand thread start
2016-06-10 15:24:21 opencon thread start
2016-06-10 15:24:21 addcon thread start
2016-06-10 15:24:21 init message: Done loading
Member

jonasschnelli commented Jun 10, 2016

Tested ACK 9d25362 (AARCH only).
Gitian built successful.
Got different hashes then @laanwj, but I did a merge build:

0614d92 Merge branch 'arm-bins' of https://github.com/theuni/bitcoin
9d25362 build: add armhf/aarch64 gitian builds
980e7eb depends: only build qt on linux for x86_64/x86
67db011 Merge #8177: developer notes: updates for C++11
ac8d041 qt: translations update
fde0ac4 Merge #8133: build: Finish up out-of-tree changes
    c68ea35758ed49abf31e97a516b705ca2d8002699e5b8c82370f39b7d5dec7ce  bitcoin-0.12.99-aarch64-linux-gnu-debug.tar.gz
    5b14e326dff5ccf118df8ca6efadbc6ba6bf7042dd82b81a3c27274722c414eb  bitcoin-0.12.99-aarch64-linux-gnu.tar.gz
    1101faad27ea7f39e876b4274f8601468bf2c534b8bb4038ca783af4cccdcc0b  bitcoin-0.12.99-arm-linux-gnueabihf-debug.tar.gz
    bab4c0213d2a76e72517bab413eebf421ab7dcf67ff245a0e99f25f8cb81e593  bitcoin-0.12.99-arm-linux-gnueabihf.tar.gz
    454b5ab2a3edaad347d9a6c12cfadc921fc89be79922695cdb3a240d23ce4be1  bitcoin-0.12.99-i686-pc-linux-gnu-debug.tar.gz
    bb74034acd026637531e87199f9b902fdef9420767c6eb96ad0878d2c492d465  bitcoin-0.12.99-i686-pc-linux-gnu.tar.gz
    733f39ad756212d5b321b439a66328e4d005956cf03c4252c5ff383b21583d32  bitcoin-0.12.99-x86_64-linux-gnu-debug.tar.gz
    2145f811a8a5892087948a5f827f791a199da8d0a5e631aff8148ed5c4b163c6  bitcoin-0.12.99-x86_64-linux-gnu.tar.gz
    bdd273ce240272abeff7980895c75bb25456446ac6ec2cdc8f2e36d7f4c3eb7a  src/bitcoin-0.12.99.tar.gz
- in_manifest: |-
    b07e1e71c64e656612820ea76a377730c4e1c79cd29d432051fd99efe29a8f07  bitcoin-linux-0.13-desc.yml
    git:0614d929ddc55be702d910ff3f3f4481db0a6192 bitcoin

https://bitcoin.jonasschnelli.ch/pulls/8188/bitcoin-linux-0.13-build.assert

Successfully ran test_bitcoin and bitcoind on my Pine64 2GB.

ubuntu@pine64:~$ cat /proc/cpuinfo 
Processor   : AArch64 Processor rev 4 (aarch64)
processor   : 0
processor   : 1
processor   : 2
processor   : 3
Features    : fp asimd aes pmull sha1 sha2 crc32 
CPU implementer : 0x41
CPU architecture: AArch64
CPU variant : 0x0
CPU part    : 0xd03
CPU revision    : 4
ubuntu@pine64:~$ ./test_bitcoin 
Running 204 test cases...

*** No errors detected
ubuntu@pine64:~$ ./bitcoind --dbcache=300 --datadir=/disk2/fullnode/ --prune=550 --printtoconsole
2016-06-10 15:21:26 
2016-06-10 15:21:26 Bitcoin version v0.12.99.0-0614d92
2016-06-10 15:21:26 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
---snip
2016-06-10 15:22:57 init message: Verifying blocks...
2016-06-10 15:22:57 Verifying last 288 blocks at level 3
2016-06-10 15:22:57 [0%]...[10%]...[20%]...[30%]...[40%]...[50%]...[60%]...[70%]...[80%]...[90%]...[DONE].
2016-06-10 15:24:19 No coin database inconsistencies in last 42 blocks (43555 transactions)
2016-06-10 15:24:19  block index          171850ms
2016-06-10 15:24:19 init message: Loading wallet...
2016-06-10 15:24:19 nFileVersion = 129900
2016-06-10 15:24:19 Keys: 101 plaintext, 0 encrypted, 101 w/ metadata, 101 total
2016-06-10 15:24:20  wallet                  742ms
2016-06-10 15:24:20 Unsetting NODE_NETWORK on prune mode
2016-06-10 15:24:20 init message: Pruning blockstore...
2016-06-10 15:24:20 mapBlockIndex.size() = 415682
2016-06-10 15:24:20 nBestHeight = 414857
2016-06-10 15:24:20 setKeyPool.size() = 100
2016-06-10 15:24:20 mapWallet.size() = 0
2016-06-10 15:24:20 mapAddressBook.size() = 1
2016-06-10 15:24:20 init message: Loading addresses...
2016-06-10 15:24:20 torcontrol thread start
2016-06-10 15:24:21 Loaded 57944 addresses from peers.dat  1023ms
2016-06-10 15:24:21 init message: Loading banlist...
2016-06-10 15:24:21 dnsseed thread start
2016-06-10 15:24:21 net thread start
2016-06-10 15:24:21 msghand thread start
2016-06-10 15:24:21 opencon thread start
2016-06-10 15:24:21 addcon thread start
2016-06-10 15:24:21 init message: Done loading
@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Jun 11, 2016

Member

Concept ACK

Member

sipa commented Jun 11, 2016

Concept ACK

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Jun 13, 2016

Member

odroid64 testnet node synced to height 869998 over the weekend. All seems to work fine. No apparent difference in behavior or performance from natively-built.

Member

laanwj commented Jun 13, 2016

odroid64 testnet node synced to height 869998 over the weekend. All seems to work fine. No apparent difference in behavior or performance from natively-built.

@laanwj laanwj merged commit 9d25362 into bitcoin:master Jun 13, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

laanwj added a commit that referenced this pull request Jun 13, 2016

Merge #8188: Add armhf/aarch64 gitian builds
9d25362 build: add armhf/aarch64 gitian builds (Cory Fields)
980e7eb depends: only build qt on linux for x86_64/x86 (Cory Fields)

This was referenced Jun 13, 2016

laanwj added a commit to laanwj/bitcoin that referenced this pull request Jun 21, 2016

doc: Mention ARM executables in release process
Mention ARM executables in the release process documentation
(these were introduced in #8188).
As well as that Linux tarballs have changed name to contain an
architecture tuple, instead of `linux32`/`linux64`.
Also mention that `-debug` files should not be uploaded (these were
introduced in #8167).

codablock added a commit to codablock/dash that referenced this pull request Sep 16, 2017

Merge #8188: Add armhf/aarch64 gitian builds
9d25362 build: add armhf/aarch64 gitian builds (Cory Fields)
980e7eb depends: only build qt on linux for x86_64/x86 (Cory Fields)

codablock added a commit to codablock/dash that referenced this pull request Sep 19, 2017

Merge #8188: Add armhf/aarch64 gitian builds
9d25362 build: add armhf/aarch64 gitian builds (Cory Fields)
980e7eb depends: only build qt on linux for x86_64/x86 (Cory Fields)

schinzelh added a commit to dashpay/dash that referenced this pull request Oct 19, 2017

doc: Mention ARM executables in release process
Mention ARM executables in the release process documentation
(these were introduced in #8188).
As well as that Linux tarballs have changed name to contain an
architecture tuple, instead of `linux32`/`linux64`.
Also mention that `-debug` files should not be uploaded (these were
introduced in #8167).

schinzelh added a commit to dashpay/dash that referenced this pull request Oct 19, 2017

doc: Mention ARM executables in release process
Mention ARM executables in the release process documentation
(these were introduced in #8188).
As well as that Linux tarballs have changed name to contain an
architecture tuple, instead of `linux32`/`linux64`.
Also mention that `-debug` files should not be uploaded (these were
introduced in #8167).

@schinzelh schinzelh referenced this pull request Oct 19, 2017

Closed

[WIP] Update build system to Bitcoin 0.13.2 #1692

22 of 24 tasks complete

codablock added a commit to codablock/dash that referenced this pull request Dec 22, 2017

Merge #8188: Add armhf/aarch64 gitian builds
9d25362 build: add armhf/aarch64 gitian builds (Cory Fields)
980e7eb depends: only build qt on linux for x86_64/x86 (Cory Fields)

@Sjors Sjors referenced this pull request Jun 18, 2018

Closed

QT cross compile to ARM #13495

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment