[Merged by Bors] - Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4) #1497

paulhauner · 2020-08-10T06:01:10Z

Issue Addressed

NA

Proposed Changes

Adds support for using the cross project to produce cross-compiled binaries using Docker images.

Provides quite clean and simple cross-compiles cause all the complexity is hidden in Dockerfiles. It does require you to be in the docker group though.

Details

Adds shortcut commands to Makefile
Ensures reqwest and discv5 use vendored openssl libs (i.e., static not shared).
Switches to a commit of blst that has a renamed C function to avoid a collision with openssl (upstream issue: Symbol collision with openssl supranational/blst#21).
Updates ring to the latest satisfiable version, since an earlier version was causing issues with cross.
Off-topic, but adds extra message about Windows support as suggested by Discord user.

Additional Info

~~Blocked on [Merged by Bors] - Upgrade logs #1495~~
There are no tests in CI for this yet for a few reasons:
- I'm hesitant to add more long-running tasks.
- Short-term bitrot should be avoided since we'll use it each release.
- In the long term I think it would be good to automate binary creation on a release.
I observed the binaries increase in size from 50mb to 52mb after these changes.

Squashed commit of the following: commit fe3ecc0 Author: Age Manning <Age@AgeManning.com> Date: Mon Aug 10 13:59:29 2020 +1000 Report port overlapping and failure to bind to user commit d3dc6a9 Author: Age Manning <Age@AgeManning.com> Date: Mon Aug 10 12:01:27 2020 +1000 Upgrade address conflict logs

…-compile

michaelsproul

Just a few typos that need fixing before merge. We can address the remaining portability issues separately.

michaelsproul · 2020-08-11T00:09:03Z

book/src/become-a-validator-source.md

-> Note:  if you're not familiar with Rust or you'd like more detailed instructions, see our  [installation guide](./installation.md).
+> Notes:
+>   - If you're not familiar with Rust or you'd like more detailed instructions, see our  [installation guide](./installation.md).
+>   - Windows is presently only supported via [WSL](target/aarch64-unknown-linux-gnu).


typo in link target, should be WSL not aarch64

michaelsproul · 2020-08-11T00:09:27Z

book/src/cross-compiling.md

@@ -0,0 +1,41 @@
+# Cross-compiling
+
+Lighthouse supports cross-compiling, allowing users to run a binary one


typo: "on one platform"?

michaelsproul · 2020-08-11T00:44:26Z

book/src/cross-compiling.md

+	for most users).
+- `build-x86_64-portable`: builds a version x86_64 processors which avoids
+	using some modern CPU instructions that might cause an "illegal
+	instruction" error on older CPUs.


There's still an issue with pre-SSSE3 CPUs. I've opened #1504 to track it

michaelsproul · 2020-08-11T00:44:28Z

book/src/cross-compiling.md

+
+The `Makefile` in the project contains four targets for cross-compiling:
+
+- `build-x86_64`: builds an optimized version for x86_64 processors (suitable


I'm surprised that this produces a portable binary, but it seems to (I ran the v0.2.2 release on my laptop just fine). The binary must still be compiled with -march=native inside the Docker image, which would tailor it to the host machine, so I'm not sure if we just got lucky with the choice of host. I've opened #1505 to track this.

paulhauner · 2020-08-11T05:13:49Z

Thanks @michaelsproul, I believe I added all the changes required for now.

michaelsproul

bors r+

## Issue Addressed NA ## Proposed Changes Adds support for using the [`cross`](https://github.com/rust-embedded/cross) project to produce cross-compiled binaries using Docker images. Provides quite clean and simple cross-compiles cause all the complexity is hidden in Dockerfiles. It does require you to be in the `docker` group though. ## Details - Adds shortcut commands to `Makefile` - Ensures `reqwest` and `discv5` use vendored openssl libs (i.e., static not shared). - Switches to a [commit](sigp/blst@284f705) of blst that has a renamed C function to avoid a collision with openssl (upstream issue: supranational/blst#21). - Updates `ring` to the latest satisfiable version, since an earlier version was causing issues with `cross`. - Off-topic, but adds extra message about Windows support as suggested by Discord user. ## Additional Info - ~~Blocked on #1495~~ - There are no tests in CI for this yet for a few reasons: - I'm hesitant to add more long-running tasks. - Short-term bitrot should be avoided since we'll use it each release. - In the long term I think it would be good to automate binary creation on a release. - I observed the binaries increase in size from 50mb to 52mb after these changes.

bors · 2020-08-11T06:04:56Z

Pull request successfully merged into master.

Build succeeded:

## Issue Addressed NA ## Proposed Changes Adds support for using the [`cross`](https://github.com/rust-embedded/cross) project to produce cross-compiled binaries using Docker images. Provides quite clean and simple cross-compiles cause all the complexity is hidden in Dockerfiles. It does require you to be in the `docker` group though. ## Details - Adds shortcut commands to `Makefile` - Ensures `reqwest` and `discv5` use vendored openssl libs (i.e., static not shared). - Switches to a [commit](sigp/blst@284f705) of blst that has a renamed C function to avoid a collision with openssl (upstream issue: supranational/blst#21). - Updates `ring` to the latest satisfiable version, since an earlier version was causing issues with `cross`. - Off-topic, but adds extra message about Windows support as suggested by Discord user. ## Additional Info - ~~Blocked on #1495~~ - There are no tests in CI for this yet for a few reasons: - I'm hesitant to add more long-running tasks. - Short-term bitrot should be avoided since we'll use it each release. - In the long term I think it would be good to automate binary creation on a release. - I observed the binaries increase in size from 50mb to 52mb after these changes.

paulhauner added 3 commits August 10, 2020 15:42

First working cross-compile

66065ab

Rename build target

f977ea9

paulhauner added work-in-progress PR is a work-in-progress blocked labels Aug 10, 2020

paulhauner added 3 commits August 10, 2020 19:26

First working cross-compile

e9ed847

Rename build target

0a68ec0

Add option for portable cross builds

48fd312

paulhauner force-pushed the cross-compile branch from 912134f to 48fd312 Compare August 10, 2020 09:28

Fix ARM cross commands

9419bea

paulhauner removed the blocked label Aug 10, 2020

paulhauner added 3 commits August 10, 2020 21:25

Remove unused Cross.toml

bedf6d6

Merge branch 'cross-compile' of github.com:sigp/lighthouse into cross…

867e829

…-compile

Remove old Cross.toml

814d0fd

paulhauner added low-hanging-fruit Easy to resolve, get it before someone else does! ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Aug 10, 2020

paulhauner marked this pull request as ready for review August 10, 2020 23:18

paulhauner added 2 commits August 11, 2020 09:39

Update docs

4618b38

Merge branch 'cross-compile' of github.com:sigp/lighthouse into cross…

dd6346c

…-compile

michaelsproul requested changes Aug 11, 2020

View reviewed changes

michaelsproul added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Aug 11, 2020

Fix typos in docs

857e0be

paulhauner added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Aug 11, 2020

michaelsproul approved these changes Aug 11, 2020

View reviewed changes

michaelsproul added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Aug 11, 2020

bors bot changed the title ~~Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4)~~ [Merged by Bors] - Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4) Aug 11, 2020

bors bot closed this Aug 11, 2020

paulhauner deleted the cross-compile branch October 5, 2020 08:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Merged by Bors] - Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4) #1497

[Merged by Bors] - Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4) #1497

Uh oh!

paulhauner commented Aug 10, 2020 •

edited

Loading

Uh oh!

michaelsproul left a comment

Uh oh!

michaelsproul Aug 11, 2020

Uh oh!

michaelsproul Aug 11, 2020

Uh oh!

michaelsproul Aug 11, 2020

Uh oh!

michaelsproul Aug 11, 2020

Uh oh!

paulhauner commented Aug 11, 2020

Uh oh!

michaelsproul left a comment

Uh oh!

bors bot commented Aug 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,41 @@
		# Cross-compiling

		Lighthouse supports cross-compiling, allowing users to run a binary one


		The `Makefile` in the project contains four targets for cross-compiling:

		- `build-x86_64`: builds an optimized version for x86_64 processors (suitable

[Merged by Bors] - Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4) #1497

[Merged by Bors] - Cross-compile to vendored x86_84, aarch64 (Raspberry Pi 4) #1497

Uh oh!

Conversation

paulhauner commented Aug 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue Addressed

Proposed Changes

Details

Additional Info

Uh oh!

michaelsproul left a comment

Choose a reason for hiding this comment

Uh oh!

michaelsproul Aug 11, 2020

Choose a reason for hiding this comment

Uh oh!

michaelsproul Aug 11, 2020

Choose a reason for hiding this comment

Uh oh!

michaelsproul Aug 11, 2020

Choose a reason for hiding this comment

Uh oh!

michaelsproul Aug 11, 2020

Choose a reason for hiding this comment

Uh oh!

paulhauner commented Aug 11, 2020

Uh oh!

michaelsproul left a comment

Choose a reason for hiding this comment

Uh oh!

bors bot commented Aug 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

paulhauner commented Aug 10, 2020 •

edited

Loading