Skip to content

Conversation

@AgeManning
Copy link
Member

@AgeManning AgeManning commented Oct 29, 2020

Description

This PR updates Lighthouse to tokio 0.3. It includes a number of dependency updates and some structural changes as to how we create and spawn tasks.

This also brings with it a number of various improvements:

  • Discv5 update
  • Libp2p update
  • Fix for recompilation issues
  • Improved UPnP port mapping handling
  • Futures dependency update
  • Log downgrade to traces for rejecting peers when we've reached our max

@AgeManning
Copy link
Member Author

This compiles but requires compatibility layers on any dependencies that require tokio 0.2.

We can add compatibility layers, but the two main deps we need upgraded are warp and rust-libp2p.

It seems rust-libp2p has a PR pending. I'm going to hold off until we fix a few issues in rust-libp2p and it gets updated to 0.3.

@paulhauner paulhauner added the A0 label Nov 9, 2020
@AgeManning
Copy link
Member Author

Without us updating the entire ecosystem, this is now out of our hands, so I'm going to set this to a1

@AgeManning AgeManning added A1 and removed A0 labels Nov 17, 2020
* update dependencies

* update warp dependency repository

* cargo fmt
@AgeManning
Copy link
Member Author

Thanks to Ben, we're back on for A0

@AgeManning AgeManning added A0 and removed A1 labels Nov 17, 2020
@paulhauner
Copy link
Member

paulhauner commented Nov 25, 2020

I unblocked this since we've already released v1.0.0.

@paulhauner
Copy link
Member

All our Pyrmont nodes are now running the 420 commit, the latest at the time of this comment.

@AgeManning
Copy link
Member Author

bors r+

bors bot pushed a commit that referenced this pull request Nov 27, 2020
## Description

This PR updates Lighthouse to tokio 0.3. It includes a number of dependency updates and some structural changes as to how we create and spawn tasks.

This also brings with it a number of various improvements:

- Discv5 update
- Libp2p update
- Fix for recompilation issues
- Improved UPnP port mapping handling
- Futures dependency update
- Log downgrade to traces for rejecting peers when we've reached our max



Co-authored-by: blacktemplar <blacktemplar@a1.net>
@AgeManning
Copy link
Member Author

bors r-

@bors
Copy link

bors bot commented Nov 27, 2020

Canceled.

@AgeManning
Copy link
Member Author

bors r+

bors bot pushed a commit that referenced this pull request Nov 28, 2020
## Description

This PR updates Lighthouse to tokio 0.3. It includes a number of dependency updates and some structural changes as to how we create and spawn tasks.

This also brings with it a number of various improvements:

- Discv5 update
- Libp2p update
- Fix for recompilation issues
- Improved UPnP port mapping handling
- Futures dependency update
- Log downgrade to traces for rejecting peers when we've reached our max



Co-authored-by: blacktemplar <blacktemplar@a1.net>
@bors
Copy link

bors bot commented Nov 28, 2020

@bors bors bot changed the title Upgrade to tokio 0.3 [Merged by Bors] - Upgrade to tokio 0.3 Nov 28, 2020
@bors bors bot closed this Nov 28, 2020
bors bot pushed a commit that referenced this pull request Dec 8, 2020
## Issue Addressed

Closes #1264

## Proposed Changes

* Milagro BLS: tweak the feature flags so that Milagro doesn't get compiled if we're using BLST. Profiling showed that it was consuming about 1 minute of CPU time out of 60 minutes of CPU time (real time ~15 mins). A 1.6% saving.
* Reduce monomorphization: compiling for 3 different `EthSpec` types causes a heck of a lot of generic functions to be instantiated (monomorphized). Removing 2 of 3 cuts the LLVM+linking step from around 250 seconds to 180 seconds, a saving of 70 seconds (real time!). This applies only to `make` and not the CI build, because we test with the minimal spec on CI.
* Update `web3` crate to v0.13. This is perhaps the most controversial change, because it requires axing some deposit contract tools from `lcli`. I suspect these tools weren't used much anyway, and could be maintained separately, but I'm also happy to revert this change. However, it does save us a lot of compile time. With #1839, we now have 3 versions of Tokio (and all of Tokio's deps). This change brings us down to 2 versions, but 1 should be achievable once web3 (and reqwest) move to Tokio 0.3.
* Remove `lcli` from the Docker image. It's a dev tool and can be built from the repo if required.
bors bot pushed a commit that referenced this pull request Dec 8, 2020
## Issue Addressed

Closes #1264

## Proposed Changes

* Milagro BLS: tweak the feature flags so that Milagro doesn't get compiled if we're using BLST. Profiling showed that it was consuming about 1 minute of CPU time out of 60 minutes of CPU time (real time ~15 mins). A 1.6% saving.
* Reduce monomorphization: compiling for 3 different `EthSpec` types causes a heck of a lot of generic functions to be instantiated (monomorphized). Removing 2 of 3 cuts the LLVM+linking step from around 250 seconds to 180 seconds, a saving of 70 seconds (real time!). This applies only to `make` and not the CI build, because we test with the minimal spec on CI.
* Update `web3` crate to v0.13. This is perhaps the most controversial change, because it requires axing some deposit contract tools from `lcli`. I suspect these tools weren't used much anyway, and could be maintained separately, but I'm also happy to revert this change. However, it does save us a lot of compile time. With #1839, we now have 3 versions of Tokio (and all of Tokio's deps). This change brings us down to 2 versions, but 1 should be achievable once web3 (and reqwest) move to Tokio 0.3.
* Remove `lcli` from the Docker image. It's a dev tool and can be built from the repo if required.
bors bot pushed a commit that referenced this pull request Dec 9, 2020
## Issue Addressed

Closes #1264

## Proposed Changes

* Milagro BLS: tweak the feature flags so that Milagro doesn't get compiled if we're using BLST. Profiling showed that it was consuming about 1 minute of CPU time out of 60 minutes of CPU time (real time ~15 mins). A 1.6% saving.
* Reduce monomorphization: compiling for 3 different `EthSpec` types causes a heck of a lot of generic functions to be instantiated (monomorphized). Removing 2 of 3 cuts the LLVM+linking step from around 250 seconds to 180 seconds, a saving of 70 seconds (real time!). This applies only to `make` and not the CI build, because we test with the minimal spec on CI.
* Update `web3` crate to v0.13. This is perhaps the most controversial change, because it requires axing some deposit contract tools from `lcli`. I suspect these tools weren't used much anyway, and could be maintained separately, but I'm also happy to revert this change. However, it does save us a lot of compile time. With #1839, we now have 3 versions of Tokio (and all of Tokio's deps). This change brings us down to 2 versions, but 1 should be achievable once web3 (and reqwest) move to Tokio 0.3.
* Remove `lcli` from the Docker image. It's a dev tool and can be built from the repo if required.
@AgeManning AgeManning deleted the tokio-0.3 branch January 7, 2021 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A1 ready-for-review The code is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants