Beta VII

This release provides breaking changes in the networking layer and in the db schema.

This version upgrades the discovery mechanism of Lighthouse to Discovery v5.1. This means Lighthouse will not be able to discover peers with previous versions of Lighthouse. There has also been a DB schema update. Running v0.3.1 on an older database will cause Lighthouse to migrate the older db to the new version. Users will not be able to downgrade their db after this.

We encourage all Medalla users to upgrade to this version to participate in the new networking changes and to improve peer discovery.

Changes: Summary

• Improved RPC validation (#1117)
• Improved docs and various bug fixes (#1756, #1759, #1761, #1760, #1762, #1747, #1800)
• Update the gossipsub message-id to latest specification (#1752)
• Add gossipsub topic filters (#1767)
• Correct issues and improve sync logic (#1791, #1804, #1796)
• Allow the Validator Client to start without validators (#1779)
• Implement further node endpoints (#1778)
• Reduce memory usage (#1798)
• Optimise attester slashing (#1745)
• Update to discv5.1 (#1786)
• Update libp2p (#1819)

DockerHub

• sigp/lighthouse:v0.3.1

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

King Flippy Nips

This is a major release which contains breaking features. The breaking features require users to perform:

• A resync of the database, since databases from v0.2.x are incompatible v0.3.0
• Some data directory migration for existing setups (see below)

Data Directory Migration

The directory structure has changed, such that the ~/.lighthouse dir (AKA "datadir") now contains one sub-directory for each --testnet value. What was previously ~/.lighthouse/beacon is now ~/.lighthouse/medalla/beacon or ~/.lighthouse/altona/beacon.

If you have an existing ~/.lighthouse directory you will need to move it into a sub-directory that matches the testnet name.

Example

Assume there is an existing ~/.lighthouse directory for Medalla:

.lighthouse
  |- beacon
  |- validators
  |- secrets

The directory should be re-arranged as such:

.lighthouse
  |- medalla
      |- beacon
      |- validators
      |- secrets

Note: you will likely need to edit the validators/validator_definitions.yml file to reflect the additional directory nesting.

Changes: Summary

Breaking Features

• The previous HTTP API has been replaced with a partially-complete standard Eth2.0-API implementation. See which endpoints are implemented at #1434.
• Prometheus metrics are no longer served on the same port as the HTTP API (default 5052). Metrics must be enabled with the --metrics flag and their address/port/CORS settings are managed separately to the API.
• The ~/.lighthouse directory structure has changed (mentioned earlier in the release notes).
• Lockfiles on EIP-2335 keypairs are now strictly enforced. The --delete-lockfiles flag can be used to recover from a scenario where the validator client detects existing lockfiles from a previous crash. Use --delete-lockfiles with caution and read the CLI help first.
• The Metamask deposit functionality has been removed from the Lighthouse docs. We direct users to the Eth2 launch pad instead.
• The slashing protection database is now stricter by default. The validator client will refuse to start if it finds validator keys without a corresponding database. When migrating your datadir to the new layout you should ensure you move slashing_protection.sqlite to ~/.lighthouse/medalla/validators/slashing_protection.sqlite (or $datadir/validators/slashing_protection.sqlite). More information here: https://lighthouse-book.sigmaprime.io/slashing-protection.html#misplaced-slashing-database
• Use 24-word mnemonics by default, with options for other lengths.

Non-Breaking

• Add the --testnet zinken option, with full support for the upcoming testnet.
• The --wss-checkpoint flag has been added to protect against weak subjectivity attacks. Users are not expected to require this flag until after mainnet launch.
• There is now a mechanism for caching BLS keys obtained from EIP-2335 keystores, to prevent long wait times when starting the VC. This cache should be transparent to users in most cases.
• Add a RESTful HTTP API to the VC.
• Add UPnP networking support.
• Support for the slashing interchange format.
• Various security fixes.
• Various stability fixes.

Changes: Detail

• (origin/staging) Strict slashing protection by default (#1750)
• Bump version to v0.3.0 (#1743)
• Update docs for v0.3.0 (#1742)
• Validator dir creation (#1746)
• Add zinken testnet (#1741)
• Improve command help (#1740)
• Implement key cache to reduce keystore loading times for validator_client (#1695)
• Clean up obsolete TODOs (#1734)
• Update external deps (#1711)
• Upgrade discovery and restructure task execution (#1693)
• Improve error handling in network processing (#1654)
• From panic to crit (#1726)
• Document need for port 9000 to be open (fix #730) (#731)
• Increase content-id length (#1725)
• Sync fixes (#1716)
• Use Drop impl to send worker idle message (#1718)
• Update libp2p (#1728)
• Tidy some TODOs (#1721)
• Address a couple of TODOs (#1724)
• Add check for head/target consistency (#1702)
• Add UPnP support for Lighthouse (#1587)
• Wallet creation: Make mnemonic length configurable, default to 24 words. (#1697)
• Update tiny-bip39 dependency to one implementing zeroize (#1701)
• Weak subjectivity start from genesis (#1675)
• Implement VC API (#1657)
• Implement slashing protection interchange format (#1544)
• Add database schema versioning (#1688)
• Implement standard eth2.0 API (#1569)
• Directory restructure (#1532)
• Fix validator lockfiles (#1586)
• Update key derivation to latest EIP-2333 (#1633)
• Update Cargo.lock (#1735)
• Improve error handling in network processing (#1654)
• From panic to crit (#1726)
• Document need for port 9000 to be open (fix #730) (#731)
• Increase content-id length (#1725)
• Allow truncation of pubkey cache on creation (#1686)
• Remove macos tests (#1687)

Dockerhub

• sigp/lighthouse:v0.3.0

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

Scary Terry

This release updates Lighthouse for the upcoming Spadina testnet launch.

• Adds the Spadina genesis state and associated boot nodes.
• Updates discovery (improves NAT filtering in the DHT).
• Network bug fixes related to peer management.

Changes

• 1ef4f0e Updates gossipsub attestation verification (#1667)
• 7bf1490 Adds Spadina genesis file and extended Spadina bootnodes (#1681)
• 6c1d7f5 Update Lighthouse Spadina bootnode (#1685)
• ae28773 Network bug fixes (#1684)

Dockerhub

• sigp/lighthouse:v0.2.13

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

Running Bird

This release contains new features and a fix to an error introduced in the previous release (v0.2.11).

• Adds support for the --testnet spadina flag.
• Reverts a change that caused "Illegal instruction" on ARM processors.
• Adds a --staking flag to the beacon node which enables the API and Eth1 syncing.
• Allow for interactive password entry on all account manager flows.

Changes

• 5688f21 Spadina support (v2) (#1670)
• 258b284 Update consensus code and tests to v0.12.3 (#1655)
• 3412a3e Remove saturating arith from state_processing (#1644)
• 28b6d92 Remove banned peers from DHT and track IPs (#1656)
• 15638d1 Beacon node does not quit on eth1 errors (#1663)
• b8013b7 Super Silky Smooth Syncs, like a Sir (#1628)
• 80e52a0 Subscribe to core topics after sync (#1613)
• 80ecafa Add --staking flag (#1641)
• b75df29 minimize the number of places we are calling update_pubkey_cache (#1626)
• 1801dd1 Interactive account passwords (#1623)
• 62c8548 Revert "Update BLST, add force-adx support (#1595)" (#1649)
• a97ec31 Subscribe to subnets an epoch in advance (#1600)
• 7aceff4 Add safe_sum and use it in state_processing (#1620)
• 4fca306 Update BLST, add force-adx support (#1595)

Dockerhub

• sigp/lighthouse:v0.2.12

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Note: we do not yet provide binaries for MacOS or Windows native.

Stair Goblins

Solves an issue which caused the beacon node to resume from the database using a canonical head that is no longer consistent with fork choice. The canonical head is now derived from fork choice when resuming from DB.

Also adds the new "trusted peers" feature (#1640).

Changes

• bd39cc8 Apply hotfix for inconsistent head (#1639)
• 14ff385 Add trusted peers (#1640)

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Cromulons

Solves an issue which caused some nodes to stall during sync.

Changes

• a17f748 Fix bad assumption when checking finalized descendant (#1629)
  • Fixes a "batch failed" error during syncing.
• 49ab414 Shift gossipsub validation (#1612)
• 2074bec Gossipsub message id to shortened bytes (#1607)
• e5fc6ba Remove redundant decompression in process_deposit (#1610)
• c9596fc Temporary Sync Work-Around (#1615)
• c6abc56 Prevent large step-size parameters (#1583)
• 7f1b936 ignore too early / too late attestations instead of penalizing them (#1608)
• 810de2f Static testnet configs (#1603)
  • Solves issues downloading github files experienced by users behind the GFW. Thanks @danielschonfeld.
• 0525876 Dial cached enr's before making subnet discovery query (#1376)
• d79366c Prevent printing binary in RPC errors (#1604)
• b19cf02 Penalise bad peer behaviour (#1602)
• dfe5077 Remove references to rust-docs (#1601)

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Jan-Michael Vincent

This release contains new user facing functionality (key recovery) and fixes an issue which cause validators to produce attestations which would not be included on chain. We have observed significant improvement in rewards using this release.

Changes

• Mnemonic key recovery (#1579)
  • Allows users to recover validator keys from mnemonics, thanks to @realbigsean
• Update state before producing attestation (#1596)
  • Critical bug fix by @pawanjay176 which improves validator attestation inclusion.
• Don't quote slot and epoch, for now (#1597)
• Use simple logger builder pattern (#1594)
• conforming to the p2p specs, requiring error_messages to be bound (#1593)
• Update discv5 to v10 (#1592)
• Add serde_utils module with quoted u64 support (#1588)
• Revert "add a github action for build multi-arch docker images (#1574)" (#1591)
• Avoid Printing Binary String to Logs (#1576)
• add a github action for build multi-arch docker images (#1574)
• Allow to use the same password when importing multiple keystores (#1479) (#1510)
  • Improves UX for importing from EF launchpad, thanks to @adetante
• Minor documentation fixes (#1297)
• Extended Gossipsub metrics (#1577)
• Revert 1502 - Switching docker user to lighthouse (#1578)
• Creates a new lighthouse user and makes it the default user to be use… (#1502)
• Increase logging channel capacity (#1570)
• Eth1 network exit on wrong network id (#1563)
• Use Gossipsub 1.1 (#1516)
  • Major networking milestone, this protocol will be used for mainnet. Congrats to @AgeManning and @blacktemplar.
• fix change flag name end_after_checks to continue_after_checks (#1573)

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

The President of the Miniverse

This release contains a fix to a known issue with the database which caused excessive database usage (>20gb when it should be ~8gb) and missing beacon state errors whilst syncing.

This release is compatible with existing databases, but you must re-sync if you wish to benefit from these fixes.

To reiterate, to repair a broken Lighthouse database please re-sync (lighthouse bn --purge-db) using this release.

Changes

• Alternative (to BeaconChainHarness) BeaconChain testing API (#1380)
• Check Cargo.lock freshness on CI (#1565)
  • Prevents releases from indicating the the git tree is dirty in the release tag.
• Fix bug in database pruning (#1564)
  • Resolves issue regarding excessive database usage and issues during sync. Requires a full resync.
• Fix order of testnet config load (#1558)
  • Fixes issues when using non-mainnet specifications (only relevant to developers).

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Shmlangela Shmlobinson-Shmlower

This release contains a fix to a known deadlock condition (#1562) and should provide significant stability improvements for nodes which run high validator counts (>= 64).

Changes

• Remove rayon from rest_api (#1562)
  • Prevents deadlock.
• Shift HTTP server heavy-lifting to blocking executor (#1518)
  • Avoids locking the core executory.
• reuse beacon_node methods for initializing network configs in boot_node (#1520)
• Docs: Fix reference to incorrect password file. (#1556)
• ban IP addresses if too many banned peers for this IP address (#1543)

Known Issues

There is a known issue with database pruning, causing failures during syncing and high disk usage. This is a major priority and a fix will be released ASAP.

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.

Water-T

This release provides a significant stability improvement (#1546) and protects nodes against a DoS vector.

Changes

• Refactor attestation service (#1415)
• Shutdown like a Sir (#1545)
• Add remaining network ops to queuing system (#1546)
  • Prevents locking when processing large numbers of slashings, as experienced on Medalla.

Binaries

Binaries are supplied for two platforms:

• x86_64-unknown-linux-gnu: AMD/Intel 64-bit processors (most desktops, laptops, servers)
• aarch64-unknown-linux-gnu: 64bit ARM processors (Raspberry Pi 4)

Additionally there is also a -portable suffix which indicates if the portable feature is used:

• Without portable: uses modern CPU instructions to provide the fastest signature verification times (may cause Illegal instruction error on older CPUs)
• With portable: approx. 20% slower, but should work on all modern 64-bit processors.