Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Accept empty header set in range headers validation #4189

Merged
merged 8 commits into from
Jul 28, 2022
Merged

Accept empty header set in range headers validation #4189

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b67d557
Accept empty header set in range headers validation
shemnon Jul 28, 2022
68865d7
spotless
shemnon Jul 28, 2022
dff648b
changelog
shemnon Jul 28, 2022
0cb74d5
optional changes
shemnon Jul 28, 2022
f273e5d
Merge branch 'main' of github.com:hyperledger/besu into erigonShortRe…
shemnon Jul 28, 2022
f702945
changelog
shemnon Jul 28, 2022
507839b
spotless
shemnon Jul 28, 2022
1a85782
Merge branch 'main' into erigonShortResponse
macfarla Jul 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@
- Deprecation warning for Ropsten, Rinkeby, Kiln [#4173](https://github.com/hyperledger/besu/pull/4173)

### Bug Fixes

- Stop producing stack traces when a get headers response only contains the range start header [#4189](https://github.com/hyperledger/besu/pull/4189)

## 22.7.0-RC3
Known/Outstanding issues:
Besu requires a restart post-merge to re-enable remote transaction processing [#3890](https://github.com/hyperledger/besu/issues/3890)

### Known/Outstanding issues:
- Besu requires a restart post-merge to re-enable remote transaction processing [#3890](https://github.com/hyperledger/besu/issues/3890)

### Additions and Improvements
- Engine API: Change expiration time for JWT tokens to 60s [#4168](https://github.com/hyperledger/besu/pull/4168)
Expand All @@ -28,7 +28,6 @@ Besu requires a restart post-merge to re-enable remote transaction processing [#
- https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/22.7.0-RC3/besu-22.7.0-RC3.tar.gz / sha256: `6a1ee89c82db9fa782d34733d8a8c726670378bcb71befe013da48d7928490a6`
- https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/22.7.0-RC3/besu-22.7.0-RC3.zip / sha256: `5de22445ab2a270cf33e1850cd28f1946442b7104738f0d1ac253a009c53414e`


## 22.7.0-RC2

### Additions and Improvements
Expand Down
14 changes: 3 additions & 11 deletions ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/sync/range/RangeHeaders.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,21 @@
*/
package org.hyperledger.besu.ethereum.eth.sync.range;

import static com.google.common.base.Preconditions.checkArgument;

import org.hyperledger.besu.ethereum.core.BlockHeader;

import java.util.List;
import java.util.Objects;
import java.util.Optional;

import com.google.common.base.MoreObjects;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RangeHeaders {
private static final Logger LOG = LoggerFactory.getLogger(RangeHeaders.class);

private final SyncTargetRange range;
private final List<BlockHeader> headersToImport;

public RangeHeaders(
final SyncTargetRange checkpointRange, final List<BlockHeader> headersToImport) {
if (headersToImport.isEmpty()) {
LOG.debug(String.format("Headers list empty. Range: %s", checkpointRange.toString()));
}
checkArgument(!headersToImport.isEmpty(), "Must have at least one header to import");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only question here is if this changes the sync process in any meaningful way - could this cause sync to hang, for example, if we have a peer that is repeatedly returning nothing? It seems that previously we would throw here, and I'm guessing the sync process would be interrupted because of the exception.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajsutton - thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throwing the exception did not disconnect the peer, so we still had the peer.

But per spec (a) an empty response was 100% fine and (b) Erigon's response was not empty, we just trimmed off the one value we already had. So there are two routes we may be passing in an empty resposne down the pipeline and both cases the peer is behaving validly, so disconnecting doesn't seem right.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not saying we should disconnect, just want to make sure we're not introducing anything that could cause the sync to stall / get stuck. But we handle useless responses elsewhere, so even if we got stuck asking for the same range of headers from a single unresponsive peer, I think we would eventually disconnect and move on ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen any pauses. Rather than breaking the pipe an empty stream is the result, and that would get aggregated the same as a broken step, as nothing gets added to the next set of object for the next step in the pipeline. I haven't seen any slowdown overnight and I started a fresh sync and it's proceeding the same as before with no observable differences.

Side note: I'm really looking forward to being able to bit-torrent the blocks prior to the merge. EIP-4444 and friends will be welcomed with open arms.

this.range = checkpointRange;
this.headersToImport = headersToImport;
}
Expand All @@ -49,8 +41,8 @@ public List<BlockHeader> getHeadersToImport() {
return headersToImport;
}

public BlockHeader getFirstHeaderToImport() {
return headersToImport.get(0);
public Optional<BlockHeader> getFirstHeaderToImport() {
return headersToImport.size() > 0 ? Optional.of(headersToImport.get(0)) : Optional.empty();
}

@Override
Expand Down
51 changes: 28 additions & 23 deletions ...rc/main/java/org/hyperledger/besu/ethereum/eth/sync/range/RangeHeadersValidationStep.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,30 +43,35 @@ public RangeHeadersValidationStep(
@Override
public Stream<BlockHeader> apply(final RangeHeaders rangeHeaders) {
final BlockHeader rangeStart = rangeHeaders.getRange().getStart();
final BlockHeader firstHeaderToImport = rangeHeaders.getFirstHeaderToImport();

if (isValid(rangeStart, firstHeaderToImport)) {
return rangeHeaders.getHeadersToImport().stream();
} else {
final String rangeEndDescription;
if (rangeHeaders.getRange().hasEnd()) {
final BlockHeader rangeEnd = rangeHeaders.getRange().getEnd();
rangeEndDescription =
String.format("#%d (%s)", rangeEnd.getNumber(), rangeEnd.getBlockHash());
} else {
rangeEndDescription = "chain head";
}
final String errorMessage =
String.format(
"Invalid range headers. Headers downloaded between #%d (%s) and %s do not connect at #%d (%s)",
rangeStart.getNumber(),
rangeStart.getHash(),
rangeEndDescription,
firstHeaderToImport.getNumber(),
firstHeaderToImport.getHash());
throw new InvalidBlockException(
errorMessage, firstHeaderToImport.getNumber(), firstHeaderToImport.getHash());
}
return rangeHeaders
.getFirstHeaderToImport()
.map(
firstHeader -> {
if (isValid(rangeStart, firstHeader)) {
return rangeHeaders.getHeadersToImport().stream();
} else {
final String rangeEndDescription;
if (rangeHeaders.getRange().hasEnd()) {
final BlockHeader rangeEnd = rangeHeaders.getRange().getEnd();
rangeEndDescription =
String.format("#%d (%s)", rangeEnd.getNumber(), rangeEnd.getBlockHash());
} else {
rangeEndDescription = "chain head";
}
final String errorMessage =
String.format(
"Invalid range headers. Headers downloaded between #%d (%s) and %s do not connect at #%d (%s)",
rangeStart.getNumber(),
rangeStart.getHash(),
rangeEndDescription,
firstHeader.getNumber(),
firstHeader.getHash());
throw new InvalidBlockException(
errorMessage, firstHeader.getNumber(), firstHeader.getHash());
}
})
.orElse(Stream.empty());
}

private boolean isValid(final BlockHeader expectedParent, final BlockHeader firstHeaderToImport) {
Expand Down
18 changes: 18 additions & 0 deletions .../src/test/java/org/hyperledger/besu/ethereum/eth/sync/RangeHeadersValidationStepTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSpec;

import java.util.List;
import java.util.stream.Stream;

import org.junit.Before;
Expand Down Expand Up @@ -107,4 +108,21 @@ public void shouldThrowExceptionWhenValidationFails() {
+ firstHeader.getHash()
+ ")");
}

@Test
public void acceptResponseWithNoHeaders() {
var emptyRangeHeaders =
new RangeHeaders(new SyncTargetRange(syncTarget, rangeStart, rangeEnd), List.of());

final Stream<BlockHeader> result = validationStep.apply(emptyRangeHeaders);
assertThat(result).isEmpty();

verifyNoMoreInteractions(
protocolSchedule,
protocolSpec,
protocolContext,
headerValidator,
validationPolicy,
syncTarget);
}
}