Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Builder API #209

Closed
wants to merge 41 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
c374995
add builder api spec
lightclient Apr 21, 2022
e28576a
add bls and ssz to word list
lightclient Apr 21, 2022
075a2bb
fix typo
lightclient Apr 21, 2022
8d869f4
remove version from builder api spec
lightclient Apr 21, 2022
305d33e
address feedback from workshop
lightclient Apr 24, 2022
dc69e00
additional validity checks on timestamp
lightclient Apr 24, 2022
e6732a1
add link refs to stuct defs
lightclient Apr 24, 2022
40d5b44
fix typo
lightclient Apr 24, 2022
7a27240
missing v1 on proposer slashing def
lightclient Apr 24, 2022
069b2c4
another typo
lightclient Apr 24, 2022
79ea26e
missing v1
lightclient Apr 24, 2022
49c742d
fix some errors in ssz defs
lightclient Apr 24, 2022
4c44494
add status check to builder
lightclient Apr 25, 2022
b402f04
don't return null in happy case
lightclient Apr 25, 2022
b006213
rework signature defs a bit
lightclient Apr 25, 2022
4338877
missing closing backtick
lightclient Apr 25, 2022
25764b8
Update src/builder/specification.md
lightclient Apr 27, 2022
c814872
Update src/builder/specification.md
lightclient Apr 27, 2022
d0ae0d1
fix wrong message in get payload
lightclient Apr 27, 2022
f666d35
Update src/builder/specification.md
lightclient Apr 27, 2022
bd65467
Update src/builder/specification.md
lightclient Apr 28, 2022
6e73da2
Apply suggestions from code review
lightclient Apr 28, 2022
989c3bf
remove diagram
lightclient Apr 28, 2022
dfc10b6
soften requirement for ELs to implement interface
lightclient Apr 28, 2022
3614eec
Apply suggestions from code review
lightclient Apr 28, 2022
a90e003
change builder receipt to builder bid
lightclient Apr 28, 2022
e249cb3
blind -> blinded
lightclient Apr 29, 2022
66afd7f
verify registered validator
lightclient Apr 29, 2022
4ae07f3
typo fix
lightclient Apr 29, 2022
280bff1
relax get payload header assumptions
lightclient Apr 29, 2022
40547ba
competing chains note
lightclient Apr 29, 2022
1cbd4e7
add gas target to validator registration
lightclient Apr 30, 2022
4bba25c
rename to parent hash in get header
lightclient Apr 30, 2022
572f150
fix typo
lightclient Apr 30, 2022
3777486
fix numbering
lightclient May 2, 2022
190b7f3
fix incorrect error num
lightclient May 2, 2022
42e1fbe
wrong value ident
lightclient May 2, 2022
fcddc0d
better value ident
lightclient May 2, 2022
14963f3
use gas limit instead of gas target
lightclient May 2, 2022
d891d16
remove impl detail
lightclient May 3, 2022
e7dfb37
change validity delta on timestamps to 10 sec
lightclient May 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions src/builder/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Builder JSON-RPC API

The Builder JSON-RPC API is a collection of methods that external block building software must
implement. This interface allows the communication between the consensus layer and external builders.

This API is in *active development* and currently [specified as a markdown document](./specification.md).
A schema will follow once the specification stabilizes.
316 changes: 316 additions & 0 deletions src/builder/specification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,316 @@
# Builder API

This document specifies the Builder API methods that the Consensus Layer uses to interact with external block builders.

lightclient marked this conversation as resolved.
Show resolved Hide resolved
## Structures

### `ExecutionPayloadV1`

Mirror of [`ExecutionPayloadV1`][execution-payload].

### `ExecutionPayloadHeaderV1`
Equivalent to `ExecutionPayloadV1`, except `transactions` is replaced with `transactionsRoot`.
- `parentHash`: `DATA`, 32 Bytes
- `feeRecipient`: `DATA`, 20 Bytes
- `stateRoot`: `DATA`, 32 Bytes
- `receiptsRoot`: `DATA`, 32 Bytes
- `logsBloom`: `DATA`, 256 Bytes
- `prevRandao`: `DATA`, 32 Bytes
- `blockNumber`: `QUANTITY`, 64 Bits
- `gasLimit`: `QUANTITY`, 64 Bits
- `gasUsed`: `QUANTITY`, 64 Bits
- `timestamp`: `QUANTITY`, 64 Bits
- `extraData`: `DATA`, 0 to 32 Bytes
- `baseFeePerGas`: `QUANTITY`, 256 Bits
- `blockHash`: `DATA`, 32 Bytes
- `transactionsRoot`: `DATA`, 32 Bytes

### `BlindedBeaconBlockV1`
- `slot`: `QUANTITY`, 64 Bits
- `proposerIndex`: `QUANTITY`, 64 Bits
- `parentRoot`: `DATA`, 32 Bytes
- `stateRoot`: `DATA`, 32 Bytes
- `body`: `object`, [`BlindedBeaconBlockBodyV1`](#blindedbeaconblockbodyv1)

### `BlindedBeaconBlockBodyV1`
- `randaoReveal`: `DATA`, 96 Bytes
- `eth1Data`: `object`, [`Eth1DataV1`](#eth1datav1)
- `graffiti`: `DATA`, 32 Bytes
- `proposerSlashings`: `Array`, [`ProposerSlashingV1`](#proposerslashingv1)
- `attesterSlashings`: `Array`, [`AttesterSlashingV1`](#attesterslashingv1)
- `attestations`: `Array`, [`AttestationV1`](#attestationv1)
- `deposits`: `Array`, [`DespositV1`](#depositv1)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a typo in De**s**positV1, right ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch, here is a button light client can click to fix

Suggested change
- `deposits`: `Array`, [`DespositV1`](#depositv1)
- `deposits`: `Array`, [`DepositV1`](#depositv1)

- `voluntaryExits`: `Array`, [`SignedVoluntaryExitV1`](#signedvoluntaryexitv1)
- `syncAggregate`: `object`, [`SyncAggregateV1`](#syncaggregatev1)
- `executionPayloadHeader`: `object`, [`ExecutionPayloadHeaderV1`](#executionpayloadheaderv1)

### `Eth1DataV1`
- `depositRoot`: `DATA`, 32 Bytes
- `depositCount`: `QUANTITY`, 64 Bits
- `blockHash`: `DATA`, 32 Bytes

### `ProposerSlashingV1`
- `signedHeader1`: `object`, [`SignedBeaconBlockHeaderV1`](#signedbeaconblockheaderv1)
- `signedHeader2`: `object`, [`SignedBeaconBlockHeaderV1`](#signedbeaconblockheaderv1)

### `BeaconBlockHeaderV1`
- `slot`: `QUANTITY`, 64 Bits
- `proposerIndex`: `QUANTITY`, 64 Bits
- `parentRoot`: `DATA`, 32 Bytes
- `stateRoot`: `DATA`, 32 Bytes
- `bodyRoot`: `DATA`, 32 Bytes

### `AttesterSlashingV1`
- `attestation1`: `object`, [`IndexedAttestationV1`](#indexedattestationv1)
- `attestation2`: `object`, [`IndexedAttestationV1`](#indexedattestationv1)

### `IndexedAttestationV1`
- `attestingIndices`: `Array`, `QUANTITY`, 64 Bits
- `data`: `object`, [`AttestationDataV1`](#attestationdatav1)
- `signature`: `DATA`, 96 Bytes

### `AttestationDataV1`
- `slot`: `QUANTITY`, 64 Bits
- `index`: `QUANTITY`, 64 Bits
- `beaconBlockRoot`, `DATA`, 32 Bytes
- `source`: `object`, [`CheckpointV1`](#checkpointv1)
- `target`: `object`, [`CheckpointV1`](#checkpointv1)

### `CheckpointV1`
- `epoch`: `QUANTITY`, 64 Bits
- `root`: `DATA`, 32 Bytes

### `AttestationV1`
- `aggregationBits`: `DATA`, 0 to 256 Bytes
- `data`: `object`, [`AttestationDataV1`](#attestationdatav1)
- `signature`: `DATA`, 96 Bytes

### `DepositV1`
- `proof`: `Array`, 32 Bytes
- `data`: `object`, [`DepositDataV1`](#depositdatav1)

### `DepositDataV1`
- `pubkey`: `DATA`, 48 Bytes
- `withdrawalCredentials`: `DATA`, 32 Bytes
- `amount`: `QUANTITY`, 64 Bits
- `signature`: `DATA`, 96 Bytes

### `VoluntaryExitV1`
- `epoch`: `QUANTITY`, 64 Bits
- `validatorIndex`: `QUANTITY`, 64 Bits

### `SyncAggregateV1`
- `syncCommitteeBits`: `DATA`, 64 Bytes
- `syncCommitteeSignature`: `DATA`, 96 Bytes

#### Signed Containers

### `SignedBeaconBlockHeaderV1`
- `message`: `object`, [`BeaconBlockHeader`](#beaconblockheaderv1)
- `signature`: `DATA`, 96 Bytes

### `SignedVoluntaryExitV1`
- `message`: `object`, [`VoluntaryExitV1`](#voluntaryexitv1)
- `signature`: `DATA`, 96 Bytes

#### SSZ Objects

Consider the following definitions supplementary to the definitions in [`consensus-specs`][consensus-specs].

##### `ValidatorRegistrationV1`

```python
class ValidatorRegistrationV1(Container):
feeRecipient: Bytes20
gasLimit: uint64
timestamp: uint64
lightclient marked this conversation as resolved.
Show resolved Hide resolved
pubkey: BLSPubkey
```

##### `BuilderBidV1`

```python
class BuilderBidV1(Container):
lightclient marked this conversation as resolved.
Show resolved Hide resolved
header: ExecutionPayloadHeader
lightclient marked this conversation as resolved.
Show resolved Hide resolved
value: uint256
pubkey: BLSPubkey
```

###### `BlindedBeaconBlock`

```python
class BlindedBeaconBlock(Container):
slot: Slot
proposer_index: ValidatorIndex
parent_root: Root
state_root: Root
body: BlindedBeaconBlockBody
```

###### `BlindedBeaconBlockBody`

```python
class BlindedBeaconBlockBody(Container):
randao_reveal: BLSSignature
eth1_data: Eth1Data
graffiti: Bytes32
proposer_slashings: List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]
attester_slashings: List[AttesterSlashing, MAX_ATTESTER_SLASHINGS]
attestations: List[Attestation, MAX_ATTESTATIONS]
deposits: List[Deposit, MAX_DEPOSITS]
voluntary_exits: List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]
sync_aggregate: SyncAggregate
execution_payload_header: ExecutionPayloadHeader
```

## Errors

The list of error codes introduced by this specification can be found below.

| Code | Message | Meaning |
| - | - | - |
| -32000 | Server error | Generic client error while processing request. |
| -32001 | Unknown hash | No block with the provided hash is known. |
| -32002 | Unknown validator | Unknown validator. |
| -32003 | Unknown fee recipient | No known mapping between validator and fee recipient. |
| -32004 | Unknown block | Block does not match the provided header. |
| -32005 | Invalid signature | Provided signature is invalid. |
| -32006 | Invalid timestamp | Provided timestamp was invalid. |
| -32600 | Invalid request | The JSON sent is not a valid Request object. |
| -32601 | Method not found | The method does not exist / is not available. |
| -32602 | Invalid params | Invalid method parameter(s). |
| -32603 | Internal error | Internal JSON-RPC error. |
| -32700 | Parse error | Invalid JSON was received by the server. |

Each error returns a `null` `data` value, except `-32000` which returns the `data` object with a `err` member that explains the error encountered.

For example:

```
$ curl https://localhost:8550 \
-X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","method":"engine_getPayloadV1","params": ["0x1"],"id":1}'
{
"jsonrpc": "2.0",
"id": 1,
"error": {
"code": -32000,
"message": "Server error",
"data": {
"err": "Database corrupted"
}
}
}
```

## Routines

### Signing

All signature operations should follow the [standard BLS operations][bls] interface defined in `consensus-specs`.
lightclient marked this conversation as resolved.
Show resolved Hide resolved

There are two types of data to sign over in the Builder API:
* In-protocol messages, e.g. [`BlindedBeaconBlock`](#blindedbeaconblock), which should compute the signing root using [`compute_signing_root`][compute-signing-root] and use the domain specified for beacon block proposals.
* Builder API messages, e.g. [`builder_registerValidatorV1`](#builder_registerValidatorV1) and the response to [`builder_getHeader`](#response-2), which should compute the signing root using [`compute_signing_root`][compute-signing-root] and the domain `DomainType('0xXXXXXXXX')` (TODO: get a proper domain).

As `compute_signing_root` takes `SSZObject` as input, client software should convert in-protocol messages to their SSZ representation to compute the signing root and Builder API messages to the SSZ representations defined [above](#sszobjects).

## Methods
lightclient marked this conversation as resolved.
Show resolved Hide resolved

### `builder_status`
lightclient marked this conversation as resolved.
Show resolved Hide resolved

#### Request

- method: `builder_status`
- params: `null`

#### Response

- result: `enum`, `"OK" | null`
- error: code and message set in case the builder is not operating normally.

#### Specification

1. Builder software **MUST** return `-32000: Server error` if it is unable to respond to requests. `err` **SHOULD** be set explaining the issue and `result` **MUST** be `null`.

### `builder_registerValidatorV1`

#### Request

- method: `builder_registerValidatorV1`
- params:
1. `message`: `object`
1. `feeRecipient`: `DATA`, 20 Bytes - Address of account which should receive fees.
2. `gasLimit`: `QUANTITY`, 64 bits - Target gas limit for block.
3. `timestamp`: `QUANTITY`, 64 bits - Unix timestamp of announcement.
4. `pubkey`: `DATA`, 48 Bytes - BLS public key of validator.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidenote: Depending on ethereum/beacon-APIs#206, pubkey might be replaced with validatorIndex. This can be changed in a follow-up PR when needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the thing w/ validatorIndex is that we would like the option to re-use indices in the future (we aren't anywhere near that yet but where possible designing new feature to be forwards compatible w/ this possibility)

pubkeys will always uniquely identify a validator, there could be a world where index does not

for this use case, it would always be clear who the index refers to but it is something to keep in mind in the event there are knock-on effects down the road to software in this stack

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can expect builders look up the pubkey by index shortly before block construction. If we want to have pubkey be part of this call, we might need to it to ethereum/beacon-APIs#206 as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the thing w/ validatorIndex is that we would like the option to re-use indices in the future (we aren't anywhere near that yet but where possible designing new feature to be forwards compatible w/ this possibility)

Good point. As long as registration request validity is restricted to 10s or any other small amount of time this shouldn't be an issue. I admit that relying on timeout as a replay protection in this case may lead to unpredictable consequences if timeout for some reason will be removed in the future. If it isn't difficult for implementations to add pubkey to BN api request then we better have pubkey, furthermore, beacon API supports querying validators by pubkey which can be utilised by builders to verify that such validator exists in a validator set.

2. `signature`: `DATA`, 96 Bytes - BLS signature over [`ValidatorRegistrationV1`](#validatorregistrationv1).

#### Response

- result: `enum`, `"OK" | null`
lightclient marked this conversation as resolved.
Show resolved Hide resolved
- error: code and message set in case an exception happens while registering the validator.

#### Specification
1. Builder software **MUST** verify `pubkey` corresponds to an active or pending validator, otherwise return error `-32002: Unknown validator`.
2. Builder software **MUST** verify `signature` is valid under `pubkey`, otherwise return error `-32005: Invalid Signature`.
3. Builder software **MUST** respond to requests where `timestamp` is less than or equal to the latest announcement from the validator or more than 10 seconds in the future with error `-32007: Invalid timestamp`.
4. Builder software **MUST** return `result` as `"OK"` if the request succeeds, `null` otherwise.

### `builder_getHeaderV1`

#### Request

- method: `builder_getHeaderV1`
- params:
1. `slot`: `QUANTITY`, 64 Bits - Slot number of the block proposal.
2. `pubkey`: `DATA`, 48 Bytes - BLS public key of validator.
3. `parentHash`: `DATA`, 32 Bytes - Hash of execution layer block the proposer will use as the proposal's parent.

#### Response

- result: `object`
- `message`: `object`
- `header`: [`ExecutionPayloadHeaderV1`](#executionpayloadheaderv1).
- `value`: `QUANTITY`, 256 Bits - Payment in wei that will be paid to the `feeRecipient` account.
- `pubkey`: `DATA`, 48 Bytes - BLS public key associated with the builder.
Copy link

@metachris metachris May 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidenote: this might actually be the pubkey of the relay, or we might want signatures by both builder and relay. Definitely need the relay signature if the validator wants to whitelist a specific relay and require it to sign this.

- `signature`: `DATA`, 96 Bytes - BLS signature over [`BuilderBidV1`](#builderbidv1).
- error: code and message set in case an exception happens while getting the header.

#### Specification
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a restriction to disallow requests where input slot is in the future?
I'm not 100% sure if that's necessary, but I wonder what would be the intended behavior if anyone calls builder_getHeaderV1 earlier than the assigned slot. Curious to hear ppl's thoughts

Copy link

@metachris metachris May 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relay/builder should perhaps just return an error, or nothing at all. Seems similar to the situation where a relay does not (yet) have a block for the requested parentHash.

1. Builder software **SHOULD** respond immediately with the `header` that increases the `feeRecipient`'s balance by the most.
2. Builder software **MUST** return a `header` whose `parentHash` matches the request's `parentHash`. If `parentHash` is not known by the builder, it **MUST** return `-32001: Unknown hash`.
3. Builder software **MUST** return `-32002: Unknown validator` if `pubkey` does not map to the validator that is expected to propose at `slot`.
4. Builder software **MUST** return `-32003: Unknown fee recipient` if the builder does not have a `feeRecipient` mapped to the validator.
5. Builder software **MAY** set the `feeRecipient` for the block to a different address than the address mapped to the validator so long as a payment equal to `value` is made to `feeRecipient`.
6. Builder software **MUST** return a header whose `gasLimit` is equal to the validator's registered `gasLimit` or as close as possible under the constraints of the consensus rules.

### `builder_getPayloadV1`

#### Request

- method: `builder_getPayloadV1`
- params:
1. `message`: [`BlindedBeaconBlock`](#blindedbeaconblockv1).
2. `signature`: `DATA`, 96 Bytes - BLS signature over [`BlindedBeaconBlock`](#blindedbeaconblockv1).

#### Response

- result: [`ExecutionPayloadV1`](#executionpayloadv1).
lightclient marked this conversation as resolved.
Show resolved Hide resolved
- error: code and message set in case an exception happens while proposing the payload.

#### Specification
1. Builder software **MUST** be able to un-blind the [`ExecutionPayloadHeaderV1`](#executionpayloadheaderv1) in `message`, otherwise the return `-32004: Unknown block`.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Builder software **MUST** be able to un-blind the [`ExecutionPayloadHeaderV1`](#executionpayloadheaderv1) in `message`, otherwise the return `-32004: Unknown block`.
1. Builder software **MUST** be able to un-blind the [`ExecutionPayloadHeaderV1`](#executionpayloadheaderv1) in `message`, otherwise return the error `-32004: Unknown block`.

2. Builder software **MUST** verify that `signature` is a BLS signature over `block` using [`verify_block_signature`][verify-block-signature] from the validator that is expected to propose in the slot. If the signature is determined to be invalid or from a different validator than expected, the builder **MUST** return `-32005: Invalid signature`. It's possible that competing chains could have different proposer shuffling, causing multiple proposer indexes to be valid for a given slot.

[consensus-specs]: https://github.com/ethereum/consensus-specs
[bls]: https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#bls-signatures
[compute-signing-root]: https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#compute_signing_root
[bytes20]: https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md#aliases
[uint256]: https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md#basic-types
[execution-payload-header]: https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#executionpayloadheader
[execution-payload]: https://github.com/ethereum/execution-apis/blob/main/src/engine/specification.md#executionpayloadv1
[hash-tree-root]: https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md#merkleization
[beacon-block]: https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#beaconblock
[verify-block-signature]: https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#beacon-chain-state-transition-function
4 changes: 3 additions & 1 deletion wordlist.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
apis
attributesv
bls
bytecode
configurationv
eip
Expand All @@ -21,9 +22,10 @@ rlp
rpc
schemas
secp
sha
ssz
statev
statusv
sha
uint
updatedv
url
Expand Down