[WIP] EIP-7594: PeerDAS protocol

[hwwhww]

Previous work and discussion history: hwwhww#1
Now rebased on PR #3557 branch.

• Sort of the docs dependencies

[dapplion]

Should add the deprecation of /eth2/beacon_chain/req/blob_sidecars_by_root/1/ and /eth2/beacon_chain/req/blob_sidecars_by_range/1/

[g11tech]

may be by range can be deprecated, by root might still be useful to pull a complete blob? but anyway it can be pulled by root and index so i think we are good to deprecate 👍

[jimmygchen]

Is there a replacement for by_range for syncing?

I'm not familiar with sync, but I'm guessing we would be doing sampling for every slot as well when syncing - does that mean we request SAMPLES_PER_SLOT (8) samples via DataColumnSidecarByRoot for every block?

Sampling during sync:

• We don't need to sample finalized blocks since we know data was available based on the votes
• However, for latest head blocks without enough votes, we might have to sample before attesting.
  (Thanks @AgeManning)

Retrieving historic column data:
I guess we'll still need to retrieve all the custody_column data for the last 4096 epochs (so we can serve them to peers), so would be useful to have an interface for batched requests?
e.g. requesting List[Tuple[Slot, ColumnIndex]] or List[Tuple[Slot, SubnetId]]?

[dapplion]

We don't need to sample finalized blocks since we know data was available based on the votes

When syncing you don't know what's finalized ahead of time. Requiring a majority assumption is not okay IMO, why not sample if the network can serve you the samples?

[jimmygchen]

Yeah that makes sense to me I think. I guess it would be the same as Deneb today - we perform data availability check (sampling now instead) for all slots within the last MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS epochs.

[nisdas]

Is there more information on how the trailing slot fork choice works during network partitions or longer forks ? During any event where you might temporarily have your peers with split views(hard fork, consensus bug), the peers that you are connected to for the particular column subnets will neither have the block or the data. So when sampling for n-1 from these peers, they will return nothing and you would mark the data as unavailable.

How does sampling work when the network is undergoing stress and possible churn ? ( constant peer disconnections and new connections to find peers with the desired column subnets) . Is there a mode where we can fall back to so that a block and its data which was correctly broadcasted and propagated is marked as available rather than unavailable during such a situation.

[jimmygchen]

We would also need a DataColumnSidecarsByRange for syncing right?

Nodes would request the columns they are assigned to custody:

(
  start_slot: Slot
  count: uint64
  columns: List[ColumnIndex]
)

[djrtwo]

Yes -- especially if this retains the "Clients MUST support requesting sidecars since minimum_request_epoch," statement and thus cannot be used for syncing data prior to the finalized epoch in the normal case.

A syncing node will also need to do DAS since their checkpoint-sync.

I wonder if this should support columns or column (many or just one). It's not clear to me what the loadbalancing strategy would be given the multiple things this needs to support (historic sampling vs historic custody gathering) and given how peers have various custodies.

[jimmygchen]

For Deneb, we have MAX_REQUEST_BLOB_SIDECARS (768) calculated from MAX_REQUEST_BLOCKS_DENEB * MAX_BLOBS_PER_BLOCK

Should MAX_REQUEST_DATA_COLUMN_SIDECARS be derived in a similar way, i.e. maximum of 128 blocks of sidecars in a single request: MAX_REQUEST_BLOCKS_DENEB * NUMBER_OF_COLUMNS (16384)?

These should probably be added to the config files configs/mainnet.yaml and configs/minimal.yaml too.

[jimmygchen]

Implementation question - my apologies if it's off-topic for this PR - is there any value for a super-full node to custody the extended columns / rows (besides being able to serve data efficiently)? as it would always be able to reconstruct the full matrix without asking from its peers.

If it stores just the raw blobs, it would only need to store 50% of the samples for 1D extended blobs, or just 25% of the samples of the 2D extended blobs:

Max Blobs \ Data Stored	Storing raw blob only	1D extended	2D extended
6	96 GB	192 GB	384 GB
32	512 GB	1 TB	2 TB
64	1TB	2 TB	4 TB
128	2TB	4 TB	8 TB

[cskiraly]

Implementation question - my apologies if it's off-topic for this PR - is there any value for a super-full node to custody the extended columns / rows (besides being able to serve data efficiently)? as it would always be able to reconstruct the full matrix without asking from its peers.

Reconstructing a column when you have only row encoding is quite expensive. Say you need column j. Then for every segment i,j in row i, you would need to read half of the row i to do the reconstruction. This basically means reading in the whole data amount to reconstruct a single row.

So while you are always able to reconstruct, it can be costly, reading in and doing erasure coding on lots of data. I think it should be an implementation decision, but for the 1D extension I would keep the whole extended data for recent blocks, and only maybe discard half for old ones.

For the 2D extension, I would most probably store half of it, but in a more intelligent way than keeping the K columns or K rows. Basically what you want to do is to keep the upper left corner (original data in the systematic encoding), and the lower right corner ("double-parity" data) of the encoding. This allows you to efficiently restore any row, column or individual segment, while keeping only half of the data. Again, for old data that most probably won't be queried, you might want to fall back to keeping only the original (or some other K half-rows), while paying the price of a full restore when queried.

We might even go further, and instead of keeping the upper left corner and lower right corner, select a set A of K row IDs and a set B of K column IDs. Then you store chunk i,j iff
$i \in A \land j \in B \lor i \notin A \land j \notin B$
This allows the same fast reconstruction, but randomises who stores what. This can be done with local randomness to improve resilience, or with peerID based randomness to reduce load of queries from good behaving nodes who can query nodes where they know the node can serve cheap without reconstruction.

Of course, a super-full node would have to balance the above with its own needs to access the original data. If you actually want to read the original data frequently for your own use, it would be stupid not to store the original part (upper left corner) of the systematic encoding.

[dankrad]

Implementation question - my apologies if it's off-topic for this PR - is there any value for a super-full node to custody the extended columns / rows (besides being able to serve data efficiently)? as it would always be able to reconstruct the full matrix without asking from its peers.

If it stores just the raw blobs, it would only need to store 50% of the samples for 1D extended blobs, or just 25% of the samples of the 2D extended blobs:
Max Blobs \ Data Stored Storing raw blob only 1D extended 2D extended
6 96 GB 192 GB 384 GB
32 512 GB 1 TB 2 TB
64 1TB 2 TB 4 TB
128 2TB 4 TB 8 TB

All these are off by a factor of 2, assuming that target blobs are 1/2 of max blobs. (The EIP-1559 style pricing mechanism makes it impossible to significantly exceed the target for such a long period by increasing the price exponentially)

[fradamt]

As discussed with @dankrad, I think this should be higher than 1, because it is the most important parameter in determining how strong are the pre-sampling guarantees we get, i.e., how hard it is for an unavailable block to get a majority of the votes and become canonical, even when employing the trailing fork-choice. Ideally, we would want to ensure that only a small fraction of the validators can be tricked into voting for an unavailable block.

Imo, a suitable value would be >= 6. We might then also want to increase the number of subnets to 64, to keep the bandwidth consumption low. For reference, (CUSTODY_REQUIREMENT = 6, DATA_COLUMN_SIDECAR_SUBNET_COUNT = 64) and everything else unchanged means that the total bandwidth consumption with MAX_BLOBS_PER_BLOCK = 16 is 60% of 4844. With 32 blobs, it's 120% of 4844. It can also be lowered by increasing the subnet count further if needed.

[djrtwo]

Note that this is currently subnet custody which has many columns depending on the parametrization (4 currently if the parametrization is 128 columns across 32 subnets (128/32))

I find the way we are doing this parametrization a bit confusing or at least difficult to parse (as seen by the above misunderstanding)

The name should at least say explicitly if it's COLUMN_ or SUBNET_CUSTODY_REQUIREMENT

[fradamt]

I agree that the name should be SUBNET_CUSTODY_REQUIREMENT. Still, custodying a single subnet is not the same as custodying 4 random columns, and in fact gives you almost no pre-sampling guarantees.

For example, someone can make 49% of the subnets available (not making a subnet available here means not publishing any of the columns for that subnet), and 49% of validators will still succeed in downloading all columns they need. If on the other end the custody requirement had been 4 random columns, making 49% of columns available would only trick 1/16 of the validators. This is also the case if validators custody 4 subnets.

Another way to think about it is: whatever unit we choose for custody ends up being the relevant sampling unit for the purpose of pre-peer-sampling security. If we choose subnet custody, we are doing subnet sampling during distribution and column sampling later. If we want subnet sampling to be meaningfully secure by itself, we need the subnet custody requirement to be high enough.

[jimmygchen]

As this is derived from other preset values, would it make more sense to keep it in the preset? (i.e. modifying this NUMBER_OF_COLUMNS in config would require a corresponding change on the FIELD_ELEMENTS_PER_BLOB/FIELD_ELEMENTS_PER_CELL values?

# `uint64((FIELD_ELEMENTS_PER_BLOB * 2) // FIELD_ELEMENTS_PER_CELL)` (= 128)

[ppopth]

I don't know if it should be a config value, but I think it can be. I think it's okay for the config values to be derived from the preset values, but not the other way around. Preset values shouldn't be derived from config values.

[asn-d6]

Suggested change

	assert spec.FIELD_ELEMENTS_PER_BLOB * 2 % spec.config.NUMBER_OF_COLUMNS == 0
	assert spec.FIELD_ELEMENTS_PER_EXT_BLOB % spec.config.NUMBER_OF_COLUMNS == 0

Low priority. Feel free to merge without.

[asn-d6]

Suggested change

	| `NUMBER_OF_COLUMNS` | `uint64((FIELD_ELEMENTS_PER_BLOB * 2) // FIELD_ELEMENTS_PER_CELL)` (= 128) | Number of columns in the extended data matrix. |
	| `NUMBER_OF_COLUMNS` | `uint64(FIELD_ELEMENTS_PER_EXT_BLOB // FIELD_ELEMENTS_PER_CELL)` (= 128) | Number of columns in the extended data matrix. |

Low priority, feel free to merge without this.

[jtraglia]

Should ExtendedMatrix use MAX_BLOB_COMMITMENTS_PER_BLOCK too?

[djrtwo]

A bunch of minor nits/questions.

Getting there :)

[djrtwo]

Suggested change

	| `TARGET_NUMBER_OF_PEERS` | `70` | Suggested minimum peer count |
	| `TARGET_NUMBER_OF_PEERS` | `70` | Suggested minimum peer count to support peer sampling |

[djrtwo]

I'm in support of moving in this direction. Open a PR to this one. We'll likely get this merged in the meantime and can review the mapping proposal in isolation

[djrtwo]

just sanity checking that this 256-bit to 64-bit casting is well-defined in the spec (or at least done somewhere else)

My initial check shows that we are just using the ssz-typing lib here which will be opinionated but not in a way that is actually defined in the SSZ spec (no mention of casting as it's a serialization spec)

[djrtwo]

Why cast this down to 64-bit? why not just pass in the 256-bit value to hash?

[djrtwo]

actually we're doing it under the hood in places with bytes_to_uint64 so I guess it's well defined

[ppopth]

Why cast this down to 64-bit? why not just pass in the 256-bit value to hash?

I think it's because you need to do the + i thing. You need to cast it to 64-bit first to do the math. I guess doing the math under uint256 is not supported by all the programming languages.

However, I think it should be uint64(node_id) + i rather than uint64(node_id + i)

[djrtwo]

This seems like it shouldn't be here. It's an assertion we should have in our tests, but that it's not a possible condition here given the if subnet_id not in subnet_ids condition above for set append. and we generally don't have non-triggerable asserts in our functions.

[ppopth]

Agree with this. This assertion is always false. The intention is only to catch a bug.

[djrtwo]

Note that this is currently subnet custody which has many columns depending on the parametrization (4 currently if the parametrization is 128 columns across 32 subnets (128/32))

I find the way we are doing this parametrization a bit confusing or at least difficult to parse (as seen by the above misunderstanding)

The name should at least say explicitly if it's COLUMN_ or SUBNET_CUSTODY_REQUIREMENT

[djrtwo]

Suggested change

	*[New in Deneb:EIP4844]*
	*[New in EIP7594]*

[djrtwo]

I don't think we should mix assert and bool return value. All failures in relation to the input data in this function should likely affect the bool return value rather than some throwing and some returning

[djrtwo]

Or if these values should be checked in a different manner, potentially have them in an elevated pre-processing function or in the p2p gossip conditions or something. I understand the desire for the bool to just be on the proof verification and not on the well-structuredness of the data, but still mixing asserts and bool return make this a difficult function to test/parse

[djrtwo]

I agree with @ppopth that the name here is misleading. I would suspect it to take in a sidecar not a column_index value. I might argue for a retro-patch to the naming convenction in deneb

[djrtwo]

Yes -- especially if this retains the "Clients MUST support requesting sidecars since minimum_request_epoch," statement and thus cannot be used for syncing data prior to the finalized epoch in the normal case.

A syncing node will also need to do DAS since their checkpoint-sync.

I wonder if this should support columns or column (many or just one). It's not clear to me what the loadbalancing strategy would be given the multiple things this needs to support (historic sampling vs historic custody gathering) and given how peers have various custodies.

[djrtwo]

Is this a MUST field or is the value assumed to be MINIMUM if it is not found there. and what if someone puts in the field and puts a value less than MINIMUM, e.g. 0.

[ppopth]

I would say it should be a MUST field. If the field is absent, the node should assume that the ENR is invalid according to the current fork.

[Nashatyrev]

Not sure it makes a huge difference
However I'd prefer it to be optional and specify extra subnet count. The way of finding voluntary nodes would be different from finding regular (obligatory) nodes. So putting there anything when a node doesn't commit to serving extra subnets looks a bit pointless to me.

Anyway it needs to specified explicitly here

[hwwhww]

Thank you all for reviewing! <3

This PR is still WIP, and many comments are not addressed yet. But it's too long to review properly now. I think it's a good time to merge it now and I will collect the unsolved comments and clean up in the later PRs.

[Nashatyrev]

Doesn't BLSFieldElement type imply that x < BLS_MODULUS?
What is the meaning of % BLS_MODULUS operation then?

[ralexstokes]

it should, but the way the typing is set up a BLSFieldElement is any possible uint256 and so the reduction is necessary