EIP-7549: replace committee_bits with committee_indices

[mkalinin]

Proposal

Replaces committee_bits: Bitvector which is introduced by EIP-7549 with committee_indices: List[CommitteeIndex]. It gives more flexibility to a block proposer in producing on chain aggregates out of network aggregates received from the wire.

Today a block can contain up to MAX_ATTESTATIONS aggregates with the same committee index. Since EIP-7549 reduces this parameter from 128 to 8 it leaves much less space for duplicates. This proposal attempts to relax this limitation. Accommodating for more duplicates might be helpful in the case of bad network conditions when there are many valuable aggregates with a slight overlap in the aggregation_bits.

Analysis

This PR uses CommitteeIndex(uint64) type for a committee index which gives an overhead to the size of on chain aggregates. Using uint8 instead might be an option to diminish the overhead.

Todo

• Fix tests

[terencechain]

I wonder if we should enforce these committee indices to be sorted

[mkalinin]

tbh, I don’t know, do we have any reason for enforcing them to be sorted?

[terencechain]

Nothing other than it being cheap to do and nice to display, but whoever displays it can also sort it out of band. I'm indifferent, but thought it's worth considering so I don't miss anything.

[mkalinin]

Yeah, it worth considering! but beyond potential UX improvement I don’t see any reason to enforce that. So by default i’d leave it as is unless there are other reasons for doing that

[tbenr]

I feel like enforcing sorting makes things more clear and preserve equality: with non-sorted we may have Attestation1.hash_root != Attestation2.hash_root but they could have exact same coverage. (correct me if i'm wrong)
Having a unique way of expressing the same thing maybe isn't super important but seems a nice property to have.

[tbenr]

as per offline discussion, worth noting that, as we introduce potential duplications in index list, then the the assumption same attestation -> same hash_root will be broken anyway

[arnetheduck]

as we introduce potential duplications in index list

duplications would imply that signature is different already (since each entry represents a term being added to the aggregate sig), which was the motivation for this PR..

I'd be interested to see how much of an improvement this possibility actually provides, ie how often is it that when we're packing attestations, we would benefit from overlapping aggregates - this would imply that aggregators observed different attestations which should be rare.

[arnetheduck]

basically, if I casually look at the aggregates coming in from aggregators, apart from poor quality due to timing most likely, it's rare that there are multiple overlapping aggregates of equal quality.

[mkalinin]

If we take a look at on chain data (i picked a few random blocks — not that representative tho), they have duplicates. Probably, those duplicates are added by the proposer from what it has observed in the attestation subnet itself as the duplicates are usually scarce.

The issue that we have discussed with @tbenr is that the same attestation can have different representations. Consider we have n aggregates from the same committee in one on chain aggregate, if we enforce the order of committee indices in the array then by changing the order of the aggregation bits we can get up to n! different on chain aggregates with distinct hash_tree_root() each that are actually carrying the same information.

[arnetheduck]

changing the order of the aggregation bits

Sure, as long as they're permutations - I misunderstood it as entries with different number of duplicates being discussed.

I'm bringing it up because one alternative encoding is to list the validators that were not included - usually, there's only a few of those and this way we would probably have a significantly smaller list, but that again brings us back to the only-one-item territory - it's been discussed as an optimisation strategy as well, ie keep an aggregated committee key and subtract - probably makes more sense for sync committee messages but still.