SIMD: (APExB) Asynchronous Program Execution and Broadcast

[aeyakovenko]

Problem

Transmitting blocks of user transactions and executing those transactions blocks fork choice. Only a single leader can propose user transactions at the moment, which means that client latency is going to be on average half way around the world.

Solution

Propose a design to split out user transactions into separate user blocks that are transmitted concurrently with blocks of consensus votes. This naturally follows into asynchronous execution of the user blocks.

[buffalu]

proposer?

[aeyakovenko]

Isn't builder a better name? Proposer is too generic, since the leader also proposes a block

[buffalu]

a proposer proposes the block to the network that others vote on. in this case i think they'd be proposing the ordering?

[aeyakovenko]

leader proposes the block technically. The builder builds UserBlocks that are broadcast over turbine, but it's the leader, just like today, that actually makes the block that validators vote on.

[buffalu]

how do they determine which one to add? is this in the world of bankless leader (leader isn't executing transactions, just doing poh recording + shred signing?)

[buffalu]

what if multiple UserBlockEntry contain same tx?

[aeyakovenko]

Second one is skipped

[aeyakovenko]

Leader adds the UserBlockEntry to PoH as soon as it is received. Since UserBlocks are transmitted over turbine, if a leader sees it, it's very likely the rest of the network has seen it as well.

[buffalu]

i thought the leader was doing the ordering by poh?

[buffalu]

"Each UserBlock is assumed to have been created simultaneously during
the UserBlockSlot that it was encoded by the leader. For each
UserBlock, the transactions are ordered by priority fee before
execution."

[aeyakovenko]

leader has 200ms to include all the valid UserBlocks. Each UserBlock is assumed to have been created at the same time. So the TX ordering between the user blocks is based on priority fees in the transactions. The PoH ordering of user blocks is just used as a tie breaker.

[buffalu]

im confused where they repair from here if they're the only one getting the UserBlock. unless its propagated to the entire cluster somehow

[aeyakovenko]

UserBlocks are sent over turbine. So repair and everything else works as is.

[bji]

im confused where they repair from here if they're the only one getting the UserBlock. unless its propagated to the entire cluster somehow

All validators that would vote on the fork containing the user block must get all user blocks for that fork before voting on it. Voting on a fork without even verifying that the user blocks it references exist would allow attacks on the network by malignant nodes that get user blocks entries inserted into blocks but never provide the actual transaction data. Not sure how anyone can expect to sanely evaluate the state of a fork for which certain transactions are hidden and never made available.

[aeyakovenko]

That is correct. Before voting on a fork each validator has to have all the data, including all the data form all the user blocks in that fork. Otherwise there is no way to guarantee that everyone can execute because the data could be withheld

[buffalu]

a little confused in this paragraph; is the builder hashing their block with PoH? if the leader processes userblocks at the 200ms mark, why does the userblock need poh? can the userblock just be submitted with a poh in the past to get in the block earlier, which means all userblock submitters will not advance poh to game the system?

[aeyakovenko]

UserBlocks don't need PoH, but follow the same ledger format with entries. Fixed it in the update.

[buffalu]

LFG

[buffalu]

looks like we might want to include a bundle primitive in here too to avoid splitting up bundles that get reordered/not guaranteed to execute atomically. can imagine bundle like:
[user_high_priority_tx, arb_low_priority_tx]

[aeyakovenko]

Added the BundleTransaction

[buffalu]

thoughts:

• this is probably my favorite proposal so far.
• i like this proposal much more for the global centralization issues around solana.
• the fact that state is always changing on solana (shreds always being broadcasted) means colo means a lot. its good for throughput, but really encourages colo so you can ingest shreds -> replay -> geyser notif -> send mev tx as fast as possible. its also hard AF to simulate things correctly in our block engine w/o significant RPC and replay modifications and will be even harder for others that don't have access to that :)
• assuming this proposal suggests that state is paused for ~x ms (200ms in the proposal, could prob crank down even more to like 50ms), this would mean that someone could be semi-competitive doing mev from across the globe (latency will never not matter).

[buffalu]

this is also nice bc you can spend more time scheduling ahead of time and massively parallelize the execution of these transactions.

• sort by priority
• DAG
• batch
• execute

[buffalu]

to keep terminology, does RecordBatch or EntryBatch make more sense? this is essentially what the current validator sends to poh but in batch form? Vec<Vec>

[eugene-chen]

is this within a single UserBlock, or within the stage where multiple UserBlocks are merged?

[aeyakovenko]

Within the stage when they are merged.

[bji]

Fork choice does depend on program execution: the stake program can change stake weight of voters which influences fork choice. Stake weight can only change at epoch boundaries though so there must be a 'sync' at epoch boundaries, where validators must have "caught up" on executing all tx at an epoch boundary before they can vote beyond that epoch boundary.

Is this wrong?

[aeyakovenko]

Yes. Nodes need to be able to compute a snapshot once an epoch.

[bji]

Yes. Nodes need to be able to compute a snapshot once an epoch.

Would prefer if this issue was addressed more completely than a one-liner. Given that the whole purpose of this proposal is to increase asynchronous execution, having a big synchronization point once per epoch seems like a big deal. It's not clear to me that having a period of asynchronous execution followed by a synchronization point is an overall win, given that it will introduce a period of time at the beginning of each epoch where "nothing new happens until everyone is caught up". Is asynchronous execution during later parts of an epoch worth the reduction in throughput during early parts of an epoch?

[aeyakovenko]

Nodes can't fall behind that much because the overall CU limits are set for synchronous execution. But with the option of async execution it is much easier to catch up. Raw ledger processing without dealing with the network is 20-30x times faster.

[bji]

thoughts:

* this is probably my favorite proposal so far.

* i like this proposal much more for the global centralization issues around solana.

* the fact that state is **always changing** on solana (shreds always being broadcasted) means colo means **a lot**.  its good for throughput, but really encourages colo so you can ingest shreds -> replay -> geyser notif -> send mev tx as fast as possible. its also hard AF to simulate things correctly in our block engine w/o significant RPC and replay modifications and will be even harder for others that don't have access to that :)

* assuming this proposal suggests that state is paused for ~x ms (200ms in the proposal, could prob crank down even more to like 50ms), this would mean that someone could be semi-competitive doing mev from across the globe (latency will never not matter).

Might want to temper your enthusiasm. This will make arbitrage alot more difficult since unless you can execute transactions faster than your competitors, you will be at a disadvantage. They will know the "current state" but you will only know "state at some time in the past". Trying to write arbitrage tx against old state seems like a really good way to waste priority fees.

[buffalu]

This will make arbitrage alot more difficult since unless you can execute transactions faster than your competitors, you will be at a disadvantage

this will make arbitrage easier to be competitive, not harder, bc its less about latency and more about how much you're willing to pay

[bji]

This predefined scheduling mechanism should be greatly expanded. It is making the same mistake that the original Solana design made: not thinking through all of the expected costs of transaction execution and ensuring that transactions are prioritized for execution in a way that maximizes efficiency. Transactions should be ordered in decreasing order by (total_fees / expected_execution_cost). This would mean that assuming that the expected_execution_cost can be approximated closely, then transactions will execute in priority order based on how much they pay to execute versus how much they cost to execute.

However, since you want to predefine the scheduling mechanism here and bake it into the consensus algorithm, you will have to predefine how to compute expected execution costs. This should include write locks, read locks, total accounts referenced, compute units paid for, byte size of tx, etc.

You should also define all the mandatory fees - it would be a good idea to enforce a mandatory fee for each of the "cost" categories mentioned in the previous paragraph. This will improve fee predictability for users.

[bji]

TL;DR ordering by priority fee alone is wholly inadequate.

[aeyakovenko]

I am not sure I agree. The UserBlock will be limited in the amount of CUs it can take up, so the order of execution can be based on priority fee which is lamports per CU. CUs will take into account the cost of locks and reads and writes. That is outside the scope of this design. A different design should be covering how each resource is counted towards the CUs used by the transaction. TXs must request the CUs up front, so the amount of compute used is known ahead of time.

[bji]

I do not believe this is correct. Builders will have to transmit some amount of the MEV to validators via priority fees in order to ensure that their BundleTransactions execute. A leader does not have to accept a BundleTransaction, it can ignore it, if the total fees paid by all tx in the bundle + the bundle priority fee do not make the bundle attractive enough to schedule ahead of other tx/bundles.

For this reason, builders will have to compete on priority fees for bundles, which will naturally transmit a significant fraction of MEV to validators.

[aeyakovenko]

How can the leader ignore a BundleTransaction? It's part of the UserBlock, so the entire block must be accepted, including the BundleTransaction.

[aeyakovenko]

added some clarifications

[hydrogenbond007]

Hey kinda confused here, so the builders can reorder txns and capture the MEV but what makes a block attractive for the leader to be included in the slot rather than just aiming for lower latency?
Builders must offer some reward (from the priority fee) to the validator, right?

[aeyakovenko]

priority fees and base fees go to the leader

[bji]

im confused where they repair from here if they're the only one getting the UserBlock. unless its propagated to the entire cluster somehow

All validators that would vote on the fork containing the user block must get all user blocks for that fork before voting on it. Voting on a fork without even verifying that the user blocks it references exist would allow attacks on the network by malignant nodes that get user blocks entries inserted into blocks but never provide the actual transaction data. Not sure how anyone can expect to sanely evaluate the state of a fork for which certain transactions are hidden and never made available.

[bji]

Yes. Nodes need to be able to compute a snapshot once an epoch.

Would prefer if this issue was addressed more completely than a one-liner. Given that the whole purpose of this proposal is to increase asynchronous execution, having a big synchronization point once per epoch seems like a big deal. It's not clear to me that having a period of asynchronous execution followed by a synchronization point is an overall win, given that it will introduce a period of time at the beginning of each epoch where "nothing new happens until everyone is caught up". Is asynchronous execution during later parts of an epoch worth the reduction in throughput during early parts of an epoch?

[lheeger-jump]

It would be really awesome to get a state machine diagram for how clients are supposed to behave under this new scheme.

[lheeger-jump]

Can you elaborate? Why is that the case now?

[aeyakovenko]

See the UserBlock execution section.

[hydrogenbond007]

@lheeger-jump can make one on excali if you like, been reading about the eth proposal to have multiple concurrent block producers and this kinda feels related.

[aeyakovenko]

yes, everything is converging. I think the main difference is that solana has turbine and will provision validators to saturate the bandwidth available to them

[bji]

This will make arbitrage alot more difficult since unless you can execute transactions faster than your competitors, you will be at a disadvantage

this will make arbitrage easier to be competitive, not harder, bc its less about latency and more about how much you're willing to pay

But latency is also "how much you're willing to pay".

[aeyakovenko]

@lheeger-jump

By clients do you mean wallets? How do they decide if something has been confirmed?

[segfaultdoc]

Is the leader just another node in the turbine path receiving user blocks?

[aeyakovenko]

Yep, if the leader observes the UserBlocks is very likely that the supermajority of the cluster has as well.

[apfitzge]

Some initial comments/concerns. Nice write up!

[apfitzge]

is N here meant to be a fixed limit or an arbitrary number?

[aeyakovenko]

Configured by the cluster. Builders need to be scheduled ahead of time.

[apfitzge]

This is a reasonable place to start imo. However, I have concerns that this could have unintentional side effects that lead to smaller blocks.

Validators are not spread evenly across the world, nor are users. There is significant clustering in North America and Europe.

As a hypothetical, let's say we have 2 builders 1 in Amsterdam and the other in Wellington NZ (roughly opposite sides of the world). The majority of users would simply target their nearest builder which will likely be Amsterdam. Amsterdam block gets packed and leaves people out while the Wellington block sits (relatively) idle.

It's certainly not a liveness issue, but I wonder if there is something that could be done to reduce this potential impact.

---

NOT WELL THOUGHT OUT SPITBALL IDEA

Could we have more builders than can possibly fit into the UserBlockSlot, and leader takes the most heavily packed UserBlockEntries. This might be more complicated since then the leader has to scan these entries before selecting it.

Example above: 3 builders (AMS, WEL, +TOR).
AMS, TOR get packed because they are geographically (reduced latency) closer to users, and WEL gets excluded by the leader.

ugh, thinking about this more...maybe not a good idea. There's no way for non-leader to verify the leader chose the most packed block. Though the leader would (hopefully) be incentivized to choose the most packed due to fee-collection?

[aeyakovenko]

Yea I generally agree, we could figure out some way to do work sharing. But I think this is a v2 optimization.

[mschneider]

Another reason for probably very small blocks: the builder only knows the compute limits requested not the compute units consumed.

[apfitzge]

Something to consider here is how to handle duplicate transactions.

[hydrogenbond007]

sorry I might be dumb but don't we already have dedup features?

[apfitzge]

No problem my previous message was not clear 😄

We have deduping that prevents txs from appearing in multiple different slots. With this proposal, we have multiple builders who could potentially include the same transaction in their UserBlock since they can't know ahead of time what the other will include. We need to ignore one of them so we don't process the transaction twice, which is straight-forward to do.

My previous comment was mainly highlighting an edge-case for the economics of this deduplication. They both included a valid transaction in their UserBlock, so how do we handle fees?

Potentially a few options:

1. builder proportion of fees for tx split evenly between the two builders
2. first builder's block to reach leader

1 seems a better option, but it's also going to entirely depend how we split fees between the leader and builders in general.

[nikhayes]

Could the transaction being sent contain information specifying which leader to process it? That would avoid duplicates and it's easier to filter the block for transactions referencing one of the particular leaders to know which fee rewards it should receive.

[aeyakovenko]

Duplicate TXs are skipped. See line 141

[7layermagik]

Maybe being able to specify the leader is also a feature though? Having your transaction processed by a certain leader might be undesirable for mev reasons so there could be reason for specificity?

Also, if filling blocks is based on requested/estimated cu's, skipping dupe transactions would lead to empty space whereas this solution wouldnt.

[aeyakovenko]

Wdym processed by a certain leader? Either the UserBlcok is included or it's not. The big question is if it can be included in any slot, or only the scheduled slot.

[apfitzge]

I believe what is meant is a mechanism to make sure a transaction is processed by a specific builder

Having your transaction processed by a certain leader might be undesirable for mev reasons so there could be reason for specificity?

If there's a certain builder I know does some MEV that is undesirable for my transaction, then I would request my tx is processed by the other.

[hydrogenbond007]

Sorry again a dumb idea
Is there any way to encrypt the txns beforehand and reveal the transaction details only after they are included in the block to prevent bad reordering? Homomorphic encryption could allow operations to be run on the encrypted builder blocks

[mschneider]

I believe what is meant is a mechanism to make sure a transaction is processed by a specific builder

Having your transaction processed by a certain leader might be undesirable for mev reasons so there could be reason for specificity?

If there's a certain builder I know does some MEV that is undesirable for my transaction, then I would request my tx is processed by the other.

if the builder selection was based on the first few bits of the signature:

• every transaction can only appear in exactly one user block, removing the need to dedup parallel blocks
• builders could easily filter duplicate transactions submitted to the wrong builder by spammers
• power users could choose builders by bit-flipping data in their message
• users who don't care get properly load balanced

[apfitzge]

A bit confused on this part - why should we begin to allow a builder to include transactions without proper signatures?

Shouldn't the builder's be running sigverify and basic checks?

[aeyakovenko]

Because if it is asynchronous, nodes may detect the invalid signature much later, well after they vote. The only thing validators need to do to vote on forks is have the UserBlock data and verify the format and the builder signature.

[ilya-bobyr]

There is still an assumption that Builders validate transactions, making an effort to propose only UserBlocks with something that is valid, right?
As Builders are incentives for including transactions, they do not want to include garbage, and they could also be slashed for including transactions with invalid signatures or invalid in some other obvious way.
But, as pointed out, this should not stop the validator from including the UserBlock hash, even if it contains invalid transactions.

[hydrogenbond007]

great write up and would like to suggest some convo on something like Whisk which will help protect the selected leaders from dos attacks.
and once slashing is started, having leaders ddosed could become a concern
(sorry if the idea seems dumb still learning)

[ilya-bobyr]

Does this effectively creates something similar to the Ethereum mempool, but on chain?
As people are going to publish their transactions, yet the final order is determined only at the end of the UserBlockSlot.
Someone observing transactions sent to Builders can reliably front run any of those transactions, by including a transaction with a higher fee, as long as the current UserBlockSlot is not over yet.
And they can do so using any of the available Builders.
I am not necessarily criticizing - more like trying to make sure my understanding of the proposed mechanics is correct.

Or is the assumption that Builders do not share the transaction information until the end of the UserBlockSlot?

[sakridge]

@aeyakovenko is that right? can two transactions write to an account in a block?

[mschneider]

200ms is very short for random paths across the globe, I would assume that a naive builder just need to send the block pretty early if they want it to arrive in time for the UBS. if there's a second builder closer to the proposer that could be used to inject txs in the last 20ms.

the whole block reorg on the proposer is kinda weird. it opens up for this kind of latency race. this solution is very close to PBS but it violates a fundamental assumption: builders decide on final order.

[ilya-bobyr]

There is still an assumption that Builders validate transactions, making an effort to propose only UserBlocks with something that is valid, right?
As Builders are incentives for including transactions, they do not want to include garbage, and they could also be slashed for including transactions with invalid signatures or invalid in some other obvious way.
But, as pointed out, this should not stop the validator from including the UserBlock hash, even if it contains invalid transactions.

[ilya-bobyr]

Wouldn't this mean that the most optimal behavior for the users would be to send every transaction for inclusion to every Builder?
If duplicates are removed for free, users lose nothing. Yet, as the UserBlock hash order matters for transaction ordering in case of a conflict, it is beneficial to have all possible hashes assigned to your transaction.
It also removes the need of determining which Builder has the best RTT for the given user.

If it is the case, the downside is that it may effectively reduce the effective slot capacity, if too many transactions would turn out to be just duplicates in the final state.

[apfitzge]

I have this concern as well, and it seems to be that @7layermagik may as well: #45 (comment)

[7layermagik]

Hmm yeah I guess u could have something in the transaction that specifies which leader it can be included by. If they want it to be processed by both leaders it costs double

[aeyakovenko]

or charge the fee on both if it is included in multiple user blocks by different leaders. spammer would need to pre fund a bunch of fee payers, which wouldn't be free.

[apfitzge]

I don't think just charging fees for inclusion by multiple builders is sufficient.
That probably opens users up to attacks because builders are transmitting over turbine as they build.
Someone listening on turbine could just re-transmit processed transactions to other builders w/ hope of inclusion, and screwing over users on getting double-fee charged.

We can probably do some combination of hash/bit-based builder routing (like @mschneider suggested elsewhere) so signed message is only includable by certain builders.

This approach seems to check most of the boxes I think we want:

1. Users can select a specific builder by fiddling with bits
2. Users can send to multiple builders, but must pay to do so.

If users want to send to multiple builders, they'd want to have some sort of gate on the tx to prevent it from actually executing twice; seems reasonable this could just be done with a sequencer.

[ilya-bobyr]

IIUC, changes described in this proposal make leaders vote on slots without seeing the transaction actual content. At the same time, Builders also do not seem to be doing anything useful with the transactions, except of forwarding them.
Sorry if this is something that has been discussed in the past, I did not read the previous SIMDs.
But I wonder what would happen if the next step is taken in the same direction.
And Builders do not see the transactions that they pack, either.

If users submit the fee and just the transaction hash to the Builders in UserBlockSlot N-1, and the actual transactions are submitted (and forwarded) during UserBlockSlot N.

For every transaction hash submitted in UserBlockSlot N-1, a matching transaction needs to be present in UserBlockSlot N, for the transaction to be included in the final state.
It would delay the moment when a complete state can be computed from the end of UserBlockSlot N-1 to some point during UserBlockSlot N, when the transaction itself is present.
But from the user standpoint, the latency is the same as in this proposal.
When a hash is included, the transaction is optimistically included.
They can also share the hash with a counterparty, if they want to show the transaction is pending before the end of UserBlockSlot N-1.

[aeyakovenko]

@ilya-bobyr leaders and validators must see the transactions before voting. They just don't need to complete execution before voting. But data must be available locally.

[ilya-bobyr]

@ilya-bobyr leaders and validators must see the transactions before voting. They just don't need to complete execution before voting. But data must be available locally.

Are you saying that sharing the whole transaction details with a delay of 1 UserBlockSlot may also delay the voting process?
In the "Fork Choice" section, it says:

the validator evaluates the forks as usual.

"As usual" means including execution, no?

I'm still relatively new to the Solana protocol. But I was under the impression, that transaction execution is the main validation performed by the validators.
Voting without execution seems like a reduction in safety.

As described in this proposal, IIUC, transactions for the last UserBlockSlot in slot N might also not be fully available for all the validators until some later point during slot N+1 processing. As the Builder only needs to share the hash with the leader, for the block to be produced.
In case when Builders are sharing transactions with a 1 UserBlockSlot delay, the situation seems similar. Though I agree, the delay might be longer. Increasing the number of UserBlockSlots per slot would reduce this delay.

[IntokuSatori]

In this vision is the Builder set mutually exclusive with the Leader set? Are these Builders envisioned to be different types of nodes, or just normal validators? If normal validators, how will a Builder be prevented from being scheduled as a Leader in the same slot? (Assuming the sets aren't mutually exclusive)

[ilya-bobyr]

I'm not the original author, but the way I understand it is:

1. Yes, Builders are selected from the pool of validators.
2. The same node is never both a Leader and a Builder.
   It seems to be just a technical detail of the selection function.

Currently, there is a function that determines the leader schedule based on the stake balances for the epoch of all the participating nodes.
This function can be arbitrary, as long as everyone can deterministically compute it and produce the same schedule.
So, it can be augmented to select 9 nodes per leader block instead of 1.
The first node will be the leader, producing 4 consecutive blocks.
And the rest 8 nodes will be split into 4 groups of 2 nodes, working as Builders in each of the 4 blocks for this leader.

And there could probably be certain details that I am overlooking.
But, in any case, I think, all these details became important only if some other questions are settled and overall the approach is decided to be the right one to address the latency issue.

[metaproph3t]

How would UserBlocks reduce latency?

For example, suppose that I am in Wellington, there's a builder in Sydney, and the leader is in Los Angeles. Either way, I want my transaction to be included in the slot, which requires it traveling to Los Angeles. Today, I would send my transaction directly to the Los Angeles leader (134ms). If this proposal is implemented, I would send my transaction to the Sydney builder (35ms). But for me to consider my transaction confirmed, the builder still needs to send it to Los Angeles (180ms), no? This would bring the total latency up to 215ms if so.

[apfitzge]

I think the idea is it reduces latency to nodes choosing which transactions make it into the block, not total confirmation time.

Let's re-use the locations in your example, and suppose I see a really great opportunity in some market.
Currently, you have to send to LA which will delay you 180+ms at least. At that point, it's more likely you missed out regardless of your priority fee because the leader will start packing their block as soon as possible so they can collect the most fees.

There's actually 2 separate aspects in this proposal that seem like they might help you get the opportunity:

1. Priority Re-ordering (could be done indepedent of 2)
2. Multiple Builders (seems dependent on 1)

Let's say we just have priority re-ordering without multiple leaders. This means if you pay a higher priority fee than anyone else going for the opportunity, you would get it...if limits were not already hit AND you made it in time. 180ms + delay is a significant fraction of the 400ms block-time, and it's possible (maybe likely) your tx doesn't make it into the leader's queue for consideration early enough to get included.

Having multiple builders increases your chances that one is closer to you make it to the closest one in time for them to include you in their user block and before limits are hit.

Not sure what @aeyakovenko intended here though - because if this UserBlock isn't transmitted early enough then the builder misses the leader's slot and you still don't get included. Maybe this only helps with the not hitting limits aspect, but you'd still have to be very early.

[7layermagik]

What is the advantage of having multiple block builders separated from a single leader vs having multiple leaders who are also responsible for buiiding blocks? Multi leader seems a bit more censorship resistant

[ilya-bobyr]

What do you mean by "multiple leaders"?
Is there a proposal describing this kind of setup?

Currently, a leader is tasked with producing the next block in the chain.
Ignoring forks, every block in the chain contains a single child block.
How would multiple parallel leaders produce a single subsequent block to be added to the chain?

[7layermagik]

I think it's actually in the original whitepaper in section 4.4 https://github.com/solana-labs/whitepaper/blob/master/solana-whitepaper-en.pdf

Most of my understanding of how it could look like over the past year or so was informed from twitter discussions :) Basically, you'd need the bankless leader setup first, and from there you would go with an architecture similar to that described in the whitepaper.

[aeyakovenko]

The multi leader approach will likely create more forks. It's an attributable fault for a leader to produce two blocks for the same slot, but with multiple leaders if there is a turbine failure, the network has a similar partition but no one is at fault.

[7layermagik]

Ahh I see. Makes sense.

I had assumed there was some benefit related to ddos resistance with having multiple leaders in the past.... I guess with single leaders what can we do to improve ddos resistance? I worry that there might be a high incentive in the future for competitors to try and "kill" solana with liveness attacks. Maybe could have "sentry nodes" in front of leaders (?). I don't think many validators would upgrade to 100gbps pipes.

[aeyakovenko]

@ilya-bobyr
Nodes evaluate forks by evaluating PoH entries to select the heaviest fork. That doesn't require executing any UserBlocks.

Sharing the UserBlock hash with the leader won't help, the rest of the network won't have it and will ignore that fork until they have the data for the UserBlocks.

[aeyakovenko]

@ilya-bobyr leader also received the UserBlocks through turbine. It requires a large percentage of colluding stake to create a partition in turbine, so if the leader receives the UserBlock it's guaranteed to have been propagated to the rest of the network as well.

[apfitzge]

Can you clarify the auction process here? I would have assumed an auction process where we take the top N bids with time-priority, not a dutch auction.

[aeyakovenko]

There are N slots to auction, so a Dutch auction makes the most sense. One bid could get 70% of the block and the other could get 30%

[apfitzge]

That doesn't sound like a Dutch auction as I understand them, so maybe I'm just getting confused by the terminology.

It'd help to give an example in the document to clarify.

[apfitzge]

Welp, since I'm the one who was confused I looked it up: https://www.investopedia.com/terms/d/dutchauction.asp

Seems to be an overloaded term, I was thinking of it as the first way they describe; i.e. a descending price auction, first bid wins.

I'm guessing that you mean the style they describe in the "Understanding Dutch Auctions for Initial Public Offerings (IPOs)" section:

Once all the bids are submitted, the allotted placement is assigned to the bidders from the highest bids down, until all the allotted shares are assigned. However, the price that each bidder pays is based on the lowest price of all the allotted bidders, or essentially the last successful bid.

[aeyakovenko]

The auction is run for the next epoch. So it takes a full epoch to finish.

[aeyakovenko]

Alternative to an auction would be extending staking to pick persistent builders. That choice might be necessary if slashing for duplicate UserBlocks is necessary

[aeyakovenko]

Slashing duplicate UserBlocks shouldn't ever be necessary, since the leader would have picked 1 of the proposed blocks and the network would either accept or reject the leaders fork. Everyone with inconsistent data can dump it and repair the right version.

[segfaultdoc]

Alternative to an auction would be extending staking to pick persistent builders. That choice might be necessary if slashing for duplicate UserBlocks is necessary

how do you prove duplicate user blocks?

[aeyakovenko]

Same as you prove duplicate blocks. Last shred is different

[mschneider]

any form of unsealed auction can be timed, so not ideal for this use-case.
sealed auctions can be realized two ways:

1. using 2 phases: commit & reveal: a working scheme is not trivial, should be a separate proposal. there are a ton of edge-cases with non-revealed bids being often equivalent to last-block voting.
2. using a trusted party, which can then MEV in novel ways, so probably not worth exploring

Can we focus for this SIMD on random builders and then make a new SIMD for a persistent builder auction? If not, why? I didn't see anything motivating this addition to the protocol.

[segfaultdoc]

how do you prevent the same N builders from winning every auction. imagine a whale builder bidding at a small loss every epoch pricing everyone out, only to earn all the mev for a volatile epoch

[aeyakovenko]

Someone else can outbid them. As long as the auction process isn't censored it's fine. That's kind of the point of persistent builders.

[hydrogenbond007]

This is kinda the same with flashbots right now with 3 block builders making 70% of the blocks because validators are looking for blocks with higher gas fees and optimized uncle rate and don't care much where the block is coming from.

[apfitzge]

Am I missing something, shouldn't this be filtering by first N bits?

If we only look at first bit, then we'd get 2 sets of distinct fee-payers, rather than N (number of concurrent builders).

[mschneider]

if we use a power of 2 to limit the number of builders we can use ln(N) bits, as error is always 0. Else using ln(N) + 4 is probably safer from a load shedding perspective (error < 6.5%)

[mschneider]

manipulating the fee payer is really annoying, for signature, I can just add a noop-instruction and rehash. why would we use fee payer and not signature?

[aeyakovenko]

@apfitzge yea, I meant N bits

[aeyakovenko]

@mschneider for well behaving users, I think its more predictable to use a fee payer that is closest to the leader. so latency sensitive users can rotate fee payers automatically

[mschneider]

technically turbine is defined as never to forward to the leader. this change needs a bit of clarification, i guess shreds are never forwarded to their builder and the leader becomes the turbine root unless builder = leader?

[mschneider]

Any well formed UserBlocks that were received from the previous or current UserBlockSlot are added to the leaders PoH ledger as UserBlockEntry.

and adds it to PoH as soon as the leaders PoH has started the UserBlockSlot.

I don't really understand how these two work together, I actually understand them as two different consensus mechanisms, could you clarify?

a) leader receives shreds over turbine, once a user block slot can be fully decoded locally, it creates a PoH entry. What's the relative order between the UserBlocks in replay? Can the leader still reorder the two UserBlocks?

b) leader receives shreds over turbine, once every 200ms a UserBlockSlot starts. the leader adds the decoded UserBlockEntry to poh if it has been received in time. What if the shreds don't arrive in time? What's the relative order between the UserBlocks in replay?

[aeyakovenko]

If the shreds don't arrive to the leader in time, they don't add the UserBlockEntry to the block.

Similarly, validators don't vote on Blocks that contain UserBlockEntries they haven't received yet.

[aeyakovenko]

Leader can add the UserBlockEntry on the next block

[0xSol]

This SIMD has been inactive for 6 months. The status changed to Stagnant per current SIMD process. Close it for now, but feel free to reopen with new updates if still relevant, or create a new SIMD as needed.