Compostability: Extracting value from “dead” spam in a system with ephemeral transaction life

[nikhayes]

Problem

Bots currently spam hundreds of thousands to millions of transactions within short periods of time. The theoretical amount of compute to process these transactions far exceeds the amount of blockspace available for these transactions to be processed before they time out, and this behavior greatly harms the UX for normal users currently. Other blockchains have higher guarantees that a sent transaction will be executed and charged through the use of mempools/nonces, which helps to disincentivize spam, along with disincentivization via higher base fees for executed transactions (which is in the works for Solana). Pricing executed transactions on a chain where transactions quickly expire doesn't offer a full economic solution to the issue of excess spam when a bot can send millions of transactions but only need to worry about the fees from a small fraction of executed transactions. Quic should help limit spam, but why not also extract more value from spamming?

Proposed Solution

Transactions currently have a finite "life" of 151 slots before a transaction timeout. For this change, a new window would be created after the initial slot window where a transaction would become "stale" rather than fully timed out (i.e. a 50 slot window). The transaction would become a "bitesize' version that could be executed and charged a "stale" fee. After this window, any transactions not processed would time out as they do currently.

When executing a “stale” transaction, only the fee payer of the transaction would be charged and the rest of the normal transaction would be ignored. This would create a very low compute transaction (apparently lower than a transfer) that would have minimal write/read locks and could therefore be processed at very high throughput (potentially higher than the 100k tps benchmarks testnet does with transfers?).
-Cherryworm had mentioned the following regarding the processing of “stale” fees from the same signer: “Should be way less compute if it's done batched” “The effort of decreasing lamports is pretty much free. The only costly thing is loading and storing accounts, and you can skip quite some steps compared to regularly loading accounts (you don't even have to acquire locks if you sum up fees for each signer). You also only have to load a single account per tx”

The fee could be based on the estimated compute of the normal transaction in full and could potentially include some form of deterministic congestion pricing (perhaps just adopting the same account-based deterministic congestion pricing that might be used in future -- this is mentioned in the issue linked below), but only normal transaction execution should be able to increment congestion prices so that validators are incentivized to pack blocks of normal transactions fully.

Stale fees would go into a pool that is split by validators at the end of the epoch, which would help disincentivize selfish validators from processing just stale transactions which are low compute. Validators would need to cooperate in some ways to track transactions that have become stale and share them with other validators (Zan @bji was thinking of some ways in which forwarding stale transactions could tie into the % earned from the fee pool at the end of epoch). Technically validators could coordinate and not process normal transactions to wait until they become stale, but this would require widespread coordination and ultimately hurt the network, which wouldn’t be in the interest of most validators. The "stale" transactions shouldn't take long to process though, so if that were the strategy, validators would spend idle time waiting around and not executing normal transactions, which doesn't make much sense economically. This is something to ponder more about though.

Another interesting point that @ruuda mentioned was that charging "stale" transactions would complement and even strengthen congestion fee dynamics (if I understood correctly). When a NFT drop begins out of the blue, users will avoid congestion fees by sending transactions with older blockhashes (cheaper but will time out sooner), which will result in congestion fees taking longer to take effect and also result in more spam. Having "stale" fees would disincentivize this behavior however because old transactions would time out sooner so there would be a greater risk of getting charged. Jon and Ruud had some more ideas in this thread #24827 (comment)

Possible Issues:
A UX concern might be that normal users could be hurt/annoyed by getting charged for timed out transactions. Users could be "reimbursed" however if validators could share these fees with their stakers. This could be built into the staking reward system somehow.

Validators delaying normal transactions to charge them as lower compute “stale” transactions instead. Discussed a bit already.

Can this be coordinated with existing transaction processing well?

Resources:
https://jstarry.notion.site/Transaction-confirmation-d5b8f4e09b9c4a70a1f263f82307d7ce#fa335d8c80464b1d957e0156c73f35d3

[nikhayes]

It might make sense for smaller validators which don't have that many leader slots to help in the process of storing transactions, to be resent to leaders once they expire. If validators continuously receive a lot of spam, they can't indefinitely forward transactions that haven't expired yet or else things will snowball. Bots could also always spam new transactions and never retry the same ones, to avoid being charged in the case only "stale" transactions are forwarded. Perhaps leaders once done with their slot can just forward unprocessed transactions to other validators (not next leader) for them to store/organize in memory until the transactions expire. Validators could cooperate in some process to create the most valuable "stale" transactions bundles to be sent to upcoming leaders. This would be a cooperative system (there's no real reason to compete with each other since they all benefit from having the best groups of stale transactions sent to leaders) and maybe some designs could be adopted from what Jito Labs is working on?

Also, if I understood correctly, @CherryWorm mentioned that these small compute transactions just charging the signer could be bundled from the same signer into batches, and this would amortize some of the compute costs (which come from loading the account mainly?). If so, then there will be a natural incentive to prioritize stale transactions from the spamming signers and send stale transactions in organized bundles.

[nikhayes]

Also, I'm curious if you have any ideas that could fit into all this @ryoqun. Was seeing some of your comments such as this one #21883 (comment)