Improvement to gas pricing and prioritization

[ilblackdragon]

This discussion is now separated into two:

• On minimally volatile gas pricing #544
• Transaction priority fee #541

The below is outdated now.

---

Introduction

The high level goal for NEAR blockchain network design has been to have enough capacity for everyone's transactions and scale with demand. At the same time there are temporary congestions on individual shards that naturally happen when there is a spike in usage for specific contract.

Current approach increases gas price for the whole network if there is enough congestion across all shards, which doesn't address local spikes and also means that price goes up for other users in shards that are not affected by congestion. Because transactions on NEAR can run for up to 64 blocks and given potential gas price increase during this time, a multiple of actually used $NEAR is taken out from the user's account.

Gas price is also expressed as a constant amount of $NEAR, which means that increase in $NEAR price will price out some users whose accounting is in USD equivalent.

There has been alternatives discussed, including #476 but they are complex and haven't gained traction. Below proposal is trying to propose a simple change based on lots of discussions that happen by people working on NEAR Protocol.

Proposal

This proposal outlines switching to a model where transaction has a priority fee that defines prioritization of receipt execution. Global gas price is overall floor and tracking constant USD price.

This proposal has three components:

• Remove gas price increase based on chunks saturation. Instead each block producer can vote for gas price either up or down up to 1% when publishing block (e.g. block is considered valid if block[T].gas_price = block[T-1].gas_price +- 1%. The expectation is that validators will track $NEAR : $USD price and based on that adjust gas_price to maintain it minimally volatile in USD equivalent (a honest 2/3 will be able to maintain close to USD price).
• Transaction gets a priority_fee field, which defaults to 0 for old transaction format. Priority_fee fields get added to the receipt and gets propagated with all receipts created. When transaction included, the amount burnt is priority_fee + transaction.gas * block[T].gas_price. There is no refunds anymore required as there is no initial overcharging.
• Incoming receipts are sorted by descending value of priority fee, if fees are equal the earliest arrived are preferred. This sorting is used when chunks are executed.
• Given now the full fee is burnt, developers are encouraged to be more accurate about gas estimation to reduce extra spent (currently there is no reason not to specify 300Tg for every tx) which can help with execution planning.

Pros:

• No more refunds: reduces the load on the network that refunds create, users don't have to have multiple of the expected fee on their account.
• Prioritization where it is needed: DeFi transactions, liquidations and similar highly time sensitive operations can be prioritized by the sender. For the network this captures additional value as this priority fee serves as an auction to discover what is the value arbitrage/liquidators are willing to pay.
• Regular gas price will be minimally volatile in terms of USD, meaning that independent of $NEAR price changes users and apps will have predictable costs.
• As intra-shard parallelism added, the priority fee can be even more directed toward single contract's capacity for congested contracts.

Cons:

• Receipt priority queue is complex and has additional computational complexity.
• Priority fee form an auction. Ethereum had challenges with a direct fee auction when network is constantly congested as there was no good way to predict the lowest fee included. The main counter point here is that we expect NEAR network to be mostly non congested (e.g. network will reshard if there is constant congestion in one of the shards) and so priority fee is really about priority vs inclusion.

Protocol Changes

• Change SignedTransaction to be versioned and add priority_fee field
• Change gas_price verification logic to not use the previous gas_used.
• Add priority queue to receipts (see implementation detail note below)
• Remove refunds

Implementation detail, that it's expected that each shard will also be multi-threaded when executing transaction as each smart contract is independent and their receipts can be executed in parallel. Meaning, it might be beneficial to execute lower priority fee receipt in parallel to higher priority fee on a different contract. Incoming receipt data structure should support a way to easily stratify by receiver_id.

Not directly protocol change, but to add to the node logic to fetch $NEAR:$USD price and config for which APIs to use (can be on-chain from Ref pool for example). This should define if when block is produced gas_price should be increased or decreased by 1% by given validator. This can be done as a separate work item and potentially maintained externally to nearcore codebase by specifying that price will come from another binary/service. Side benefit that this forms an oracle for $NEAR price in $USD.

User interface

Wallets can provide a pretty convenient UI to the end user by showing:

• Standard -- fixed fee based on gas in $USD -- expected time to execution (normally 2s as well but can change)
• Priority -- +$X -- 2s

Uber eat example

Alternatives

Independent gas markets per shard

A more properly priced auction for resources (see Dynamic Pricing for Non-fungible Resources:
Designing Multidimensional Blockchain Fee Markets and Proof that EIP-1559 is online optimization) is to have every shard to have individual fee market with some cross-shard update for accounting for cross-shard overall usage.

The challenge in this approach, is that when transaction is included into the block node doesn't know which shards it will travel to and what gas prices will be at that moment. Which means protocol would need to sufficiently overcharge for 64 blocks in the future from maximum price across all shards (current approach) and keep track at which price gas was spent on which shard to refund back the left overs.

Alternatively the receipts can hold "NEAR" instead of gas as they travel between shards. This NEAR is used to buy gas on arrival to the shard. After receipt is executed, the left over gas is "sold" back to get the left over NEAR and it gets split between outgoing receipts. This has major downside that if some shard has very high prices, the outgoing receipts may not have any NEAR left even though there was supposedly still "gas" to send.

[starpause]

If comparable, is the performance gain from "No more refunds: reduces the load on the network that refunds create" higher than the cost of "Receipt priority queue is complex and has additional computational complexity"?

[ilblackdragon]

Priority queue is needed independent if there are refunds are not - priority fees are needed when there is contested contract / account like Burrow for liquidations or Ref for swaps.

[ilblackdragon]

This discussion is now separated into two:

• On minimally volatile gas pricing #544
• Transaction priority fee #541