Gas Price Algorithm #1948

MitchTurner · 2024-06-07T16:23:44Z

Closes: #1957

Part of #1624

Add the gas price algorithm with some tools for analyzing its behavior.

The algorithm is made up of two portions:

AlgorithmUpdaterV1
AlgorithmV1

The updater holds the state of the historical data and can generate new algorithms as its data is updated. The algorithm just takes the block_bytes and can calculate the price from that data.

I've included a lot of docs on the actual code, so see that for more details. LMK if something is missing :)

Checklist

Breaking changes are clearly marked as such in the PR description and changelog
New behavior is reflected in tests
The specification matches the implemented behavior (link update PR if changes are needed)

Before requesting review

I have reviewed the code myself
I have created follow-up issues caused by this PR and linked them here

After merging, notify other teams

[Add or remove entries as needed]

Rust SDK
Sway compiler
Platform documentation (for out-of-organization contributors, the person merging the PR will do this)
Someone else?

…ce for previous block

Dentosal · 2024-06-18T11:51:54Z

crates/gas-price-algorithm/gas-price-analysis/Cargo.toml

+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+


Suggested change

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

I just removed this.

Dentosal · 2024-06-18T16:09:42Z

crates/gas-price-algorithm/gas-price-analysis/Cargo.toml

+name = "gas-price-analysis"
+version = "0.1.0"
+edition = "2021"
+


Suggested change

publish = false

Dentosal · 2024-06-18T16:11:02Z

crates/gas-price-algorithm/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "gas-price-algorithm"


Suggested change

name = "gas-price-algorithm"

name = "fuel-gas-price-algorithm"

Dentosal · 2024-06-18T16:11:23Z

Cargo.toml

@@ -81,6 +83,7 @@ fuel-core-tests = { version = "0.0.0", path = "./tests" }
 fuel-core-upgradable-executor = { version = "0.30.0", path = "./crates/services/upgradable-executor" }
 fuel-core-wasm-executor = { version = "0.30.0", path = "./crates/services/upgradable-executor/wasm-executor", default-features = false }
 fuel-core-xtask = { version = "0.0.0", path = "./xtask" }
+gas-price-algorithm = { version = "0.29.0", path = "./crates/gas-price-algorithm" }


Suggested change

gas-price-algorithm = { version = "0.29.0", path = "./crates/gas-price-algorithm" }

fuel-gas-price-algorithm = { version = "0.29.0", path = "./crates/gas-price-algorithm" }

Dentosal · 2024-06-18T16:11:50Z

crates/gas-price-algorithm/gas-price-analysis/Cargo.toml

@@ -0,0 +1,14 @@
+[package]
+name = "gas-price-analysis"


Suggested change

name = "gas-price-analysis"

name = "fuel-gas-price-analysis"

MitchTurner · 2024-06-18T19:31:07Z

@Dentosal I made some small logical changes wrt minimum gas prices.

crates/fuel-gas-price-algorithm/src/lib.rs

Dentosal

Still some change suggestions unapplied

@Dentosal

## Version v0.31.0 ### Added - [#2014](#2014): Added a separate thread for the block importer. - [#2013](#2013): Added a separate thread to process P2P database lookups. - [#2004](#2004): Added new CLI argument `continue-services-on-error` to control internal flow of services. - [#2004](#2004): Added handling of incorrect shutdown of the off-chain GraphQL worker by using state rewind feature. - [#2007](#2007): Improved metrics: - Added database metrics per column. - Added statistic about commit time of each database. - Refactored how metrics are registered: Now, we use only one register shared between all metrics. This global register is used to encode all metrics. - [#1996](#1996): Added support for rollback command when state rewind feature is enabled. The command allows the rollback of the state of the blockchain several blocks behind until the end of the historical window. The default historical window it 7 days. - [#1996](#1996): Added support for the state rewind feature. The feature allows the execution of the blocks in the past and the same execution results to be received. Together with forkless upgrades, execution of any block from the past is possible if historical data exist for the target block height. - [#1994](#1994): Added the actual implementation for the `AtomicView::latest_view`. - [#1972](#1972): Implement `AlgorithmUpdater` for `GasPriceService` - [#1948](#1948): Add new `AlgorithmV1` and `AlgorithmUpdaterV1` for the gas price. Include tools for analysis - [#1676](#1676): Added new CLI arguments: - `graphql-max-depth` - `graphql-max-complexity` - `graphql-max-recursive-depth` ### Changed - [#2015](#2015): Small fixes for the database: - Fixed the name for historical columns - Metrics was working incorrectly for historical columns. - Added recommended setting for the RocksDB - The source of recommendation is official documentation https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning#other-general-options. - Removed repairing since it could corrupt the database if fails - Several users reported about the corrupted state of the database after having a "Too many descriptors" error where in logs, repairing of the database also failed with this error creating a `lost` folder. - [#2010](#2010): Updated the block importer to allow more blocks to be in the queue. It improves synchronization speed and mitigate the impact of other services on synchronization speed. - [#2006](#2006): Process block importer events first under P2P pressure. - [#2002](#2002): Adapted the block producer to react to checked transactions that were using another version of consensus parameters during validation in the TxPool. After an upgrade of the consensus parameters of the network, TxPool could store invalid `Checked` transactions. This change fixes that by tracking the version that was used to validate the transactions. - [#1999](#1999): Minimize the number of panics in the codebase. - [#1990](#1990): Use latest view for mutate GraphQL queries after modification of the node. - [#1992](#1992): Parse multiple relayer contracts, `RELAYER-V2-LISTENING-CONTRACTS` env variable using a `,` delimiter. - [#1980](#1980): Add `Transaction` to relayer 's event filter #### Breaking - [#2012](#2012): Bumped the `fuel-vm` to `0.55.0` release. More about the change [here](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.55.0). - [#2001](#2001): Prevent GraphQL query body to be huge and cause OOM. The default body size is `1MB`. The limit can be changed by the `graphql-request-body-bytes-limit` CLI argument. - [#1991](#1991): Prepare the database to use different types than `Database` for atomic view. - [#1989](#1989): Extract `HistoricalView` trait from the `AtomicView`. - [#1676](#1676): New `fuel-core-client` is incompatible with the old `fuel-core` because of two requested new fields. - [#1676](#1676): Changed default value for `api-request-timeout` to be `30s`. - [#1676](#1676): Now, GraphQL API has complexity and depth limitations on the queries. The default complexity limit is `20000`. It is ~50 blocks per request with transaction IDs and ~2-5 full blocks. ### Fixed - [#2000](#2000): Use correct query name in metrics for aliased queries. ## What's Changed * Generate and publish code coverage reports in the CI by @Dentosal in #1947 * Gas Price Algorithm by @MitchTurner in #1948 * Use companies fork of the `publish-crates` action by @xgreenx in #1986 * Weekly `cargo update` by @github-actions in #1985 * Implement gas price updater for service by @MitchTurner in #1972 * Extract `HistoricalView` trait from the `AtomicView` by @xgreenx in #1989 * Use fresh `ReadView` for mutate queries by @xgreenx in #1990 * Prevent api spam with GQL complexity limits by @Voxelot in #1676 * Enable parsing multiple relayer listening contract addresses from environment variables by @Jurshsmith in #1992 * Prepare the database to use different types than `Database` for atomic view by @xgreenx in #1991 * Added the actual implementation for the `AtomicView::latest_view` by @xgreenx in #1994 * Weekly `cargo update` by @github-actions in #1998 * Minimize the number of panics in the codebase by @xgreenx in #1999 * feat: include Transaction events in topic0 filter for download_logs by @DefiCake in #1980 * Use correct query name for metrics by @xgreenx in #2000 * Prevent GraphQL query body to be huge and cause OOM by @xgreenx in #2001 * Adapted the block producer to react on the outdated transactions from the TxPool by @xgreenx in #2002 * Process block importer events first under P2P pressure by @xgreenx in #2006 * Implementation of the state rewind feature for the RocksDB by @xgreenx in #1996 * Upgraded `fuel-vm` to `0.55.0` by @xgreenx in #2012 * Improved metrics for the database by @xgreenx in #2007 * Updated block importer to allow more blocks to be queue by @xgreenx in #2010 * Added handling of incorrect shutdown of the off-chain GraphQL worker by @xgreenx in #2004 * Moved P2P database lookups into a separate thread by @xgreenx in #2013 * Use dedicated thread for the block importer by @xgreenx in #2014 * Small fixes for the database by @xgreenx in #2015 ## New Contributors * @Jurshsmith made their first contribution in #1992 * @DefiCake made their first contribution in #1980 **Full Changelog**: v0.30.0...v0.31.0

Add the algorithm code

ac1aa41

MitchTurner changed the title ~~Add the algorithm code~~ Gas Price Algorithm Jun 7, 2024

MitchTurner and others added 9 commits June 7, 2024 11:03

WIP

bc9efa0

Add some tests to define the updater behavior

31ea5c4

Add a couple more tests, WIP getting the second to pass

7e6546a

Fix test, refactor out single update into two

6707c3e

Refactor

09a0c95

WIP rewrite the chart generation stuff

da3c887

Add profit stuff to chart, add rewards

cdfc206

Add some naive profit corrections

15a45ed

Remove the old stuff that is not being used

03c6a4a

MitchTurner self-assigned this Jun 10, 2024

MitchTurner added 5 commits June 10, 2024 13:10

Add actual algo that the updater generates, fix tests accordingly

459f8fd

Update gas price with l2 block data

b713739

Use algo generation in simulation

91fe558

Straighten out the profit chart to be aligned after every da_recording

5c3c277

Fix the noise

da7ea98

MitchTurner mentioned this pull request Jun 11, 2024

Implement our gas price algorithm for our GasPriceService #1955

Closed

MitchTurner added 12 commits June 11, 2024 14:35

Refactor to calculate exec gas price eagarly and calculate da gas pri…

f05c594

…ce for previous block

Refactor a bit to allow p and d values, rename some stuff

644c2da

Add d value to tests

9c40f42

WIP

0783439

Use real gas limit values stablize algo a bit.

d7d7d55

Set params to somewhat steady state

106569d

Add average profit values

6db37a6

Cleanup, add naive optimisation

781424f

Refactor

88793e9

Replace d value :), start adding max change

b7c10c6

Add change amount tests, remove unused params

e245f98

Remove avg window

95ed269

xgreenx requested a review from Dentosal June 18, 2024 11:50

Dentosal reviewed Jun 18, 2024

View reviewed changes

MitchTurner added 2 commits June 18, 2024 10:29

Update lock file

305ceaa

Fix crate names, lint

1f2be1c

MitchTurner requested a review from Dentosal June 18, 2024 17:58

MitchTurner added 3 commits June 18, 2024 11:02

Update docs

ebae0f7

Move min gas price to exec to avoid error

0cc38cb

Add minimum da gas price

209e3b8

MitchTurner commented Jun 18, 2024

View reviewed changes

crates/fuel-gas-price-algorithm/src/lib.rs Outdated Show resolved Hide resolved

Update crates/fuel-gas-price-algorithm/src/lib.rs

395a91b

MitchTurner commented Jun 18, 2024

View reviewed changes

crates/fuel-gas-price-algorithm/src/lib.rs Outdated Show resolved Hide resolved

Update crates/fuel-gas-price-algorithm/src/lib.rs

31450f9

Dentosal requested changes Jun 18, 2024

View reviewed changes

Fix unchanged name, get anaysis running again

158b663

MitchTurner requested a review from Dentosal June 18, 2024 22:28

Dentosal previously approved these changes Jun 18, 2024

View reviewed changes

Update bytecode version

6caa7f7

MitchTurner dismissed Dentosal’s stale review via 6caa7f7 June 19, 2024 17:06

MitchTurner enabled auto-merge (squash) June 19, 2024 17:28

MitchTurner requested a review from Dentosal June 19, 2024 17:34

Dentosal approved these changes Jun 19, 2024

View reviewed changes

MitchTurner merged commit 4d6167d into master Jun 19, 2024
32 of 33 checks passed

MitchTurner deleted the feature/gas-price-algo branch June 19, 2024 17:36

xgreenx mentioned this pull request Jul 5, 2024

Release v0.31.0 #2016

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gas Price Algorithm #1948

Gas Price Algorithm #1948

MitchTurner commented Jun 7, 2024 •

edited

Loading

Dentosal Jun 18, 2024

MitchTurner Jun 18, 2024

Dentosal Jun 18, 2024

Dentosal Jun 18, 2024

Dentosal Jun 18, 2024

Dentosal Jun 18, 2024

MitchTurner commented Jun 18, 2024 •

edited

Loading

Dentosal left a comment

		# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

	name = "gas-price-algorithm"
	name = "fuel-gas-price-algorithm"

	gas-price-algorithm = { version = "0.29.0", path = "./crates/gas-price-algorithm" }
	fuel-gas-price-algorithm = { version = "0.29.0", path = "./crates/gas-price-algorithm" }

Gas Price Algorithm #1948

Gas Price Algorithm #1948

Conversation

MitchTurner commented Jun 7, 2024 • edited Loading

Checklist

Before requesting review

After merging, notify other teams

Dentosal Jun 18, 2024

Choose a reason for hiding this comment

MitchTurner Jun 18, 2024

Choose a reason for hiding this comment

Dentosal Jun 18, 2024

Choose a reason for hiding this comment

Dentosal Jun 18, 2024

Choose a reason for hiding this comment

Dentosal Jun 18, 2024

Choose a reason for hiding this comment

Dentosal Jun 18, 2024

Choose a reason for hiding this comment

MitchTurner commented Jun 18, 2024 • edited Loading

Dentosal left a comment

Choose a reason for hiding this comment

MitchTurner commented Jun 7, 2024 •

edited

Loading

MitchTurner commented Jun 18, 2024 •

edited

Loading