Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upgrade]Release V6 #1261

Merged
merged 9 commits into from
Jul 7, 2023
Merged

[Upgrade]Release V6 #1261

merged 9 commits into from
Jul 7, 2023

Conversation

0xzoz
Copy link
Collaborator

@0xzoz 0xzoz commented Jul 5, 2023

This PR includes changes for the v6 release including governance, tooling and testing.

  • [Governance] Proof of Fee
    Implements the first version of Proof of Fee as per the paper https://0l.network/2022/10/15/proof-of-fee-part-1/.
    Work started after the community poll: https://0l.network/2022/11/08/october-2022-governance-recap/

    Features

    • Initialize bids, and expiry.
    • Functions for auction: Fill the seats and determine the clearing price
    • Near complete refactor of epoch boundary
      • Elect validators based on auction
      • Separate Jail Mechanics
    • Remove fee calcluator, for hardcoded Baseline Fee.
    • Functions for charging arbitrary Fees to the epoch's transaction fee account.
    • Charge Fee on Admission, goes to transaction fee account
    • Pay validators Baseline fee from chain fee account
    • Burn remainder in transaction fee account
    • limits on bids: % above baseline, unlocked in account
    • Change the weight of consensus
    • Baseline fee ratcheting based on bids
    • Future proof for case where we want to limit bid retractions
    • Stretch Goal: Priority seating based on Jail reputation

    Removals

    • Remove Cases, towers don't apply for validator compliance
    • Keep TowerState, but remove validator privileges from Proof of Height
    • NodeWeight

    Testing

    • Make unrelated tests pass
    • Bid setting and initialization
    • Epoch Boundary makes progress
    • Validator election tests
    • Correct Fee payments based on compliance
    • Vouching tests
    • Thermostat adjustment tests
    • Median calc tests
    • Burn excess fees test
    • Testnet Validator creation sanity tests

  • [Governance] Musical Chairs

    Features

    • Evaluate nodes that are compliant, and get ratio of non compliant
    • Calculate the expansion, or reduction of the set based on heuristic
    • EpochBoundary uses next validator set size, to finalize auction (proof of fee)

    Testing

    • init from genesis
    • epoch boundary
    • eval compliance
    • return new size per thresholds

  • [Move] Pledge Accounts

    Features

    • struct initialization on beneficiary project, and user accounts
    • deposit and pledge
    • beneficiary withdraw
    • logic for voting on dissolving/revoking pledge
    • use coins from accounts (not integers from scaffold)

    Testing

    • init
    • deposit
    • withdraw
    • dissolve

  • [Governance] Infrastructure Escrow

    Features

    • refactor epoch boundary
    • refactor validator rewards
    • refactor burn mechanism
    • VM initializes on genesis
    • VM withdraws
    • helper script for user to pledge

    Testing

    • init on genesis
    • withdraw on epoch boundary

  • [Tower] Miner Difficulty

    Features

    • change the difficulty
    • refactor the globals
      • changes the NODE_ENV env variable to MODE_0L. Had conflicts with node.js tools
      • creates a lazy static in ChainId.rs to get an Enum for Prod. Test, etc. Consolidates the names in one place.
      • Deprecates the TEST env variable, in favor of MODE_0L = ci
      • refactors throughout where env::var was being called directly
  • [API] Make API more useful

    Features

    • return the waypoint on index path
    • an endpoint to get a data struct and its value

  • [Tooling] Genesis Wizard

    Features

    • wizard tool for running genesis from legacy data

  • [Update] Hard Fork Migrations

    Features

    • User Accounts
      • Balance restore with split
      • Slow Wallet with coin split factor.
      • Infra Escrow
      • TowerState
      • Ancestry
      • DiemAccount Cumulative balance tracker with coin split (for Donor Directed).
      • Receipts with coin split factor (for DonorDirected)
    • Root Account
      • DonorDirected & Community Wallet list

  • [Governance] Donor Directed Account Liquidations

    Features

    • Donor directed wallets can be liquidated by donors with ParticipationVote like Veto of transactions.
    • In default case the funds return to the donor wallets
    • In the community wallet case, the funds go to TransactionFee so it follows the usual burn/recycle governance.
    • The VM needs to execute this on the epoch boundary

    Testing

    • Tx tests for whole flow of liquidation

  • [Update] Dependencies

    Features

    • Update legacy recovery.
    • Remove all diem-workspace-hack dependencies
    • Bump rust to 1.66.1

* For 0L genesis tools for hard forks, and migrations: create a MoveVm session from fake data to be able to apply migrations from Move system contracts.

* wip testing fakeexector for account migrations

* Scaffolding tests for v6 genesis fork. first cleanup deprecated modules.

* add fixtures for testing recovery genesis

* cleanup fixtures
implement functional tests for:
- exporting db backup to json
- creating genesis blob from json
- creating genesis blob in one shot from db backup

TODO: launch test node from from genesis.blob
TODO: e2e tests from a fixture file

* integration test using blob from fixture to start a diem-node in test mode

* patch github actions tests

* full e2e test from db backup to starting a node

* Add diem_logger as per @corythian plus comments.

* query nodes to check if onchain state is the same as recovery.

* - include db reader util to inspect a genesis.blob.
- refactor recovery tests

* test to check validator count. cleanup test utlis

* cleanup tests

* stub a "check" feature, to check a json file agains the output genesis.blob

* finish balance comparison with error list.

* clean

* test to create single validator and check output.

* skip comparing system zero address.

* cargo fix

* cargo fmt

* create function to start a vm executor, and uses a function pointer to transform legacy data to Move arguments.

* experimenting with alternative way of starting a vm genesis session

* runs

* cleanup

* need to explicitly set the validators at genesis

* test to confirm genesis validator set, is the same as defined set.

* tests for validator set and balance passing

* change fixtures, make node start from command line

* can start a test node from genesis blob

* can genesis tools test can start a node from a genesis and query for an accounts resources

* make ol version of swarm config generator in ol/smoke_tests

* wip

* move legacyRecovery to ol/types. So now we can run the user migrations in the vm_genesis::lib. First draft builds

* create GenesisMigration user migration test. Builds.

* ol/genesis-tools builds with refactor

* refactored with clean build

* minor refactors

* can migrate the balance of end users with new design

* migration successfully migrates balances of users, validators, and operators.

* cleanup unused files. clippy,

* tests passing

* PoF requires all nodes to have equal weight in consensus per the paper.

* initial scaffold of proof-of-fee auction. Borrows code from NodeWeight. Includes tranaction api.

* implement seat filling algo, which considers the 1/3 unproven node limit on epoch changes

* patch

* patch

* builds

* implement vm function to pay fee from user account

* all validators are charged fee at epoch boundary

* cleanup  validator set proposal functions

* refactor jail function at epoch boundary

* make epoch boundry _meta_epoch.move pass

* filter for vouches on proof of fee

* initialize val account with a PoF struct

* epoch reconfiguration meta tests passing. PoF bid initialization unit tests passing. Validator meta sanity tests passing

* patch issue with failover

* add bid restrictions, and checking upon seating

* burn excess fees than needed for validators

* reward thermostat initial implementation

* genesis init fee baseline

* patch bug in thermostat

* more tests for fill_seats, and thermostat. patches

* test

* consolidate validator audits, check for the case that there are fewer bidders than the stated val set.

* add test fixtures, modify some APIs "tell don't ask" pattern. Add several tests.

* add unit tests for thermostat. Patch a couple bugs

* tests for thermostat increase/decrease

* deprecate Audit.move, and create tests for qualifications in Proofoffee.move

* Jailing mechanism needs update after deprecating Towers. Proposing here that a Voucher is the only one to unjail, since there is no other practical hurdle.

* audit tests for vouch, expired bids, no_funds

* covered all proof of fee functions, tests for all common scenarios of fill_seats, patch logic issues, all proof of fee tests passing!

* patch seating failover

* Clean up Cases.move. Patch Vouch tests fixtures

* adjust fixtures for reconfiguration tests, deprecate some tests, patch jailing

* patch all reconfig test fixtures. remove deprecated burn tests.

* continue patching tests and removing deprecated scenarios

* All transactional tests passing!

* txs transaction apis for setting and retracting bids.

* test for set and retract

* implement vm function to pay fee from user account

* cleanup  validator set proposal functions

* refactor jail function at epoch boundary

* make epoch boundry _meta_epoch.move pass

* initialize val account with a PoF struct

* add bid restrictions, and checking upon seating

* genesis initialization of musical chairs struct

* musical chairs heuristic draft complete.

* epoch boundary calls musical chairs to set next val size

* transactional tests WIP

* patch rebase issue

* add tests for eval_compliance. all transactional tests passing!

* rename variable in Globals

* patch test, remove deprecated.

* all tests passing
* create first stubs of Pledge Accounts, a primitive service for infra escrow.

* list pledgers on beneficiary struct

* can withdraw from all accounts

* pledge account feature complete, except for withdrawing of coins

* finalize infraescrow wrapper on pledge accounts

* Vm can withdraw funds

* infra escrow can withdraw

* adding tests, patching.

* tests for pledge, withdraw unit, withdraw e2e, revoke

* change APIs to use coins, not u64

* now using coins instead of temp values

* infra escrow and pledges use coins. add helpers to withdraw coins. patch tests to use coins. all tests passing.

* make pledge withdrawals proportionate to balances

* Refactor paying of fees based on Infra Escrow. Builds. Tests need to be ported.

* reordering the epoch boundary pipeline.

* refactor burning of excess network fees at boundary

* handle zero transaction fees cases

* update .exp files on tests

* more tests for val subsidy

* patching tests

* refactor burn.move patch tests

* patch many tests fixtures

* burn epoch recycling

* for testing, make the vm genesis initialize the infra escrow pledge accounts.

* minor refactor to epoch boundary

* test to confirm the infra escrow is being funded on the epoch prologue

* revert operator subsidy removal. test framework depends on this for funding end user accounts.

* patch testmode genesis subsidies to match test fixtures.

* cleanup: info! instead of println

* fix burn init

* update burn recycling to never change supply

* all tests passing

* remove deprecated files

* patch ignored tests. All tx tests passing.
* refactoring the 0L Chain setting by console env variables.
1. changes the `NODE_ENV` env variable to `MODE_0L`. Had conflicts with node.js tools
2. creates a lazy static in ChainId.rs to get an Enum for Prod. Test, etc. Consolidates the names in one place.
3. Deprecates the `TEST` env variable, in favor of MODE_0L = ci
4. refactors throughout where env::var was being called directly

* docs on benchmarking

* wip refactor difficulty

* updating test fixtures after NamedChain renames. Tower tests pass.

* update fixtures path
* add waypoint information to the index

* api endpoint for a struct on an account.
* copy genesis_tools

* wip

* copy onboard

* surgery complete
* add tower state migration

* migrate community wallet list

* ancestry struct resources

* use coin split in genesis migration

* migrate ancestry and receipts

* migrates diemaccount cumulative tracker

* migrate slow wallet

* patch community wallet migrate

* epoch migration stub.

* slow wallet unlock with coin split factor

* cleanup names
* refactor donation tracking to also keep a struct of the addresses in one place.

* refactor the liquidation of donor directed wallets, in including community wallet case

* patch community wallet veto test

* fix nasty oversight with double counting cumulative deposits.

* community pro_rata calculation and test

* refactor vm_make_payment_no_limit to account for withdraw capabilities being delegated in multisig wallets.

* donor directed wallet liquidation flow tested.

* clean print

* all tests passing.

* Squashed commit of the following:

commit ddac86e
Author: 0o-de-lally <1364012+0o-de-lally@users.noreply.github.com>
Date:   Fri Apr 14 17:36:50 2023 -0400

    Hard Fork Migrations (#13)

    * add tower state migration

    * migrate community wallet list

    * ancestry struct resources

    * use coin split in genesis migration

    * migrate ancestry and receipts

    * migrates diemaccount cumulative tracker

    * migrate slow wallet

    * patch community wallet migrate

    * epoch migration stub.

    * slow wallet unlock with coin split factor

    * cleanup names
@sirouk
Copy link
Contributor

sirouk commented Jul 7, 2023

Excellent! Thank you! 🔥🦾

@sirouk sirouk merged commit f55227c into 0LNetworkCommunity:v6 Jul 7, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants