Skip to content

Commit

Permalink
Proof of Fee (#5)
Browse files Browse the repository at this point in the history
* For 0L genesis tools for hard forks, and migrations: create a MoveVm session from fake data to be able to apply migrations from Move system contracts.

* wip testing fakeexector for account migrations

* Scaffolding tests for v6 genesis fork. first cleanup deprecated modules.

* add fixtures for testing recovery genesis

* cleanup fixtures
implement functional tests for:
- exporting db backup to json
- creating genesis blob from json
- creating genesis blob in one shot from db backup

TODO: launch test node from from genesis.blob
TODO: e2e tests from a fixture file

* integration test using blob from fixture to start a diem-node in test mode

* patch github actions tests

* full e2e test from db backup to starting a node

* Add diem_logger as per @corythian plus comments.

* query nodes to check if onchain state is the same as recovery.

* - include db reader util to inspect a genesis.blob.
- refactor recovery tests

* test to check validator count. cleanup test utlis

* cleanup tests

* stub a "check" feature, to check a json file agains the output genesis.blob

* finish balance comparison with error list.

* clean

* test to create single validator and check output.

* skip comparing system zero address.

* cargo fix

* cargo fmt

* create function to start a vm executor, and uses a function pointer to transform legacy data to Move arguments.

* experimenting with alternative way of starting a vm genesis session

* runs

* cleanup

* need to explicitly set the validators at genesis

* test to confirm genesis validator set, is the same as defined set.

* tests for validator set and balance passing

* change fixtures, make node start from command line

* can start a test node from genesis blob

* can genesis tools test can start a node from a genesis and query for an accounts resources

* make ol version of swarm config generator in ol/smoke_tests

* wip

* move legacyRecovery to ol/types. So now we can run the user migrations in the vm_genesis::lib. First draft builds

* create GenesisMigration user migration test. Builds.

* ol/genesis-tools builds with refactor

* refactored with clean build

* minor refactors

* can migrate the balance of end users with new design

* migration successfully migrates balances of users, validators, and operators.

* cleanup unused files. clippy,

* tests passing

* PoF requires all nodes to have equal weight in consensus per the paper.

* Update ci.yaml (#1239)

* [Genesis] Migration tools for genesis from legacy accounts (#1237)

* For 0L genesis tools for hard forks, and migrations: create a MoveVm session from fake data to be able to apply migrations from Move system contracts.

* wip testing fakeexector for account migrations

* Scaffolding tests for v6 genesis fork. first cleanup deprecated modules.

* add fixtures for testing recovery genesis

* cleanup fixtures
implement functional tests for:
- exporting db backup to json
- creating genesis blob from json
- creating genesis blob in one shot from db backup

TODO: launch test node from from genesis.blob
TODO: e2e tests from a fixture file

* integration test using blob from fixture to start a diem-node in test mode

* patch github actions tests

* full e2e test from db backup to starting a node

* Add diem_logger as per @corythian plus comments.

* query nodes to check if onchain state is the same as recovery.

* - include db reader util to inspect a genesis.blob.
- refactor recovery tests

* test to check validator count. cleanup test utlis

* cleanup tests

* stub a "check" feature, to check a json file agains the output genesis.blob

* finish balance comparison with error list.

* clean

* test to create single validator and check output.

* skip comparing system zero address.

* cargo fix

* cargo fmt

* create function to start a vm executor, and uses a function pointer to transform legacy data to Move arguments.

* experimenting with alternative way of starting a vm genesis session

* runs

* cleanup

* need to explicitly set the validators at genesis

* test to confirm genesis validator set, is the same as defined set.

* tests for validator set and balance passing

* change fixtures, make node start from command line

* can start a test node from genesis blob

* can genesis tools test can start a node from a genesis and query for an accounts resources

* make ol version of swarm config generator in ol/smoke_tests

* wip

* move legacyRecovery to ol/types. So now we can run the user migrations in the vm_genesis::lib. First draft builds

* create GenesisMigration user migration test. Builds.

* ol/genesis-tools builds with refactor

* refactored with clean build

* minor refactors

* can migrate the balance of end users with new design

* migration successfully migrates balances of users, validators, and operators.

* cleanup unused files. clippy,

* tests passing

* whitespace to trigger ci

* initial scaffold of proof-of-fee auction. Borrows code from NodeWeight. Includes tranaction api.

* implement seat filling algo, which considers the 1/3 unproven node limit on epoch changes

* patch

* patch

* builds

* implement vm function to pay fee from user account

* all validators are charged fee at epoch boundary

* cleanup  validator set proposal functions

* refactor jail function at epoch boundary

* make epoch boundry _meta_epoch.move pass

* filter for vouches on proof of fee

* initialize val account with a PoF struct

* epoch reconfiguration meta tests passing. PoF bid initialization unit tests passing. Validator meta sanity tests passing

* patch issue with failover

* add bid restrictions, and checking upon seating

* burn excess fees than needed for validators

* reward thermostat initial implementation

* genesis init fee baseline

* patch bug in thermostat

* more tests for fill_seats, and thermostat. patches

* test

* consolidate validator audits, check for the case that there are fewer bidders than the stated val set.

* add test fixtures, modify some APIs "tell don't ask" pattern. Add several tests.

* add unit tests for thermostat. Patch a couple bugs

* tests for thermostat increase/decrease

* deprecate Audit.move, and create tests for qualifications in Proofoffee.move

* Jailing mechanism needs update after deprecating Towers. Proposing here that a Voucher is the only one to unjail, since there is no other practical hurdle.

* audit tests for vouch, expired bids, no_funds

* covered all proof of fee functions, tests for all common scenarios of fill_seats, patch logic issues, all proof of fee tests passing!

* patch seating failover

* Clean up Cases.move. Patch Vouch tests fixtures

* adjust fixtures for reconfiguration tests, deprecate some tests, patch jailing

* patch all reconfig test fixtures. remove deprecated burn tests.

* continue patching tests and removing deprecated scenarios

* All transactional tests passing!

* txs transaction apis for setting and retracting bids.

* test for set and retract

* cleanup prints

* add struct in TransactionFee to track origin of fees FeeMaker

* refactor Burn to consider the preference of the user FeeMaker

* cleanup

* init FeeMaker on account creation

* patch tests to remove `print`

* patch epoch boundary burn ratios

* all tests passing

---------

Co-authored-by: Zoz <97761083+0xzoz@users.noreply.github.com>
  • Loading branch information
0o-de-lally and 0xzoz committed Jul 5, 2023
1 parent dc5f26d commit a6c99f1
Show file tree
Hide file tree
Showing 298 changed files with 8,038 additions and 11,721 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,6 @@ compiled_package_info:
? address: "00000000000000000000000000000001"
name: Ancestry
: DiemFramework
? address: "00000000000000000000000000000001"
name: Audit
: DiemFramework
? address: "00000000000000000000000000000001"
name: Authenticator
: DiemFramework
Expand Down Expand Up @@ -223,6 +220,9 @@ compiled_package_info:
? address: "00000000000000000000000000000001"
name: PersistenceDemo
: DiemFramework
? address: "00000000000000000000000000000001"
name: ProofOfFee
: DiemFramework
? address: "00000000000000000000000000000001"
name: Receipts
: DiemFramework
Expand Down Expand Up @@ -346,7 +346,7 @@ compiled_package_info:
? address: "00000000000000000000000000000001"
name: XUS
: DiemFramework
source_digest: E73442AD7A1B65175ABE2ACCB60D6A59204FBFD1C63E20E767E29AAC56BD3285
source_digest: 110A62F23F4C530726F3CD908ED8A7393D8E6E9D5B8E6F2543C536D0C02458CB
build_flags:
dev_mode: false
test_mode: false
Expand Down
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@


<pre><code><b>use</b> <a href="CoreAddresses.md#0x1_CoreAddresses">0x1::CoreAddresses</a>;
<b>use</b> <a href="Debug.md#0x1_Debug">0x1::Debug</a>;
<b>use</b> <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Option.md#0x1_Option">0x1::Option</a>;
<b>use</b> <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Signer.md#0x1_Signer">0x1::Signer</a>;
<b>use</b> <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector">0x1::Vector</a>;
Expand Down Expand Up @@ -67,7 +66,7 @@


<pre><code><b>public</b> <b>fun</b> <a href="Ancestry.md#0x1_Ancestry_init">init</a>(new_account_sig: &signer, onboarder_sig: &signer ) <b>acquires</b> <a href="Ancestry.md#0x1_Ancestry">Ancestry</a>{
print(&100100);
// print(&100100);
<b>let</b> parent = <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Signer.md#0x1_Signer_address_of">Signer::address_of</a>(onboarder_sig);
<a href="Ancestry.md#0x1_Ancestry_set_tree">set_tree</a>(new_account_sig, parent);
}
Expand All @@ -94,7 +93,7 @@

<pre><code><b>fun</b> <a href="Ancestry.md#0x1_Ancestry_set_tree">set_tree</a>(new_account_sig: &signer, parent: <b>address</b> ) <b>acquires</b> <a href="Ancestry.md#0x1_Ancestry">Ancestry</a> {
<b>let</b> child = <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Signer.md#0x1_Signer_address_of">Signer::address_of</a>(new_account_sig);
print(&100200);
// print(&100200);
<b>let</b> new_tree = <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_empty">Vector::empty</a>&lt;<b>address</b>&gt;();

// get the parent's ancestry <b>if</b> initialized.
Expand All @@ -103,31 +102,31 @@
<b>if</b> (<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(parent)) {
<b>let</b> parent_state = <b>borrow_global_mut</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(parent);
<b>let</b> parent_tree = *&parent_state.tree;
print(&100210);
// print(&100210);
<b>if</b> (<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_length">Vector::length</a>&lt;<b>address</b>&gt;(&parent_tree) &gt; 0) {
<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_append">Vector::append</a>(&<b>mut</b> new_tree, parent_tree);
};
print(&100220);
// print(&100220);
};

// add the parent <b>to</b> the tree
<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_push_back">Vector::push_back</a>(&<b>mut</b> new_tree, parent);
print(&100230);
// print(&100230);

<b>if</b> (!<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(child)) {
<b>move_to</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(new_account_sig, <a href="Ancestry.md#0x1_Ancestry">Ancestry</a> {
tree: new_tree,
});
print(&100240);
// print(&100240);

} <b>else</b> {
// this is only for migration cases.
<b>let</b> child_ancestry = <b>borrow_global_mut</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(child);
child_ancestry.tree = new_tree;
print(&100250);
// print(&100250);

};
print(&100260);
// print(&100260);

}
</code></pre>
Expand Down Expand Up @@ -183,40 +182,40 @@
<pre><code><b>public</b> <b>fun</b> <a href="Ancestry.md#0x1_Ancestry_is_family">is_family</a>(left: <b>address</b>, right: <b>address</b>): (bool, <b>address</b>) <b>acquires</b> <a href="Ancestry.md#0x1_Ancestry">Ancestry</a> {
<b>let</b> is_family = <b>false</b>;
<b>let</b> common_ancestor = @0x0;
print(&100300);
print(&<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(left));
print(&<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(right));
// // print(&100300);
// // print(&<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(left));
// // print(&<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(right));

// <b>if</b> (<b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(left) && <b>exists</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(right)) {
// <b>if</b> tree is empty it will still work.
print(&100310);
// // print(&100310);
<b>let</b> left_tree = <a href="Ancestry.md#0x1_Ancestry_get_tree">get_tree</a>(left);
print(&100311);
// // print(&100311);
<b>let</b> right_tree = <a href="Ancestry.md#0x1_Ancestry_get_tree">get_tree</a>(right);

print(&100320);
// // print(&100320);

// check for direct relationship.
<b>if</b> (<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_contains">Vector::contains</a>(&left_tree, &right)) <b>return</b> (<b>true</b>, right);
<b>if</b> (<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_contains">Vector::contains</a>(&right_tree, &left)) <b>return</b> (<b>true</b>, left);

print(&100330);
// // print(&100330);
<b>let</b> i = 0;
// check every <b>address</b> on the list <b>if</b> there are overlaps.
<b>while</b> (i &lt; <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_length">Vector::length</a>&lt;<b>address</b>&gt;(&left_tree)) {
print(&100341);
// // print(&100341);
<b>let</b> family_addr = <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_borrow">Vector::borrow</a>(&left_tree, i);
<b>if</b> (<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_contains">Vector::contains</a>(&right_tree, family_addr)) {
is_family = <b>true</b>;
common_ancestor = *family_addr;
print(&100342);
// // print(&100342);
<b>break</b>
};
i = i + 1;
};
print(&100350);
// // print(&100350);
// };
print(&100360);
// // print(&100360);
(is_family, common_ancestor)
}
</code></pre>
Expand Down Expand Up @@ -322,13 +321,13 @@
<b>move_to</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(child_sig, <a href="Ancestry.md#0x1_Ancestry">Ancestry</a> {
tree: migrate_tree,
});
print(&100240);
// print(&100240);

} <b>else</b> {
// this is only for migration cases.
<b>let</b> child_ancestry = <b>borrow_global_mut</b>&lt;<a href="Ancestry.md#0x1_Ancestry">Ancestry</a>&gt;(child);
child_ancestry.tree = migrate_tree;
print(&100250);
// print(&100250);

};
}
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ This module enables automatic payments from accounts to community wallets at epo
- [Function `find`](#0x1_AutoPay_find)


<pre><code><b>use</b> <a href="CommunityWallet.md#0x1_CommunityWallet">0x1::CommunityWallet</a>;
<b>use</b> <a href="DiemAccount.md#0x1_DiemAccount">0x1::DiemAccount</a>;
<pre><code><b>use</b> <a href="DiemAccount.md#0x1_DiemAccount">0x1::DiemAccount</a>;
<b>use</b> <a href="DiemConfig.md#0x1_DiemConfig">0x1::DiemConfig</a>;
<b>use</b> <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Errors.md#0x1_Errors">0x1::Errors</a>;
<b>use</b> <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/FixedPoint32.md#0x1_FixedPoint32">0x1::FixedPoint32</a>;
Expand Down Expand Up @@ -352,16 +351,6 @@ Maximum value for the Payment type selection



<a name="0x1_AutoPay_PAYEE_NOT_COMMUNITY_WALLET"></a>

Attempt to make a payment to a non-community-wallet


<pre><code><b>const</b> <a href="AutoPay.md#0x1_AutoPay_PAYEE_NOT_COMMUNITY_WALLET">PAYEE_NOT_COMMUNITY_WALLET</a>: u64 = 10024;
</code></pre>



<a name="0x1_AutoPay_PERCENT_OF_BALANCE"></a>

send percent of balance at end of epoch payment type
Expand Down Expand Up @@ -601,7 +590,6 @@ Attempt to use a UID that is already taken
<b>let</b> bal_change_since_last_run = <b>if</b> (pre_run_bal &gt; my_autopay_state.prev_bal) {
pre_run_bal - my_autopay_state.prev_bal
} <b>else</b> { 0 };

// go through the pledges
<b>while</b> (payments_idx &lt; payments_len) {
<b>let</b> payment = <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Vector.md#0x1_Vector_borrow_mut">Vector::borrow_mut</a>&lt;<a href="AutoPay.md#0x1_AutoPay_Payment">Payment</a>&gt;(payments, payments_idx);
Expand Down Expand Up @@ -648,10 +636,6 @@ Attempt to use a UID that is already taken
payment: &<b>mut</b> <a href="AutoPay.md#0x1_AutoPay_Payment">Payment</a>,
bal_change_since_last_run: u64,
): bool {
// check payees are community wallets, only community wallets are allowed
// <b>to</b> receive autopay (bypassing account limits)
<b>if</b> (!<a href="CommunityWallet.md#0x1_CommunityWallet_is_comm">CommunityWallet::is_comm</a>(payment.payee)) { <b>return</b> <b>false</b> }; // do nothing but don't delete instruction };

<a href="Roles.md#0x1_Roles_assert_diem_root">Roles::assert_diem_root</a>(vm);
<b>let</b> epoch = <a href="DiemConfig.md#0x1_DiemConfig_get_current_epoch">DiemConfig::get_current_epoch</a>();
<b>let</b> account_bal = <a href="DiemAccount.md#0x1_DiemAccount_balance">DiemAccount::balance</a>&lt;<a href="GAS.md#0x1_GAS">GAS</a>&gt;(*account_addr);
Expand Down Expand Up @@ -687,7 +671,7 @@ Attempt to use a UID that is already taken
};

<b>if</b> (amount != 0 && amount &lt;= account_bal) {
<a href="DiemAccount.md#0x1_DiemAccount_vm_make_payment_no_limit">DiemAccount::vm_make_payment_no_limit</a>&lt;<a href="GAS.md#0x1_GAS">GAS</a>&gt;(
<a href="DiemAccount.md#0x1_DiemAccount_vm_pay_from">DiemAccount::vm_pay_from</a>&lt;<a href="GAS.md#0x1_GAS">GAS</a>&gt;(
*account_addr, payment.payee, amount, b"autopay", b"", vm
);
};
Expand Down Expand Up @@ -807,16 +791,16 @@ Attempt to use a UID that is already taken
payee: <b>address</b>,
end_epoch: u64,
amt: u64
) <b>acquires</b> <a href="AutoPay.md#0x1_AutoPay_UserAutoPay">UserAutoPay</a>, <a href="AutoPay.md#0x1_AutoPay_AccountLimitsEnable">AccountLimitsEnable</a> {
) <b>acquires</b> <a href="AutoPay.md#0x1_AutoPay_UserAutoPay">UserAutoPay</a> {
<b>let</b> addr = <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Signer.md#0x1_Signer_address_of">Signer::address_of</a>(sender);
// Confirm that no payment <b>exists</b> <b>with</b> the same uid
<b>let</b> index = <a href="AutoPay.md#0x1_AutoPay_find">find</a>(addr, uid);
<b>assert</b>!(<a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Option.md#0x1_Option_is_none">Option::is_none</a>&lt;u64&gt;(&index), <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Errors.md#0x1_Errors_invalid_argument">Errors::invalid_argument</a>(<a href="AutoPay.md#0x1_AutoPay_UID_TAKEN">UID_TAKEN</a>));

// TODO: This check already <b>exists</b> at the time of execution.
<b>if</b> (<b>borrow_global</b>&lt;<a href="AutoPay.md#0x1_AutoPay_AccountLimitsEnable">AccountLimitsEnable</a>&gt;(@DiemRoot).enabled) {
<b>assert</b>!(<a href="CommunityWallet.md#0x1_CommunityWallet_is_comm">CommunityWallet::is_comm</a>(payee), <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Errors.md#0x1_Errors_invalid_argument">Errors::invalid_argument</a>(<a href="AutoPay.md#0x1_AutoPay_PAYEE_NOT_COMMUNITY_WALLET">PAYEE_NOT_COMMUNITY_WALLET</a>));
};
// // TODO: This check already <b>exists</b> at the time of execution.
// <b>if</b> (<b>borrow_global</b>&lt;<a href="AutoPay.md#0x1_AutoPay_AccountLimitsEnable">AccountLimitsEnable</a>&gt;(@DiemRoot).enabled) {
// <b>assert</b>!(<a href="CommunityWallet.md#0x1_CommunityWallet_is_comm">CommunityWallet::is_comm</a>(payee), <a href="../../../../../../../DPN/releases/artifacts/current/build/MoveStdlib/docs/Errors.md#0x1_Errors_invalid_argument">Errors::invalid_argument</a>(PAYEE_NOT_COMMUNITY_WALLET));
// };

<b>let</b> payments = &<b>mut</b> <b>borrow_global_mut</b>&lt;<a href="AutoPay.md#0x1_AutoPay_UserAutoPay">UserAutoPay</a>&gt;(addr).payments;
<b>assert</b>!(
Expand Down

0 comments on commit a6c99f1

Please sign in to comment.