Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Teams - Phase 1 #861

Open
wants to merge 76 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
63f6cf5
scaffold initialize structs for Delegation
0o-de-lally Nov 19, 2021
a58df9a
scaffold test
0o-de-lally Nov 19, 2021
1f35b3e
scaffold tests
0o-de-lally Nov 19, 2021
1554c46
test scaffold passing
0o-de-lally Nov 19, 2021
1cabb57
implement migration for delegation init. Tests.
0o-de-lally Nov 19, 2021
ecaff73
create the dont_panic tests for cases when the vm not initialized.
0o-de-lally Nov 19, 2021
69dc64d
create tribe test
0o-de-lally Nov 19, 2021
fd4c09a
create tribe transaction script test passing
0o-de-lally Nov 19, 2021
cf25c25
scaffold join tribe.
0o-de-lally Nov 20, 2021
ad23d80
renaming
0o-de-lally Nov 22, 2021
17ccfca
documentation on teams
0o-de-lally Nov 22, 2021
905acd6
documentation and passing tests
0o-de-lally Nov 22, 2021
58d88f2
towers for teams threshold cal
0o-de-lally Nov 22, 2021
f10163b
scaffold split payment to team
0o-de-lally Nov 22, 2021
fec7031
wip subsidy
0o-de-lally Nov 23, 2021
647a243
builds
0o-de-lally Nov 23, 2021
6d3d771
collective tower test passing
0o-de-lally Nov 23, 2021
b869dbb
patch join teams test
0o-de-lally Nov 23, 2021
d7ab9bb
wip
0o-de-lally Nov 23, 2021
1a0c94b
wip getting calculations right
0o-de-lally Nov 25, 2021
cbbd0a7
patch test split payment
0o-de-lally Nov 25, 2021
8eb8da2
Merge branch 'main' into teams
0o-de-lally Nov 25, 2021
268f194
split payment test passing
0o-de-lally Nov 25, 2021
d7e47cd
added test for below threshol proofs in teams
0o-de-lally Nov 25, 2021
13bdb8f
create team state resource view
Dec 1, 2021
24c283c
query for teams scaffolded
Dec 1, 2021
5ec25c9
scaffold txs for setting team
Dec 1, 2021
7dc133b
create join team move script, and txs subcommands
Dec 1, 2021
d7967cf
documentation
Dec 1, 2021
0a6a2a5
Merge branch 'main' into teams
Dec 1, 2021
5881d14
update teams docs
0o-de-lally Dec 10, 2021
94c3a08
Merge branch 'teams' of github.com:0o-de-lally/libra into teams
0o-de-lally Dec 10, 2021
6dace77
Merge branch 'main' into teams
0o-de-lally Dec 10, 2021
828de2f
build stdlib
0o-de-lally Dec 10, 2021
f18f4ae
Merge branch 'main' into teams
0o-de-lally Dec 13, 2021
c28b3a4
scaffold lazy computation of miner team threshold
0o-de-lally Dec 13, 2021
a13ef01
WIP include a sorted tower field. TBD if it will be used
0o-de-lally Dec 13, 2021
2ba08eb
calls lazy update on each tower submission
0o-de-lally Dec 13, 2021
b02bc07
create safety initialization at start of epoch boundary
0o-de-lally Dec 13, 2021
d898efc
decimal native should never panic on unwrap()
0o-de-lally Dec 14, 2021
54cd974
rms and test
0o-de-lally Dec 14, 2021
a37affd
adding functions check thresholds, activate members to teams. And tests
0o-de-lally Dec 14, 2021
e03496a
more tests with tx scripts
0o-de-lally Dec 15, 2021
bec0261
Merge branch 'main' into teams
0o-de-lally Jan 14, 2022
cbf4a3d
Merge branch 'main' into teams
0o-de-lally Feb 4, 2022
cec4487
merge
0o-de-lally Feb 4, 2022
0ef8885
patch activate_member_from_tx_script.move
0o-de-lally Feb 4, 2022
510a79a
patch join team test
0o-de-lally Feb 4, 2022
c9791d7
patch split payment test
0o-de-lally Feb 5, 2022
b95379f
Merge branch 'main' into teams
0o-de-lally Feb 7, 2022
9014aee
patch merge, add docs
0o-de-lally Feb 7, 2022
9582fd9
cleanup Validator universe deprecated apis, teams loopholes.
0o-de-lally Feb 7, 2022
959bb3a
WIP epoch burn tests
0o-de-lally Feb 7, 2022
9afcf1f
cleanup up tests
0o-de-lally Feb 7, 2022
9c8327b
patch onboarding test
0o-de-lally Feb 7, 2022
46ffc7b
patch subsidy test
0o-de-lally Feb 7, 2022
8b69941
patch case 1 reconfig test
0o-de-lally Feb 7, 2022
cbb0d86
patch slow wallet test
0o-de-lally Feb 7, 2022
13dc94a
fix community_wallet test
0o-de-lally Feb 7, 2022
f86c50f
patch onboarding flow test
0o-de-lally Feb 8, 2022
395c3ae
cleanup debug prints
0o-de-lally Feb 8, 2022
845da19
build stdlib
0o-de-lally Feb 8, 2022
2c831aa
test continue-on-error in integration tests
0o-de-lally Feb 8, 2022
598499a
patch integration tests
0o-de-lally Feb 8, 2022
c60c0c2
remove function which should not be public
0o-de-lally Feb 16, 2022
3922dc5
check permission on MigrateInitDelegation
0o-de-lally Feb 16, 2022
8b6d06c
create test for is_member_above_thresh
0o-de-lally Feb 16, 2022
bb20351
update docs
0o-de-lally Feb 16, 2022
d254dd3
rename const in TowerState
0o-de-lally Feb 16, 2022
1298631
Update about_teams.md
soaresa Feb 17, 2022
19a6cf2
add comment
0o-de-lally Feb 17, 2022
51a9ea7
check operator pct on team_init
0o-de-lally Feb 17, 2022
a90213a
check captain percet is not 0 or greater than 100
0o-de-lally Feb 17, 2022
67660c2
team switching function
0o-de-lally Feb 17, 2022
7dd61b6
switch team test
0o-de-lally Feb 17, 2022
5688b6a
patch implementation of activation to team
0o-de-lally Feb 21, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 26 additions & 6 deletions .github/workflows/integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,39 @@ jobs:
run: echo "${{ steps.branch-name.outputs.current_branch }}"
- name: Build sdlib
run: cargo r -p diem-framework --release

- name: MINING
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-mining.mk test
- name: AUTOPAY percent-change
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-autopay.mk test-percent-bal
- name: AUTOPAY fixed-once
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-autopay.mk test-fixed-once
- name: AUTOPAY all
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-autopay.mk test
continue-on-error: true
timeout-minutes: 20

- name: ONBOARD
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-onboard.mk test
continue-on-error: true
timeout-minutes: 10

- name: File permissions
run: sudo chmod -R 777 language/*

- name: UPGRADE
run: BRANCH_NAME=${{ steps.branch-name.outputs.current_branch }} RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-upgrade.mk test
continue-on-error: true
timeout-minutes: 35

- name: AUTOPAY percent-change
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-autopay.mk test-percent-bal
continue-on-error: true
timeout-minutes: 10

- name: AUTOPAY fixed-once
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-autopay.mk test-fixed-once
continue-on-error: true
timeout-minutes: 10

- name: AUTOPAY all
run: RUSTC_WRAPPER=sccache SOURCE_PATH=./ make -f ol/integration-tests/test-autopay.mk test
continue-on-error: true
timeout-minutes: 10



2 changes: 0 additions & 2 deletions json-rpc/src/data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -302,8 +302,6 @@ pub fn get_miner_state(
Some(s) => TowerStateResourceView::from_state_and_epoch(s, epoch).map_err(Into::into),
None => Err(JsonRpcError::internal_error("No account state found".to_owned())),
}


}

/// Get miner state
Expand Down
1 change: 0 additions & 1 deletion json-rpc/types/src/views.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1502,7 +1502,6 @@ impl TryFrom<AccountState> for OracleUpgradeStateView {
} else {
Err(Error::msg("could not get upgrade oracle data"))
}

}
}

Expand Down
53 changes: 39 additions & 14 deletions language/diem-framework/modules/0L/EpochBoundary.move
Original file line number Diff line number Diff line change
Expand Up @@ -24,26 +24,33 @@ module EpochBoundary {
use 0x1::DiemAccount;
use 0x1::Burn;
use 0x1::FullnodeSubsidy;
use 0x1::Teams;
use 0x1::ValidatorUniverse;

// This function is called by block-prologue once after n blocks.
// Function code: 01. Prefix: 180001
public fun reconfigure(vm: &signer, height_now: u64) {
CoreAddresses::assert_vm(vm);

// for safety maybe ititialize any structs that may not have been created in migration.
safety_init(vm);

let height_start = Epoch::get_timer_height_start(vm);

let (outgoing_compliant_set, _) =
DiemSystem::get_fee_ratio(vm, height_start, height_now);

// NOTE: This is "nominal" because it doesn't check
// NOTE: This is the "nominal" award to a miner. We haven't yet checked if the miner is above threshold
let compliant_nodes_count = Vector::length(&outgoing_compliant_set);
let (subsidy_units, nominal_subsidy_per) =
Subsidy::calculate_subsidy(vm, compliant_nodes_count);

process_fullnodes(vm, nominal_subsidy_per);

process_validators(vm, subsidy_units, *&outgoing_compliant_set);


process_burn(vm);

let proposed_set = propose_new_set(vm, height_start, height_now);

// Update all slow wallet limits
Expand All @@ -54,6 +61,11 @@ module EpochBoundary {
reset_counters(vm, proposed_set, outgoing_compliant_set, height_now)
}

fun safety_init(vm: &signer) {
Teams::vm_init(vm);

}

// process fullnode subsidy
fun process_fullnodes(vm: &signer, nominal_subsidy_per_node: u64) {
// Fullnode subsidy
Expand Down Expand Up @@ -101,6 +113,29 @@ module EpochBoundary {
Subsidy::process_fees(vm, &outgoing_compliant_set);
}

fun process_burn(vm: &signer) {
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
Burn::reset_ratios(vm);

// LEAVE THIS CODE COMMENTED for future use
// TODO: Make the burn value dynamic.
// let incoming_count = Vector::length<address>(&top_accounts) - Vector::length<address>(&jailed_set);
// let burn_value = Subsidy::subsidy_curve(
// Globals::get_subsidy_ceiling_gas(),
// incoming_count,
// Globals::get_max_node_density()
// )/4;

let burn_value = 1000000; // TODO: switch to a variable cost, as above.

let vals = ValidatorUniverse::get_eligible_validators(vm);
let i = 0;
while (i < Vector::length<address>(&vals)) {
let addr = *Vector::borrow(&vals, i);
Burn::epoch_start_burn(vm, addr, burn_value);
i = i + 1;
};
}

fun propose_new_set(vm: &signer, height_start: u64, height_now: u64): vector<address> {
// Propose upcoming validator set:
// Step 1: Sort Top N eligible validators
Expand All @@ -117,17 +152,6 @@ module EpochBoundary {

let jailed_set = DiemSystem::get_jailed_set(vm, height_start, height_now);

Burn::reset_ratios(vm);
// LEAVE THIS CODE COMMENTED for future use
// TODO: Make the burn value dynamic.
// let incoming_count = Vector::length<address>(&top_accounts) - Vector::length<address>(&jailed_set);
// let burn_value = Subsidy::subsidy_curve(
// Globals::get_subsidy_ceiling_gas(),
// incoming_count,
// Globals::get_max_node_density()
// )/4;

let burn_value = 1000000; // TODO: switch to a variable cost, as above.

let i = 0;
while (i < Vector::length<address>(&top_accounts)) {
Expand All @@ -140,7 +164,7 @@ module EpochBoundary {
Audit::val_audit_passing(addr)
) {
Vector::push_back(&mut proposed_set, addr);
Burn::epoch_start_burn(vm, addr, burn_value);
// Burn::epoch_start_burn(vm, addr, burn_value);
};
i = i+ 1;
};
Expand Down Expand Up @@ -174,5 +198,6 @@ module EpochBoundary {
AutoPay::reconfig_reset_tick(vm);
Epoch::reset_timer(vm, height_now);
}

}
}
18 changes: 18 additions & 0 deletions language/diem-framework/modules/0L/Migrations.move
Original file line number Diff line number Diff line change
Expand Up @@ -106,4 +106,22 @@ module MigrateTowerCounter {
}

}

// Module for initializing Teams on a hot upgrade of stdlib.
// since the system is likely operating and Teams are introduced as an upgrade, the structs need to be initalized.

module MigrateInitDelegation {
use 0x1::Teams;
// use 0x1::TowerState;
use 0x1::Migrations;
const UID: u64 = 2;
public fun do_it(vm: &signer) {
if (!Migrations::has_run(UID)) {
Teams::vm_init(vm);
// also initialize relevant state in TowerState
// TowerState::init_team_thresholds(vm);
Migrations::push(vm, UID, b"MigrateInitTeams");
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
}
72 changes: 60 additions & 12 deletions language/diem-framework/modules/0L/Subsidy.move
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ address 0x1 {
use 0x1::ValidatorConfig;
use 0x1::TowerState;
use 0x1::FixedPoint32;
use 0x1::Teams;

// estimated gas unit cost for proof verification divided coin scaling factor
// Cost for verification test/easy difficulty: 1173 / 1000000
Expand All @@ -49,26 +50,73 @@ address 0x1 {
if (subsidy_units > len && subsidy_units > 0 ) { // arithmetic safety check
subsidy_granted = subsidy_units/len;
} else { return };

let i = 0;
while (i < len) {
let node_address = *(Vector::borrow<address>(outgoing_set, i));
let node_address = Vector::borrow<address>(outgoing_set, i);
// Transfer gas from vm address to validator
let minted_coins = Diem::mint<GAS>(vm, subsidy_granted);
DiemAccount::vm_deposit_with_metadata<GAS>(
vm,
node_address,
minted_coins,
b"validator subsidy",
b""
);

check_team_and_pay(vm, node_address, subsidy_granted);
// refund operator tx fees for mining
refund_operator_tx_fees(vm, node_address);
refund_operator_tx_fees(vm, *node_address);
i = i + 1;
};
}

fun check_team_and_pay(vm: &signer, captain_address: &address, subsidy_granted: u64) {
// this is a solo validator. Exists during transition to delegation mode. This is a fallback condition to keep the node from halting
let captain_value = subsidy_granted;
if (Teams::team_is_init(*captain_address)) {
// split captain reward and send to captain.
let captain_pct = Teams::get_operator_reward(*captain_address);
jamesmeijers marked this conversation as resolved.
Show resolved Hide resolved
// split off the captain value
captain_value = FixedPoint32::multiply_u64(
subsidy_granted,
FixedPoint32::create_from_rational(captain_pct, 100)
);
let value_to_members = subsidy_granted - captain_value;
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
// get team members
let members = Teams::get_team_members(*captain_address);
// split the team subsidy
split_subsidy_to_team(vm, &members, value_to_members);
};
let captain_coins = Diem::mint<GAS>(vm, captain_value);

// payment to captain
DiemAccount::vm_deposit_with_metadata<GAS>(
vm,
*captain_address,
captain_coins,
b"validator subsidy",
b""
);
}

public fun split_subsidy_to_team(vm: &signer, members: &vector<address>, value_to_members: u64) {
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
let collective_height = TowerState::collective_tower_height(members);
let i = 0;
while (i < Vector::length(members)) {
let addr = Vector::borrow(members, i);
let one_height = TowerState::tower_for_teams(*addr);
if (one_height > 0) {
let payment_to_this_member = FixedPoint32::multiply_u64(
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
value_to_members,
FixedPoint32::create_from_rational(one_height, collective_height)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue I see with this is that it is essentially reducing the problem to proof of work. I think we need a system where one tall tower is preferred over many short towers.

Alternatively, we could keep the rate limit on validators (but potentially increase the limit) to prevent Sybil attacks such as this, but I think the rate limit would need to decrease with time. As it stands, the number of validators can double every 2 weeks, which poses a problem for Sybil attacks now, but won't be much of an obstacle once there are many validators.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a design idea here.

The min tower height is determined by the number of towers who are a members in a team. Basically as you add members to a team the min height increases. I think this would mitigate sybiks

Copy link
Collaborator Author

@0o-de-lally 0o-de-lally Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zmanian is this the min height of a member to join a team? Is this like musical chairs in a sense, where people get knocked off a team the as the height to join a team gets progressively higher?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mental model is that increasing height requirement on the team can prevent the n+1 team member from join but you don't get kicked out of team if the difficulty becomes higher than your height

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So then even if they make a new team, the rate of growth for that team would be limited…This idea sounds quite promising

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the sum of squares approach makes sense (or at least sum of some super-linear function, squares is simple, but maybe not optimal, depending on what exactly we want to achieve). Just to confirm, the rate limit would limit the number of teams that can be created, not the number of people that can join teams (which would be unbounded), is that correct?

Copy link
Collaborator Author

@0o-de-lally 0o-de-lally Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rate-limit I refer to is the creation of new validator accounts, as we have today. So effectively new teams. The increase would be from 2 weeks to 4 or 6 weeks. Limits speed at which validator could gain edge (and give ample time to catch attacks coming).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looping back on this discussion. The final implementation uses a threshold of the RMS of the towers. Which achieves the goal of sum of squares per @jamesmeijers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the RMS of towers threshold specifically for being able to join a team?

So phase 1 keeps the validator tower height as the metric for validator set inclusion and consensus. Then, in phase 2 of the rollout, will the collective (not RMS) towers of the validator/team be the metric in order to incentivize team formation?

In phase 2, will the collective delay tower difficulty for teams target 100/max validators to limit subsidy minting?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@misko9 This is covered in the ol/documentaiton/teams. In short yes the Phase 1 does not change qualification or consensus weights. Phase 2 Does. And there are things to work out about the sybil resistance of phase 2, including how the collective difficulty changes.

);
// let payment_to_this_member = value_to_members * pct;
let minted_coins = Diem::mint<GAS>(vm, payment_to_this_member);
DiemAccount::vm_deposit_with_metadata<GAS>(
vm,
*addr,
minted_coins,
b"team consensus payment",
b""
);
};
i = i + 1;
}
}


// Function code: 02 Prefix: 190102
public fun calculate_subsidy(vm: &signer, network_density: u64): (u64, u64) {
CoreAddresses::assert_vm(vm);
Expand Down
Loading