Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Teams - Phase 1 #861

Open
wants to merge 76 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
63f6cf5
scaffold initialize structs for Delegation
0o-de-lally Nov 19, 2021
a58df9a
scaffold test
0o-de-lally Nov 19, 2021
1f35b3e
scaffold tests
0o-de-lally Nov 19, 2021
1554c46
test scaffold passing
0o-de-lally Nov 19, 2021
1cabb57
implement migration for delegation init. Tests.
0o-de-lally Nov 19, 2021
ecaff73
create the dont_panic tests for cases when the vm not initialized.
0o-de-lally Nov 19, 2021
69dc64d
create tribe test
0o-de-lally Nov 19, 2021
fd4c09a
create tribe transaction script test passing
0o-de-lally Nov 19, 2021
cf25c25
scaffold join tribe.
0o-de-lally Nov 20, 2021
ad23d80
renaming
0o-de-lally Nov 22, 2021
17ccfca
documentation on teams
0o-de-lally Nov 22, 2021
905acd6
documentation and passing tests
0o-de-lally Nov 22, 2021
58d88f2
towers for teams threshold cal
0o-de-lally Nov 22, 2021
f10163b
scaffold split payment to team
0o-de-lally Nov 22, 2021
fec7031
wip subsidy
0o-de-lally Nov 23, 2021
647a243
builds
0o-de-lally Nov 23, 2021
6d3d771
collective tower test passing
0o-de-lally Nov 23, 2021
b869dbb
patch join teams test
0o-de-lally Nov 23, 2021
d7ab9bb
wip
0o-de-lally Nov 23, 2021
1a0c94b
wip getting calculations right
0o-de-lally Nov 25, 2021
cbbd0a7
patch test split payment
0o-de-lally Nov 25, 2021
8eb8da2
Merge branch 'main' into teams
0o-de-lally Nov 25, 2021
268f194
split payment test passing
0o-de-lally Nov 25, 2021
d7e47cd
added test for below threshol proofs in teams
0o-de-lally Nov 25, 2021
13bdb8f
create team state resource view
Dec 1, 2021
24c283c
query for teams scaffolded
Dec 1, 2021
5ec25c9
scaffold txs for setting team
Dec 1, 2021
7dc133b
create join team move script, and txs subcommands
Dec 1, 2021
d7967cf
documentation
Dec 1, 2021
0a6a2a5
Merge branch 'main' into teams
Dec 1, 2021
5881d14
update teams docs
0o-de-lally Dec 10, 2021
94c3a08
Merge branch 'teams' of github.com:0o-de-lally/libra into teams
0o-de-lally Dec 10, 2021
6dace77
Merge branch 'main' into teams
0o-de-lally Dec 10, 2021
828de2f
build stdlib
0o-de-lally Dec 10, 2021
f18f4ae
Merge branch 'main' into teams
0o-de-lally Dec 13, 2021
c28b3a4
scaffold lazy computation of miner team threshold
0o-de-lally Dec 13, 2021
a13ef01
WIP include a sorted tower field. TBD if it will be used
0o-de-lally Dec 13, 2021
2ba08eb
calls lazy update on each tower submission
0o-de-lally Dec 13, 2021
b02bc07
create safety initialization at start of epoch boundary
0o-de-lally Dec 13, 2021
d898efc
decimal native should never panic on unwrap()
0o-de-lally Dec 14, 2021
54cd974
rms and test
0o-de-lally Dec 14, 2021
a37affd
adding functions check thresholds, activate members to teams. And tests
0o-de-lally Dec 14, 2021
e03496a
more tests with tx scripts
0o-de-lally Dec 15, 2021
bec0261
Merge branch 'main' into teams
0o-de-lally Jan 14, 2022
cbf4a3d
Merge branch 'main' into teams
0o-de-lally Feb 4, 2022
cec4487
merge
0o-de-lally Feb 4, 2022
0ef8885
patch activate_member_from_tx_script.move
0o-de-lally Feb 4, 2022
510a79a
patch join team test
0o-de-lally Feb 4, 2022
c9791d7
patch split payment test
0o-de-lally Feb 5, 2022
b95379f
Merge branch 'main' into teams
0o-de-lally Feb 7, 2022
9014aee
patch merge, add docs
0o-de-lally Feb 7, 2022
9582fd9
cleanup Validator universe deprecated apis, teams loopholes.
0o-de-lally Feb 7, 2022
959bb3a
WIP epoch burn tests
0o-de-lally Feb 7, 2022
9afcf1f
cleanup up tests
0o-de-lally Feb 7, 2022
9c8327b
patch onboarding test
0o-de-lally Feb 7, 2022
46ffc7b
patch subsidy test
0o-de-lally Feb 7, 2022
8b69941
patch case 1 reconfig test
0o-de-lally Feb 7, 2022
cbb0d86
patch slow wallet test
0o-de-lally Feb 7, 2022
13dc94a
fix community_wallet test
0o-de-lally Feb 7, 2022
f86c50f
patch onboarding flow test
0o-de-lally Feb 8, 2022
395c3ae
cleanup debug prints
0o-de-lally Feb 8, 2022
845da19
build stdlib
0o-de-lally Feb 8, 2022
2c831aa
test continue-on-error in integration tests
0o-de-lally Feb 8, 2022
598499a
patch integration tests
0o-de-lally Feb 8, 2022
c60c0c2
remove function which should not be public
0o-de-lally Feb 16, 2022
3922dc5
check permission on MigrateInitDelegation
0o-de-lally Feb 16, 2022
8b6d06c
create test for is_member_above_thresh
0o-de-lally Feb 16, 2022
bb20351
update docs
0o-de-lally Feb 16, 2022
d254dd3
rename const in TowerState
0o-de-lally Feb 16, 2022
1298631
Update about_teams.md
soaresa Feb 17, 2022
19a6cf2
add comment
0o-de-lally Feb 17, 2022
51a9ea7
check operator pct on team_init
0o-de-lally Feb 17, 2022
a90213a
check captain percet is not 0 or greater than 100
0o-de-lally Feb 17, 2022
67660c2
team switching function
0o-de-lally Feb 17, 2022
7dd61b6
switch team test
0o-de-lally Feb 17, 2022
5688b6a
patch implementation of activation to team
0o-de-lally Feb 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions json-rpc/src/data.rs
Original file line number Diff line number Diff line change
Expand Up @@ -301,8 +301,6 @@ pub fn get_miner_state(
Some(s) => TowerStateResourceView::try_from(s).map_err(Into::into),
None => Err(JsonRpcError::internal_error("No account state found".to_owned())),
}


}

/// Get miner state
Expand Down
1 change: 0 additions & 1 deletion json-rpc/types/src/views.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1493,7 +1493,6 @@ impl TryFrom<AccountState> for OracleUpgradeStateView {
} else {
Err(Error::msg("could not get upgrade oracle data"))
}

}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ module EpochBoundary { // TODO: Rename to Boundary
DiemAccount::slow_wallet_epoch_drip(vm, Globals::get_unlock());
// update_validator_withdrawal_limit(vm);
};

do_migrations(vm);

reset_counters(vm, proposed_set, height_now)
}

Expand Down Expand Up @@ -177,5 +180,12 @@ module EpochBoundary { // TODO: Rename to Boundary
AutoPay::reconfig_reset_tick(vm);
Epoch::reset_timer(vm, height_now);
}

use 0x1::MigrateInitDelegation;
fun do_migrations(vm: &signer) {
// these need to run on an upgrade where a new data structure is introduced.
MigrateInitDelegation::do_it(vm);
}

}
}
33 changes: 0 additions & 33 deletions language/diem-framework/modules/0L/Migrations.move
Original file line number Diff line number Diff line change
Expand Up @@ -86,37 +86,4 @@ module Migrations {
Option::none<Job>()
}
}

/// # Summary
/// Module providing method to convert all wallets to "slow wallets"
/// migrations should have own module, since imports can cause dependency cycling.
module MigrateWallets {
use 0x1::Vector;
use 0x1::Migrations;
use 0x1::DiemAccount;
use 0x1::ValidatorUniverse;
use 0x1::CoreAddresses;

const UID: u64 = 10;

// Migration to migrate all wallets to be slow wallets
public fun migrate_slow_wallets(vm: &signer) {
CoreAddresses::assert_diem_root(vm);
if (!Migrations::has_run(UID)) {
let vec_addr = ValidatorUniverse::get_eligible_validators(vm);
// TODO: how to get other accounts?

// tag all accounts as slow wallets
let len = Vector::length<address>(&vec_addr);
let i = 0;
while (i < len) {
let addr = *Vector::borrow<address>(&vec_addr, i);
DiemAccount::vm_migrate_slow_wallet(vm, addr);
i = i + 1;
};
Migrations::push(vm, UID, b"MigrateWallets");
};
}

}
}
97 changes: 87 additions & 10 deletions language/diem-framework/modules/0L/Subsidy.move
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ address 0x1 {
use 0x1::ValidatorConfig;
use 0x1::TowerState;
use 0x1::FixedPoint32;
use 0x1::Teams;

// estimated gas unit cost for proof verification divided coin scaling factor
// Cost for verification test/easy difficulty: 1173 / 1000000
Expand Down Expand Up @@ -52,22 +53,98 @@ address 0x1 {

let i = 0;
while (i < len) {
let node_address = *(Vector::borrow<address>(outgoing_set, i));
let node_address = Vector::borrow<address>(outgoing_set, i);
// Transfer gas from vm address to validator
let minted_coins = Diem::mint<GAS>(vm, subsidy_granted);
DiemAccount::vm_deposit_with_metadata<GAS>(
vm,
node_address,
minted_coins,
b"validator subsidy",
b""
);
check_team_and_pay(vm, node_address, subsidy_granted);

// refund operator tx fees for mining
refund_operator_tx_fees(vm, node_address);
refund_operator_tx_fees(vm, *node_address);
i = i + 1;
};
}
use 0x1::Debug::print;

fun check_team_and_pay(vm: &signer, captain_address: &address, subsidy_granted: u64) {
// this is a solo validator. Exists during transition to delegation mode. This is a fallback condition to keep the node from halting
print(&100000);
let captain_value = subsidy_granted;
print(&subsidy_granted);
if (Teams::team_is_init(*captain_address)) {
print(&100100);
// split captain reward and send to captain.
let captain_pct = Teams::get_operator_reward(*captain_address);
jamesmeijers marked this conversation as resolved.
Show resolved Hide resolved
print(&captain_pct);
// split off the captain value
captain_value = FixedPoint32::multiply_u64(
subsidy_granted,
FixedPoint32::create_from_rational(captain_pct, 100)
);
print(&100200);
print(&captain_value);

let value_to_members = subsidy_granted - captain_value;
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
print(&value_to_members);

// get team members
let members = Teams::get_team_members(*captain_address);
// split the team subsidy
print(&100300);
split_subsidy_to_team(vm, &members, value_to_members);
};
print(&100400);
print(&captain_value);
let captain_coins = Diem::mint<GAS>(vm, captain_value);

let pre_balance = DiemAccount::balance<GAS>(*captain_address);
print(&pre_balance);

// payment to captain
DiemAccount::vm_deposit_with_metadata<GAS>(
vm,
*captain_address,
captain_coins,
b"validator subsidy",
b""
);
}

public fun split_subsidy_to_team(vm: &signer, members: &vector<address>, value_to_members: u64) {
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
let collective_height = TowerState::collective_tower_height(members);
print(&100310);
print(members);
print(&collective_height);
print(&value_to_members);
let i = 0;
while (i < Vector::length(members)) {
let addr = Vector::borrow(members, i);
let one_height = TowerState::tower_for_teams(*addr);
print(&100320);
print(&one_height);
if (one_height > 0) {
let payment_to_this_member = FixedPoint32::multiply_u64(
0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
value_to_members,
FixedPoint32::create_from_rational(one_height, collective_height)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue I see with this is that it is essentially reducing the problem to proof of work. I think we need a system where one tall tower is preferred over many short towers.

Alternatively, we could keep the rate limit on validators (but potentially increase the limit) to prevent Sybil attacks such as this, but I think the rate limit would need to decrease with time. As it stands, the number of validators can double every 2 weeks, which poses a problem for Sybil attacks now, but won't be much of an obstacle once there are many validators.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a design idea here.

The min tower height is determined by the number of towers who are a members in a team. Basically as you add members to a team the min height increases. I think this would mitigate sybiks

Copy link
Collaborator Author

@0o-de-lally 0o-de-lally Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zmanian is this the min height of a member to join a team? Is this like musical chairs in a sense, where people get knocked off a team the as the height to join a team gets progressively higher?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mental model is that increasing height requirement on the team can prevent the n+1 team member from join but you don't get kicked out of team if the difficulty becomes higher than your height

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So then even if they make a new team, the rate of growth for that team would be limited…This idea sounds quite promising

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the sum of squares approach makes sense (or at least sum of some super-linear function, squares is simple, but maybe not optimal, depending on what exactly we want to achieve). Just to confirm, the rate limit would limit the number of teams that can be created, not the number of people that can join teams (which would be unbounded), is that correct?

Copy link
Collaborator Author

@0o-de-lally 0o-de-lally Dec 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rate-limit I refer to is the creation of new validator accounts, as we have today. So effectively new teams. The increase would be from 2 weeks to 4 or 6 weeks. Limits speed at which validator could gain edge (and give ample time to catch attacks coming).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looping back on this discussion. The final implementation uses a threshold of the RMS of the towers. Which achieves the goal of sum of squares per @jamesmeijers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the RMS of towers threshold specifically for being able to join a team?

So phase 1 keeps the validator tower height as the metric for validator set inclusion and consensus. Then, in phase 2 of the rollout, will the collective (not RMS) towers of the validator/team be the metric in order to incentivize team formation?

In phase 2, will the collective delay tower difficulty for teams target 100/max validators to limit subsidy minting?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@misko9 This is covered in the ol/documentaiton/teams. In short yes the Phase 1 does not change qualification or consensus weights. Phase 2 Does. And there are things to work out about the sybil resistance of phase 2, including how the collective difficulty changes.

);
print(&100330);
print(&payment_to_this_member);

// let payment_to_this_member = value_to_members * pct;
// print(&payment_to_this_member);
let minted_coins = Diem::mint<GAS>(vm, payment_to_this_member);
let pre_balance = DiemAccount::balance<GAS>(*addr);
print(&pre_balance);
DiemAccount::vm_deposit_with_metadata<GAS>(
vm,
*addr,
minted_coins,
b"team consensus payment",
b""
);
};
i = i + 1;
}
}


// Function code: 02 Prefix: 190102
public fun calculate_subsidy(vm: &signer, network_density: u64): (u64, u64) {
Expand Down
162 changes: 162 additions & 0 deletions language/diem-framework/modules/0L/Teams.move
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
///////////////////////////////////////////////////////////////////////////
// 0L Module
// Teams
///////////////////////////////////////////////////////////////////////////
// Used for coordinating "Teams", equivalent to delegation
// Teams are groups which engage in work.
// some of that work is automated such as validation.
// A Team has a collective "consensus weight", which is used for voting in consensus
Copy link
Contributor

@sm86 sm86 Feb 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

know it might be late but just a thought: should we use consensus weight only for being in top 100 and consider all teams to be equal consensus weight. theoretically there is a probability of malicious node having higher consensus weight

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting. Something for @zmanian

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sm86 Can you ticket this as an idea to research in Issues, for Phase 2?

// but also for stdlib upgrades.
// Team members can share in a validator's rewards, but can also be used for coordinating
// other activity, like projects and bounties.
// File Prefix for errors: TBD
///////////////////////////////////////////////////////////////////////////

address 0x1 {
module Teams {
use 0x1::CoreAddresses;
use 0x1::Vector;
use 0x1::Signer;
use 0x1::DiemAccount;
use 0x1::ValidatorUniverse;

const ENOT_SLOW_WALLET: u64 = 1010;

struct AllTeams has key, copy, drop, store {
teams_list: vector<address>, // the team is identified by its captain.

}

struct Team has key, copy, drop, store {
captain: address, // A validator account. TODO this is redundant, since it is stored in the validator account. But placed here for future-proofing.
team_name: vector<u8>, // A validator account.
members: vector<address>,
operator_pct_reward: u64, // the percentage of the rewards that the captain proposes to go to the validator operator.
collective_tower_height_this_epoch: u64,
}

// this struct is stored in the member's account
struct Member has key, copy, drop, store {
captain_address: address, // by address of captain
mining_above_threshold: bool, // if the mining the user has done is above the system threshold to count toward delegation.

}

public fun vm_init(sender: &signer) {
CoreAddresses::assert_vm(sender);
move_to<AllTeams>(
sender,
AllTeams {
teams_list: Vector::empty()
}
);
}


public fun team_init(sender: &signer, team_name: vector<u8>, operator_pct_reward: u64) {

0o-de-lally marked this conversation as resolved.
Show resolved Hide resolved
assert(ValidatorUniverse::is_in_universe(Signer::address_of(sender)), 201301001);
// An "captain", who is already a validator account, stores the Team struct on their account.
// the AllTeams struct is saved in the 0x0 account, and needs to be initialized before this is called.

// check vm has initialized the struct, otherwise exit early.
if (!exists<AllTeams>(CoreAddresses::VM_RESERVED_ADDRESS())) {
return
};

move_to<Team>(
sender,
Team {
captain: Signer::address_of(sender), // A validator account.
team_name, // A validator account.
members: Vector::empty<address>(),
operator_pct_reward, // the percentage of the rewards that the captain proposes to go to the validator operator.
collective_tower_height_this_epoch: 0,
}
);
}

public fun join_team(sender: &signer, captain_address: address) acquires Member, Team {
let addr = Signer::address_of(sender);

// needs to check if this is a slow wallet.
// ask user to resubmit if not a slow wallet, so they are explicitly setting it, no surprises, no tears.

assert(DiemAccount::is_slow(addr), ENOT_SLOW_WALLET);


// bob wants to switch to a different Team.
if (exists<Member>(addr)) {
let member_state = borrow_global_mut<Member>(addr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to check if the account is in the member list of the old team and if so, remove them

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, working on it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed with 7dd61b6

Added switch_team private function.
Includes the switch_team.move test in functional test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think switch_team would allow a user to be added a team without having first met the mining threshold that is enforced is maybe_activate_member _to_team. Perhaps maybe_switch_team should call maybe_activate_member_to_team in order to prevent this?

Copy link
Collaborator Author

@0o-de-lally 0o-de-lally Feb 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamesmeijers please check, I refactored as you suggested. See: 5688b6a

// update the membership list of the former captain
let former_captain_state = borrow_global_mut<Team>(member_state.captain_address);
let (is_found, idx) = Vector::index_of(&former_captain_state.members, &addr);
if (is_found) {
Vector::remove(&mut former_captain_state.members, idx);
member_state.captain_address = captain_address;
};
// TODO: Do we need to reset mining_above_threshold if they are switching?
} else { // first time joining a Team.
move_to<Member>(sender, Member {
captain_address,
mining_above_threshold: false,
});
};
let captain_state = borrow_global_mut<Team>(captain_address);
Vector::push_back<address>(&mut captain_state.members, addr);
}


//////// GETTERS ////////

public fun get_all_teams(): vector<address> acquires AllTeams {
if (exists<AllTeams>(CoreAddresses::VM_RESERVED_ADDRESS())) {
let list = borrow_global<AllTeams>(CoreAddresses::VM_RESERVED_ADDRESS());
return *&list.teams_list
} else {
Vector::empty<address>()
}
}

public fun team_is_init(captain: address): bool {
exists<Team>(captain)
}

// NOTE: Important! The EpochBoundary will call this. This function cannot abort, must not halt consensus.
public fun get_operator_reward(captain: address):u64 acquires Team {
if (team_is_init(captain)) {
let s = borrow_global_mut<Team>(captain);
return *&s.operator_pct_reward
};
0
}
// find the team members
public fun get_team_members(captain: address):vector<address> acquires Team {
if (team_is_init(captain)) {
let s = borrow_global_mut<Team>(captain);
return *&s.members
};
Vector::empty<address>()
}

public fun vm_is_init(): bool {
exists<AllTeams>(CoreAddresses::VM_RESERVED_ADDRESS())
}
}

// Module for initializing Teams on a hot upgrade of stdlib.
// since the system is likely operating and Teams are introduced as an upgrade, the structs need to be initalized.

module MigrateInitDelegation {
use 0x1::Teams;
use 0x1::Migrations;
const UID: u64 = 101;
public fun do_it(vm: &signer) {
if (!Migrations::has_run(UID)) {
Teams::vm_init(vm);
Migrations::push(vm, UID, b"MigrateInitTeams");
}
}
}
}

Loading