/merit

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proof of Growth Community Changes (PoG2) #366

Merged
merged 101 commits into from Aug 15, 2018

Conversation

Reviewers

@tonypizzicato tonypizzicato

Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
3 participants
@mempko @adilwali @tonypizzicato
@mempko
Contributor

mempko commented Jun 13, 2018 •
edited
Edited 22 times
  • @mempko mempko edited Aug 15, 2018 (most recent)
  • @mempko mempko edited Aug 9, 2018
  • @mempko mempko edited Aug 9, 2018
  • @mempko mempko edited Aug 4, 2018
  • @mempko mempko edited Aug 4, 2018
  • @mempko mempko edited Jul 19, 2018
  • @mempko mempko edited Jul 19, 2018
  • @mempko mempko edited Jul 4, 2018
  • @mempko mempko edited Jul 4, 2018
  • @mempko mempko edited Jul 4, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko edited Jul 2, 2018
  • @mempko mempko created Jun 13, 2018

Community Polled changes to PoG and invite distribution.

  • Faster difficulty retargetting.
  • More PoG lottery winners.
  • CGS Computation with Coin Age.
  • Two PoG Distributions. One for older users and one for newer users.
  • Proportional Payout for two Pog Distributions.
  • Three Invite lottery pools. CGS Pool, new user pool, and random pool.
  • PoG minimum stake.
  • Change getaddressrank and getaddressleaderboard to only return CGS based stats instead of ANV.
  • Reg Test.
  • CGS optimizations.
  • Make sure compiles on Mac, Win, and Lin.
  • Start Testnet testing
  • Conclude Testnet testing

mempko added some commits Jun 12, 2018

@mempko
Started work on PoG2 modifications. 
This commit changes difficulty adjustment to 5 hours instead of a day and
also adds some PoG2 specific parameters.
9f82137
@mempko
PoG2 total winners increased. 
Increases the total amount of winners per block to 15.
00d83f0
@mempko
Adding PoG2 files which are just a copy of pog for now. 
Building the groundwork for PoG2 changes.
c491150
@mempko
Merge branch 'feature/address-rank' into feature/pog2
0c6237f
@mempko
Fix compilation.
e8c0e98

@mempko mempko added the WIP label Jun 13, 2018

mempko added some commits Jun 13, 2018

@mempko
Added GCS with coin age.
fc92917
@mempko
Fix warning for not returning anything.
ca9cb53
@mempko
Updated getaddressrank to show how rank changes based on pog2
ae7bb6b
@mempko
Added leaderboard function support.
6b1b9de
@mempko
Change to 4 weeks to about 100% for age.
1f89756
@mempko
Added network size and direct child size.
93c6ba0
@mempko
Added old and new distribution. 
1. renamed anv to cgs.
2. the address selector can now sample from two distributions, one base
on beacon age.

TODO

Need to get beacon age in an efficient way. Plan will be to create a
lazy on demand index, that will store age. If age isn't in the index it
will search for block hash and use that. Because of the way the sampling
works, most addresses should be indexed this way in the very first block
that is activated.
bbbbfa7
@mempko
Work on creating new/old ambassador rewards. 
There are two slots, the old slots and the new slots. The old
slots are reserved for older beacons and they get the majority of
ambassador rewards. The new slots are fewer and provide rewards to new
ambassadors. This balances fairness with new people experiencing the
benefits of ambassador rewards.
d66dee3
@mempko
Reuse pog datastructures in pog2
a1d1f7e
@mempko
Add support for getting beacon height from refdb.
be853af
@mempko
Add beacon height to in memory cache.
ce9bf92
@mempko
Use cache version of refdb.
667e0ec
@mempko
Merge branch 'feature/address-rank' into feature/pog2
ca3af22
@mempko
Merge branch 'feature/address-rank' into feature/pog2
5a8967d
@mempko
Several bug fixes from testing.
10205e2
@mempko
Unifying proportional payout.
6e66e96
@mempko
Invite lottery distribution pools. 
Some work on the invite lottery distribution pools. The strategy is to
do rejections sampling on which pool to choose when rewarding ever
invite. This way each pool has a set probability of getting an invite.

The pools are

1. CGS based pool. 50%
2. New user pool.  30%
3. Any user pool.  20%
15d8886
@mempko
Implemented invite sampling based on CGS.
6039d98
@mempko
Work on finding and giving oldest beacon with zero invites. 
I call them novites, as users who have no invites after a while.
b939efb
@mempko
Novite range storage and computation. 
We must store the last address idx which has no invites called a novite.
For some of the time we will distribute an invite to these addresses but
only once to prevent abuse.
5e34ab8
@mempko
Fix outdated assumptions.
7fed621
@mempko
Added POG debugging statements and fixed some bugs.
63d680f

@mempko mempko changed the title from Feature/pog2 to Proof of Growth Community Changes Jul 2, 2018

Show outdated Hide outdated src/chainparams.cpp
@@ -326,6 +334,14 @@ class CTestNetParams : public CChainParams
consensus.imp_miner_reward_for_every_x_blocks = 10; //invite every 10 minutes at a minumum, 144 per day.
consensus.imp_weights = {60, 40};
// PoG v2 settings.
consensus.pog2_blockheight = 100000;

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

testnet height is already 170k+
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

testnet height is already 170k+
Show outdated Hide outdated src/chainparams.cpp
consensus.pog2_blockheight = 300000;
consensus.pog2_total_winning_ambassadors = 15;
consensus.pog2_ambassador_percent_cut = 50; //50%
consensus.pog2_pow_target_timespan = 5 * 60 * 60; // every 5 hours

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

maybe decrease it to 3 hours?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

maybe decrease it to 3 hours?
Show outdated Hide outdated src/pog2/cgs.cpp
int height,
Entrants& entrants)
{
assert(prefviewcache);

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

why do we use prefviewcache here if we pass db to this function?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

why do we use prefviewcache here if we pass db to this function?
Show outdated Hide outdated src/pog2/cgs.cpp
assert(c.amount >= 0);
double age_scale = AgeScale(height, c);
CAmount amount = std::floor(age_scale * c.amount);

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

is my computation right here:
10 MRT coins would be accounted in aged balance only after 480 blocks?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

is my computation right here:
10 MRT coins would be accounted in aged balance only after 480 blocks?
Show outdated Hide outdated src/pog2/select.cpp
[height,&params](const Entrant& e) {
assert(height >= e.beacon_height);
const double age = height - e.beacon_height;
return age <= params.pog2_new_distribution_age;

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

we're taking only addresses which are younger than 2 days?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

we're taking only addresses which are younger than 2 days?
Show outdated Hide outdated src/validation.cpp
if(pindex && pindex->nHeight > 0) {
auto height = prefviewdb->GetReferralHeight(refOut->GetAddress());
if(height > 0) {

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

it should be height == 0?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

it should be height == 0?

This comment has been minimized.

Show comment
Hide comment
@mempko

mempko Aug 5, 2018

Contributor

no, in this case GetReferralHeight returns -1 if it isn't indexed in the DB. It's lazy indexed on demand. Which is reasonable since the height is immutable once in the chain.
Copy URL
@mempko

mempko Aug 5, 2018

Contributor

no, in this case GetReferralHeight returns -1 if it isn't indexed in the DB. It's lazy indexed on demand. Which is reasonable since the height is immutable once in the chain.

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Aug 6, 2018

Contributor

ok, then it should be if (height == -1)
why would we insert to cache if it's already there?
Copy URL
@tonypizzicato

tonypizzicato Aug 6, 2018

Contributor

ok, then it should be if (height == -1)
why would we insert to cache if it's already there?
Show outdated Hide outdated src/validation.cpp
assert(height >= params.pog2_blockheight);
assert(prefviewdb != nullptr);
static size_t max_embassador_lottery = 0;

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

typo
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

typo
Show outdated Hide outdated src/validation.cpp
pog2::Entrants entrants;
// unlikely that the candidates grew over 50% since last time.
auto reserve_size = max_embassador_lottery * 1.5;

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

isn't max_embassador_lottery is zero here?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

isn't max_embassador_lottery is zero here?
Show outdated Hide outdated src/validation.cpp
referral::NoviteRange novite_range{0,0};
const bool pog2 = pindex->nHeight >= chainparams.GetConsensus().pog2_blockheight;
if (validate || pog2) {

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

we won't sample validation for pog2?
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

we won't sample validation for pog2?
Show outdated Hide outdated src/validation.cpp
assert(height >= 0);
assert(prefviewcache);
static size_t max_embassador_lottery = 0;

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

typo
Copy URL
@tonypizzicato

tonypizzicato Jul 9, 2018

Contributor

typo
Show outdated Hide outdated src/pog2/cgs.cpp
assert(height >= 0);
assert(c.height <= height);
double age = Age(height, c) / TWO_DAYS;

This comment has been minimized.

Show comment
Hide comment
@tonypizzicato

tonypizzicato Jul 10, 2018

Contributor

extract to params
Copy URL
@tonypizzicato

tonypizzicato Jul 10, 2018

Contributor

extract to params
@adilwali

This comment has been minimized.

Show comment
Hide comment
@adilwali

adilwali Aug 9, 2018

Contributor

I have built a small simulator for this, and will be playing with many new custom growth patterns on TestNet.
Contributor

adilwali commented Aug 9, 2018 •
edited
Edited 1 time
  • @adilwali adilwali edited Aug 9, 2018 (most recent)
  • @adilwali adilwali created Aug 9, 2018

I have built a small simulator for this, and will be playing with many new custom growth patterns on TestNet.
@adilwali

This comment has been minimized.

Show comment
Hide comment
@adilwali

adilwali Aug 9, 2018

Contributor

Using the new PoG in concert with getAddressRank and getAddressLeaderboard has empowered us to test many (many many) permutations of the PoG2 algorithms.

Using multiple sets of slots is a huge win, and allows us to reward two different types of positive actors:

  1. Those with significant stake.
  2. Those who have worked to grow the community.

Since we do not allow double-selection, this will end up allowing Growth miners two paths toward earning mining rewards in Merit.
Contributor

adilwali commented Aug 9, 2018

Using the new PoG in concert with getAddressRank and getAddressLeaderboard has empowered us to test many (many many) permutations of the PoG2 algorithms.

Using multiple sets of slots is a huge win, and allows us to reward two different types of positive actors:

  1. Those with significant stake.
  2. Those who have worked to grow the community.

Since we do not allow double-selection, this will end up allowing Growth miners two paths toward earning mining rewards in Merit.
@adilwali

This comment has been minimized.

Show comment
Hide comment
@adilwali

adilwali Aug 9, 2018

Contributor

I will post some visuals as soon as I can. But the distributions are looking decidedly more robust and inclusive. This approach also feels like it is the most future-proof as the network reaches hundreds of thousands and eventually millions of users.
Contributor

adilwali commented Aug 9, 2018

I will post some visuals as soon as I can. But the distributions are looking decidedly more robust and inclusive. This approach also feels like it is the most future-proof as the network reaches hundreds of thousands and eventually millions of users.

mempko added some commits Aug 9, 2018

@mempko
Add unspent coins in an in-memory cache. 
Most of the time spent computing CGS was deserializing unspent coins
from leveldb. Storing them in a deserialized state provides a huge speed
increase. The cost is memory usage.
0fe4ecd
@mempko
Optimizations which speedup the computation by another 2-4x. 
Better memory layout.
Parallel computation of coinage.
Parallel computation of node contribution.
Fewer lookups against the in memory address index.
f39a459
@mempko
Remove unessasary locks.
75bfd36
@mempko
Rearrange cgs thread pool initialization and caching.
a092e69
@mempko
Add batching to reduce overhead in the cgs thread pool processing.
decff71
@mempko
Fix un-indexing transactions during disconnect block. 
The un-indexing should happen in lock-step with the coinv view db
updates.
8215b3a
@mempko
Added more POG logging.
0eec28a
@mempko
Optimize updating the coin cache.
4430637
@mempko
Some cleanup.
59a4b2f
@mempko
Fix lottery heap removal. 
The current indexed heap is wrong.
b3f7972
@mempko
Fix comparison so that heap is deterministic.
ac1430b
@mempko
Adding code to fix heap before starting PoG2
29fc696
@adilwali

This comment has been minimized.

Show comment
Hide comment
@adilwali

adilwali Aug 13, 2018

Contributor

Keeping the pool makes sense for now. But it's also something we could easily bring back in PoG3 if it's needed. Today, it only provides incremental utility.

Over time, of course, an eligibility pool will become a lot more interesting. (Requiring certain community size or a certain trust score in order to be considered could further strengthen the community.)
Contributor

adilwali commented Aug 13, 2018

Keeping the pool makes sense for now. But it's also something we could easily bring back in PoG3 if it's needed. Today, it only provides incremental utility.

Over time, of course, an eligibility pool will become a lot more interesting. (Requiring certain community size or a certain trust score in order to be considered could further strengthen the community.)

mempko added some commits Aug 14, 2018

@mempko
Removing lottery pool and ANV computation. 
The lottery pool and ANV computation is no longer needed.
The set for the distribution is now decided as addresses that meet the
minimum stake.
a15847c
@mempko
Adding backwards compatibility for getaddressleaderboard and rank.
7fa6596
@mempko
Renaming to anv_ranks for backwards compatibility.
ae36568
@mempko
Added filtering unspent utxos via the spent index. 
There was a bug where the unspent index contained spent coins.
This commit adds filtering unspent coins using the spent index.
d7511ed
@mempko
Exclude genesis address from the lottery.
e4c73f4
@mempko
Setting start of PoG2 around Aug 16, 10pm Pacific
b77206e
@mempko
Updating block start time for PoG2 to Around Fri, 17 Aug 2018 09:14:38
620e231

@mempko mempko merged commit da2c144 into master Aug 15, 2018

@mempko mempko deleted the feature/pog2 branch Aug 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment