Lazy reaping #4895

gavofyork · 2020-02-11T19:40:29Z

This mostly removes the functionality of OnReapAccount and introduces reference-counting on accounts to ensure that data structures referring to an account whose economic defence is the account's ED must be cleaned up before the ED is liquidated.

It's a followup/alternative to #4820

frame/system/src/lib.rs

shawntabrizi · 2020-02-18T11:14:03Z

frame/system/src/lib.rs

 		let existed = Account::<T>::contains_key(k);
-		Account::<T>::insert(k, (T::Index::default(), t));
+		Account::<T>::insert(k, AccountInfo{ nonce: Default::default(), refcount: 0, data });


Why is refcount/nonce returning to zero here?

The goal is just to insert some account data onto the user?

this looks indeed bad. good catch 👍

Suggested change

Account::<T>::insert(k, AccountInfo{ nonce: Default::default(), refcount: 0, data });

Account::<T>::mutate(k, |a| a.data = data);

This is what you want, right?

I am going to commit this suggestion.

frame/staking/src/lib.rs

frame/recovery/src/lib.rs

frame/balances/src/lib.rs

frame/contracts/src/lib.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

frame/democracy/src/lib.rs

NikVolf · 2020-02-21T16:11:48Z

/bench

Some final spam :)

parity-benchapp · 2020-02-21T16:11:50Z

Finished benchmark for branch: gav-lazy-reaping

MASTER RESULT

import block/Wasm time:
[57.708 ms 57.826 ms 57.954 ms]

import block/Native time:
[21.069 ms 21.155 ms 21.241 ms]

BRANCH RESULT

import block/Wasm time:
[59.005 ms 59.070 ms 59.135 ms]
change:
[+1.7205% +2.0417% +2.3595%] (p = 0.00 < 0.05)
Performance has regressed.

import block/Native time:
[21.010 ms 21.113 ms 21.214 ms]
change:
[-0.8625% +0.0581% +1.0399%] (p = 0.90 > 0.05)
No change in performance detected.

gavofyork · 2020-02-22T15:52:23Z

concerning that the benchmark is consistently reporting that this branch is slower.

what is it actually benchmarking? this should be waaay faster (and i believe @shawn already showed that it was).

NikVolf · 2020-02-22T19:25:28Z

concerning that the benchmark is consistently reporting that this branch is slower.

what is it actually benchmarking? this should be waaay faster (and i believe @shawn already showed that it was).

Well, it is just 2-3% (it is inside benchmark variance, so don't pay much attention, until it shows > 5-6% difference)

This benchmark is just doing import of block with 100 random transfers, and I believe code path of this pr is not touched, I mostly tested the bot on pr more or less related to performance.

I will be happy to add benchmark that should show improvements on this, btw.

…to gav-lazy-reaping

* Squash and rebase from gav-lazy-reaping * Bump version * Bump runtime again * Docs. * Remove old functions * Update frame/balances/src/lib.rs Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/contracts/src/lib.rs Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com> * Warnings * Bump runtime version * Update frame/democracy/src/lib.rs Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com> * Update frame/system/src/lib.rs * Clean up OnReapAccount * Use frame_support debug * Bump spec * Renames and fix * Fix * Fix rename * Fix * Increase time for test Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>

gavofyork requested review from andresilva, Demi-Marie, kianenigma, mxinden, pepyakin and thiolliere as code owners February 11, 2020 19:40

gavofyork added the A0-please_review Pull request needs code review. label Feb 11, 2020

paritytech-ci added the B2-breaksapi label Feb 11, 2020

gavofyork added the U1-asap No need to stop dead in your tracks, however issue should be addressed as soon as possible. label Feb 12, 2020

Squash and rebase from gav-lazy-reaping

9027e19

gavofyork force-pushed the gav-lazy-reaping branch from 6622d40 to 9027e19 Compare February 15, 2020 21:10

gavofyork added 3 commits February 15, 2020 22:18

Bump version

1081058

Bump runtime again

30cf0f3

Merge remote-tracking branch 'origin/master' into gav-lazt-reaping-new

644a1a4

shawntabrizi self-requested a review February 18, 2020 10:46

shawntabrizi reviewed Feb 18, 2020

View reviewed changes

frame/system/src/lib.rs Outdated Show resolved Hide resolved

gavofyork mentioned this pull request Feb 18, 2020

Prepare for Polkadot launch and Substrate 2.0 freeze #4961

Closed

24 tasks

Docs.

11cd085

shawntabrizi reviewed Feb 18, 2020

View reviewed changes

frame/staking/src/lib.rs Show resolved Hide resolved

shawntabrizi reviewed Feb 18, 2020

View reviewed changes

frame/recovery/src/lib.rs Outdated Show resolved Hide resolved

shawntabrizi reviewed Feb 18, 2020

View reviewed changes

frame/balances/src/lib.rs Show resolved Hide resolved

shawntabrizi reviewed Feb 18, 2020

View reviewed changes

frame/contracts/src/lib.rs Show resolved Hide resolved

paritytech deleted a comment from shawntabrizi Feb 18, 2020

gavofyork and others added 5 commits February 18, 2020 15:13

Remove old functions

8de91c4

Update frame/balances/src/lib.rs

33d1a8d

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

Update frame/contracts/src/lib.rs

c9c8e5f

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

Warnings

9f18519

Bump runtime version

572efb3

shawntabrizi reviewed Feb 18, 2020

View reviewed changes

frame/democracy/src/lib.rs Outdated Show resolved Hide resolved

shawntabrizi added 3 commits February 21, 2020 15:17

Use frame_support debug

1181bc2

Merge remote-tracking branch 'upstream/master' into gav-lazy-reaping

3c22d02

Bump spec

3741e90

shawntabrizi added A8-looksgood and removed A5-grumble labels Feb 21, 2020

Merge branch 'master' into gav-lazy-reaping

6c0c845

gavofyork added 2 commits February 22, 2020 15:36

Renames and fix

e6c09eb

Fix

c2b6076

shawntabrizi and others added 4 commits February 23, 2020 04:24

Fix rename

562f87d

Fix

f07b8e1

Merge branch 'gav-lazy-reaping' of github.com:paritytech/substrate in…

6de2216

…to gav-lazy-reaping

Increase time for test

9c59bcd

paritytech deleted a comment from parity-benchapp bot Feb 24, 2020

gnunicorn merged commit ddfa359 into master Feb 24, 2020

gnunicorn deleted the gav-lazy-reaping branch February 24, 2020 17:04

This was referenced Feb 24, 2020

Ensure accounts correct reference count by adding inc_ref and dec_ref AcalaNetwork/Acala#122

Closed

Ensure accounts correct reference count by adding inc_ref and dec_ref laminar-protocol/laminar-chain#82

Open

ascjones mentioned this pull request Feb 25, 2020

Use new AccountInfo struct for "System Account" storage data paritytech/subxt#71

Merged

jordy25519 mentioned this pull request Mar 4, 2020

Latest Upstream Merge plugblockchain/plug-blockchain#72

Merged

AurevoirXavier mentioned this pull request Mar 5, 2020

Review: Account's References darwinia-network/darwinia#317

Closed

joepetrowski mentioned this pull request Mar 10, 2020

Update governance proxy functions paritytech/txwrapper#83

Closed

xlc mentioned this pull request May 29, 2020

Allow ExistentialDeposit = 0 #6185

Merged

1 task

apopiak mentioned this pull request Jun 4, 2020

Runtime migrations tracking issue hicommonwealth/edgeware-node#164

Closed

20 tasks

apopiak added the runtime-migration label Jun 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lazy reaping #4895

Lazy reaping #4895

gavofyork commented Feb 11, 2020 •

edited

shawntabrizi Feb 18, 2020 •

edited

gavofyork Feb 19, 2020

shawntabrizi Feb 19, 2020 •

edited

shawntabrizi Feb 21, 2020

NikVolf commented Feb 21, 2020

parity-benchapp bot commented Feb 21, 2020 •

edited

gavofyork commented Feb 22, 2020

NikVolf commented Feb 22, 2020

	Account::<T>::insert(k, AccountInfo{ nonce: Default::default(), refcount: 0, data });
	Account::<T>::mutate(k, \|a\| a.data = data);

Lazy reaping #4895

Lazy reaping #4895

Conversation

gavofyork commented Feb 11, 2020 • edited

shawntabrizi Feb 18, 2020 • edited

Choose a reason for hiding this comment

gavofyork Feb 19, 2020

Choose a reason for hiding this comment

shawntabrizi Feb 19, 2020 • edited

Choose a reason for hiding this comment

shawntabrizi Feb 21, 2020

Choose a reason for hiding this comment

NikVolf commented Feb 21, 2020

parity-benchapp bot commented Feb 21, 2020 • edited

MASTER RESULT

BRANCH RESULT

gavofyork commented Feb 22, 2020

NikVolf commented Feb 22, 2020

gavofyork commented Feb 11, 2020 •

edited

shawntabrizi Feb 18, 2020 •

edited

shawntabrizi Feb 19, 2020 •

edited

parity-benchapp bot commented Feb 21, 2020 •

edited