Tracker issue for cleaning up old non-paged exposure logic in staking pallet #433
Assignees
Comments
5 tasks
Ank4n
changed the title
StakingLedger.claimed_rewards
11 tasks
Ank4n
changed the title
11 tasks
Ank4n
added a commit
that referenced
this issue
Nov 1, 2023
…1189) helps #439. closes #473. PR link in the older substrate repository: paritytech/substrate#13498. # Context Rewards payout is processed today in a single block and limited to `MaxNominatorRewardedPerValidator`. This number is currently 512 on both Kusama and Polkadot. This PR tries to scale the nominators payout to an unlimited count in a multi-block fashion. Exposures are stored in pages, with each page capped to a certain number (`MaxExposurePageSize`). Starting out, this number would be the same as `MaxNominatorRewardedPerValidator`, but eventually, this number can be lowered through new runtime upgrades to limit the rewardeable nominators per dispatched call instruction. The changes in the PR are backward compatible. ## How payouts would work like after this change Staking exposes two calls, 1) the existing `payout_stakers` and 2) `payout_stakers_by_page`. ### payout_stakers This remains backward compatible with no signature change. If for a given era a validator has multiple pages, they can call `payout_stakers` multiple times. The pages are executed in an ascending sequence and the runtime takes care of preventing double claims. ### payout_stakers_by_page Very similar to `payout_stakers` but also accepts an extra param `page_index`. An account can choose to payout rewards only for an explicitly passed `page_index`. **Lets look at an example scenario** Given an active validator on Kusama had 1100 nominators, `MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards to all nominators, the caller would need to call `payout_stakers` 3 times. - `payout_stakers(origin, stash, e)` => will pay the first 512 nominators. - `payout_stakers(origin, stash, e)` => will pay the second set of 512 nominators. - `payout_stakers(origin, stash, e)` => will pay the last set of 76 nominators. ... - `payout_stakers(origin, stash, e)` => calling it the 4th time would return an error `InvalidPage`. The above calls can also be replaced by `payout_stakers_by_page` and passing a `page_index` explicitly. ## Commission note Validator commission is paid out in chunks across all the pages where each commission chunk is proportional to the total stake of the current page. This implies higher the total stake of a page, higher will be the commission. If all the pages of a validator's single era are paid out, the sum of commission paid to the validator across all pages should be equal to what the commission would have been if we had a non-paged exposure. ### Migration Note Strictly speaking, we did not need to bump our storage version since there is no migration of storage in this PR. But it is still useful to mark a storage upgrade for the following reasons: - New storage items are introduced in this PR while some older storage items are deprecated. - For the next `HistoryDepth` eras, the exposure would be incrementally migrated to its corresponding paged storage item. - Runtimes using staking pallet would strictly need to wait at least `HistoryDepth` eras with current upgraded version (14) for the migration to complete. At some era `E` such that `E > era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to version X which will remove the deprecated storage items. In other words, it is a strict requirement that E<sub>x</sub> - E<sub>14</sub> > `HistoryDepth`, where E<sub>x</sub> = Era at which deprecated storages are removed from runtime, E<sub>14</sub> = Era at which runtime is upgraded to version 14. - For Polkadot and Kusama, there is a [tracker ticket](#433) to clean up the deprecated storage items. ### Storage Changes #### Added - ErasStakersOverview - ClaimedRewards - ErasStakersPaged #### Deprecated The following can be cleaned up after 84 eras which is tracked [here](#433). - ErasStakers. - ErasStakersClipped. - StakingLedger.claimed_rewards, renamed to StakingLedger.legacy_claimed_rewards. ### Config Changes - Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize. ### TODO - [x] Tracker ticket for cleaning up the old code after 84 eras. - [x] Add companion. - [x] Redo benchmarks before merge. - [x] Add Changelog for pallet_staking. - [x] Pallet should be configurable to enable/disable paged rewards. - [x] Commission payouts are distributed across pages. - [x] Review documentation thoroughly. - [x] Rename `MaxNominatorRewardedPerValidator` -> `MaxExposurePageSize`. - [x] NMap for `ErasStakersPaged`. - [x] Deprecate ErasStakers. - [x] Integrity tests. ### Followup issues [Runtime api for deprecated ErasStakers storage item](#426) --------- Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Ross Bulat <ross@parity.io> Co-authored-by: command-bot <>
bgallois
pushed a commit
to duniter/duniter-polkadot-sdk
that referenced
this issue
Mar 25, 2024
…aritytech#1189) helps paritytech#439. closes paritytech#473. PR link in the older substrate repository: paritytech/substrate#13498. # Context Rewards payout is processed today in a single block and limited to `MaxNominatorRewardedPerValidator`. This number is currently 512 on both Kusama and Polkadot. This PR tries to scale the nominators payout to an unlimited count in a multi-block fashion. Exposures are stored in pages, with each page capped to a certain number (`MaxExposurePageSize`). Starting out, this number would be the same as `MaxNominatorRewardedPerValidator`, but eventually, this number can be lowered through new runtime upgrades to limit the rewardeable nominators per dispatched call instruction. The changes in the PR are backward compatible. ## How payouts would work like after this change Staking exposes two calls, 1) the existing `payout_stakers` and 2) `payout_stakers_by_page`. ### payout_stakers This remains backward compatible with no signature change. If for a given era a validator has multiple pages, they can call `payout_stakers` multiple times. The pages are executed in an ascending sequence and the runtime takes care of preventing double claims. ### payout_stakers_by_page Very similar to `payout_stakers` but also accepts an extra param `page_index`. An account can choose to payout rewards only for an explicitly passed `page_index`. **Lets look at an example scenario** Given an active validator on Kusama had 1100 nominators, `MaxExposurePageSize` set to 512 for Era e. In order to pay out rewards to all nominators, the caller would need to call `payout_stakers` 3 times. - `payout_stakers(origin, stash, e)` => will pay the first 512 nominators. - `payout_stakers(origin, stash, e)` => will pay the second set of 512 nominators. - `payout_stakers(origin, stash, e)` => will pay the last set of 76 nominators. ... - `payout_stakers(origin, stash, e)` => calling it the 4th time would return an error `InvalidPage`. The above calls can also be replaced by `payout_stakers_by_page` and passing a `page_index` explicitly. ## Commission note Validator commission is paid out in chunks across all the pages where each commission chunk is proportional to the total stake of the current page. This implies higher the total stake of a page, higher will be the commission. If all the pages of a validator's single era are paid out, the sum of commission paid to the validator across all pages should be equal to what the commission would have been if we had a non-paged exposure. ### Migration Note Strictly speaking, we did not need to bump our storage version since there is no migration of storage in this PR. But it is still useful to mark a storage upgrade for the following reasons: - New storage items are introduced in this PR while some older storage items are deprecated. - For the next `HistoryDepth` eras, the exposure would be incrementally migrated to its corresponding paged storage item. - Runtimes using staking pallet would strictly need to wait at least `HistoryDepth` eras with current upgraded version (14) for the migration to complete. At some era `E` such that `E > era_at_which_V14_gets_into_effect + HistoryDepth`, we will upgrade to version X which will remove the deprecated storage items. In other words, it is a strict requirement that E<sub>x</sub> - E<sub>14</sub> > `HistoryDepth`, where E<sub>x</sub> = Era at which deprecated storages are removed from runtime, E<sub>14</sub> = Era at which runtime is upgraded to version 14. - For Polkadot and Kusama, there is a [tracker ticket](paritytech#433) to clean up the deprecated storage items. ### Storage Changes #### Added - ErasStakersOverview - ClaimedRewards - ErasStakersPaged #### Deprecated The following can be cleaned up after 84 eras which is tracked [here](paritytech#433). - ErasStakers. - ErasStakersClipped. - StakingLedger.claimed_rewards, renamed to StakingLedger.legacy_claimed_rewards. ### Config Changes - Renamed MaxNominatorRewardedPerValidator to MaxExposurePageSize. ### TODO - [x] Tracker ticket for cleaning up the old code after 84 eras. - [x] Add companion. - [x] Redo benchmarks before merge. - [x] Add Changelog for pallet_staking. - [x] Pallet should be configurable to enable/disable paged rewards. - [x] Commission payouts are distributed across pages. - [x] Review documentation thoroughly. - [x] Rename `MaxNominatorRewardedPerValidator` -> `MaxExposurePageSize`. - [x] NMap for `ErasStakersPaged`. - [x] Deprecate ErasStakers. - [x] Integrity tests. ### Followup issues [Runtime api for deprecated ErasStakers storage item](paritytech#426) --------- Co-authored-by: Javier Viola <javier@parity.io> Co-authored-by: Ross Bulat <ross@parity.io> Co-authored-by: command-bot <>
|
Referenda for upgrade containing paged exposure: |
This was referenced Apr 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This issue is to track merging of paritytech/substrate#13059 and subsequent release of paged rewards on Polkadot. Once enabled, and 84 eras have passed, we can clean up incremental migration and fallback logic from the staking pallet.
84 eras in Polkadot = 84 days.
Deprecated storage items can be cleaned from codebase on or after Era 1504 (approx: July 14, 2024) on Polkadot.
Track
Tasks
ErasStakersClipped.EraInfo::is_rewards_claimed_temp.StakingLedger.legacy_claimed_rewards.Note on bounding:
I did not bound new paged storage items since the exposure page size would decrease in future which is unsafe for
bounded vecs. Initially, page size would be 512 and page count = 1, but after running for a while, we should set the page size such that we don't see them reducing further. For example, when we clean up this ticket, we could then bound the page size to32and page count to16. From that point, we should strive to never reduce the page size and page count should only increase further.The text was updated successfully, but these errors were encountered: