Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the LocalAssets pallet-asset instance #2634

Merged
merged 45 commits into from Feb 15, 2024
Merged

Conversation

RomarQ
Copy link
Contributor

@RomarQ RomarQ commented Jan 31, 2024

What does it do?

Removes the local assets pallet.

  • Remove LocalAssets pallet_assets instance
  • Add migration to remove the storage (Clear all storage keys prefixed with LocalAssets and revert codes of local assets stored in pallet_evm::AccountCodes)
  • Replace Erc20AssetsPrecompileSet<R, IsForeign, ForeignAssetInstance> with another PrecompileSet for reverting calls on all local xc-20 addresses.
  • Apply all the tasks above to every runtime
  • Add new pallet for performing multi block migrations

What important points reviewers should know?

The mintable xc-20s assets were deprecated in #2508

Is there something left for follow-up PRs?

What alternative implementations were considered?

Are there relevant PRs or issues in other repositories (Substrate, Polkadot, Frontier, Cumulus)?

What value does it bring to the blockchain users?

@RomarQ RomarQ requested a review from librelois January 31, 2024 15:44
@RomarQ RomarQ added the B7-runtimenoteworthy Changes should be noted in any runtime-upgrade release notes label Jan 31, 2024
@RomarQ RomarQ self-assigned this Jan 31, 2024
@RomarQ RomarQ added D5-nicetohaveaudit⚠️ PR contains trivial changes to logic that should be properly reviewed. breaking Needs to be mentioned in breaking changes labels Jan 31, 2024
Copy link
Contributor

github-actions bot commented Jan 31, 2024

Coverage Report

@@                       Coverage Diff                       @@
##           master   rq/deprecate-mintable-xc20s      +/-   ##
===============================================================
- Coverage   80.91%                        74.06%   -6.85%     
- Files         287                           221      -66     
- Lines       94493                         71216   -23277     
===============================================================
- Hits        76456                         52746   -23710     
+ Misses      18037                         18470     +433     
Files Changed Coverage
/client/rpc/finality/src/lib.rs 0.00% (-100.00%) 🔽
/client/rpc/manual-xcm/src/lib.rs 0.00% (-84.62%) 🔽
/client/rpc/txpool/src/lib.rs 0.00% (-65.38%) 🔽
/client/rpc-core/txpool/src/lib.rs 0.00% (-100.00%) 🔽
/client/rpc-core/txpool/src/types/content.rs 0.00% (-70.37%) 🔽
/client/rpc-core/txpool/src/types/inspect.rs 0.00% (-88.89%) 🔽
/client/rpc-core/txpool/src/types/mod.rs 0.00% (-100.00%) 🔽
/client/vrf/src/lib.rs 0.00% (-96.00%) 🔽
/node/cli-opt/src/account_key.rs 31.15% (-1.64%) 🔽
/node/cli-opt/src/lib.rs 0.00% (-48.00%) 🔽
/pallets/asset-manager/src/lib.rs 76.26% (-7.64%) 🔽
/pallets/asset-manager/src/mock.rs 97.90% (+4.09%) 🔼
/pallets/asset-manager/src/tests.rs 99.79% (-0.21%) 🔽
/pallets/erc20-xcm-bridge/src/erc20_matcher.rs 92.77% (-4.88%) 🔽
/pallets/erc20-xcm-bridge/src/lib.rs 8.90% (-3.26%) 🔽
/pallets/erc20-xcm-bridge/src/mock.rs 0.96% (-3.55%) 🔽
/pallets/erc20-xcm-bridge/src/xcm_holding_ext.rs 81.20% (-13.67%) 🔽
/pallets/ethereum-xcm/src/lib.rs 82.09% (-2.69%) 🔽
/pallets/ethereum-xcm/src/mock.rs 32.29% (-6.88%) 🔽
/pallets/ethereum-xcm/src/tests/v1/eip1559.rs 99.41% (-0.01%) 🔽
/pallets/ethereum-xcm/src/tests/v1/eip2930.rs 99.43% (-0.01%) 🔽
/pallets/moonbeam-orbiters/src/lib.rs 69.80% (-4.39%) 🔽
/pallets/moonbeam-orbiters/src/mock.rs 99.19% (+1.98%) 🔼
/pallets/moonbeam-orbiters/src/types.rs 65.26% (+0.61%) 🔼
/pallets/moonbeam-xcm-benchmarks/src/weights/fungible.rs 0.00% (-24.24%) 🔽
/pallets/moonbeam-xcm-benchmarks/src/weights/generic.rs 0.00% (-2.96%) 🔽
/pallets/moonbeam-xcm-benchmarks/src/weights/mod.rs 0.00% (-13.11%) 🔽
/pallets/parachain-staking/src/auto_compound.rs 95.64% (-0.03%) 🔽
/pallets/parachain-staking/src/delegation_requests.rs 91.92% (-0.02%) 🔽
/pallets/parachain-staking/src/inflation.rs 85.98% (-1.52%) 🔽
/pallets/parachain-staking/src/lib.rs 90.24% (-0.38%) 🔽
/pallets/parachain-staking/src/mock.rs 99.28% (-0.14%) 🔽
/pallets/parachain-staking/src/set.rs 40.00% (-0.74%) 🔽
/pallets/parachain-staking/src/tests.rs 91.57% (-0.02%) 🔽
/pallets/parachain-staking/src/types.rs 77.35% (-0.56%) 🔽
/pallets/proxy-genesis-companion/src/lib.rs 57.14% (-24.68%) 🔽
/pallets/proxy-genesis-companion/src/mock.rs 7.98% (-19.70%) 🔽
/pallets/xcm-transactor/src/encode.rs 90.32% (-9.68%) 🔽
/pallets/xcm-transactor/src/lib.rs 85.64% (-2.84%) 🔽
/pallets/xcm-transactor/src/mock.rs 95.04% (+2.96%) 🔼
/precompiles/assets-erc20/src/lib.rs 96.63% (-0.51%) 🔽
/precompiles/assets-erc20/src/mock.rs 9.95% (-69.54%) 🔽
/precompiles/assets-erc20/src/tests.rs 100.00% (+0.17%) 🔼
/precompiles/author-mapping/src/mock.rs 5.71% (-12.57%) 🔽
/precompiles/balances-erc20/src/mock.rs 99.60% (+2.27%) 🔼
/precompiles/batch/src/mock.rs 99.68% (+0.80%) 🔼
/precompiles/call-permit/src/lib.rs 97.83% (-0.01%) 🔽
/precompiles/call-permit/src/mock.rs 6.00% (-17.27%) 🔽
/precompiles/collective/src/mock.rs 96.86% (+2.60%) 🔼
/precompiles/conviction-voting/src/lib.rs 93.41% (+0.01%) 🔼
/precompiles/conviction-voting/src/mock.rs 97.05% (+3.72%) 🔼
/precompiles/crowdloan-rewards/src/mock.rs 99.17% (+1.10%) 🔼
/precompiles/crowdloan-rewards/src/tests.rs 98.47% (-0.01%) 🔽
/precompiles/gmp/src/mock.rs 5.62% (-2.73%) 🔽
/precompiles/identity/src/mock.rs 99.02% (+1.93%) 🔼
/precompiles/pallet-democracy/src/mock.rs 99.02% (+1.06%) 🔼
/precompiles/pallet-democracy/src/tests.rs 89.29% (-0.04%) 🔽
/precompiles/parachain-staking/src/mock.rs 98.85% (+1.04%) 🔼
/precompiles/precompile-registry/src/mock.rs 6.16% (-18.36%) 🔽
/precompiles/preimage/src/mock.rs 99.35% (+1.72%) 🔼
/precompiles/proxy/src/mock.rs 97.86% (+3.72%) 🔼
/precompiles/proxy/src/tests.rs 96.79% (-0.02%) 🔽
/precompiles/randomness/src/mock.rs 98.38% (+2.90%) 🔼
/precompiles/referenda/src/mock.rs 98.52% (+1.00%) 🔼
/precompiles/relay-encoder/src/mock.rs 5.25% (-3.69%) 🔽
/precompiles/utils/src/precompile_set.rs 61.06% (-16.66%) 🔽
/precompiles/utils/src/solidity/codec/bytes.rs 81.36% (-2.54%) 🔽
/precompiles/utils/src/solidity/codec/xcm.rs 67.95% (-0.14%) 🔽
/precompiles/utils/src/testing/modifier.rs 87.67% (-0.64%) 🔽
/precompiles/utils/tests-external/lib.rs 33.22% (-15.93%) 🔽
/precompiles/xcm-transactor/src/mock.rs 95.51% (+2.50%) 🔼
/precompiles/xcm-utils/src/lib.rs 88.52% (-0.82%) 🔽
/precompiles/xcm-utils/src/mock.rs 96.84% (+1.17%) 🔼
/precompiles/xtokens/src/mock.rs 16.25% (-7.47%) 🔽
/precompiles/xtokens/src/tests.rs 99.52% (-0.01%) 🔽
/primitives/account/src/lib.rs 64.23% (+0.52%) 🔼
/primitives/ext/src/lib.rs 44.83% (-3.45%) 🔽
/primitives/rpc/debug/src/lib.rs 0.00% (-84.85%) 🔽
/primitives/rpc/txpool/src/lib.rs 0.00% (-83.33%) 🔽
/primitives/xcm/src/asset_id_conversions.rs 0.00% (-66.67%) 🔽
/primitives/xcm/src/ethereum_xcm.rs 93.07% (-1.80%) 🔽
/primitives/xcm/src/fee_handlers.rs 95.57% (-2.85%) 🔽
/primitives/xcm/src/origin_conversion.rs 0.00% (-74.07%) 🔽
/runtime/common/src/migrations.rs 0.00% (-67.48%) 🔽
/runtime/common/src/weights/pallet_assets.rs 0.00% (-9.04%) 🔽
/runtime/common/src/weights/pallet_author_mapping.rs 0.00% (-80.00%) 🔽
/runtime/common/src/weights/pallet_balances.rs 0.00% (-12.05%) 🔽
/runtime/common/src/weights/pallet_collective.rs 0.00% (-11.25%) 🔽
/runtime/common/src/weights/pallet_crowdloan_rewards.rs 0.00% (-66.18%) 🔽
/runtime/common/src/weights/pallet_moonbeam_orbiters.rs 0.00% (-9.26%) 🔽
/runtime/common/src/weights/pallet_parachain_staking.rs 0.00% (-19.83%) 🔽
/runtime/common/src/weights/pallet_utility.rs 0.00% (-44.00%) 🔽
/runtime/common/src/weights/pallet_xcm_transactor.rs 0.00% (-11.90%) 🔽
/runtime/evm_tracer/src/lib.rs 0.00% (-65.96%) 🔽

Coverage generated Thu Feb 15 14:28:07 UTC 2024

@librelois
Copy link
Collaborator

@RomarQ seem's to be a good start, but we also need the migration that remove the pallet storage in the same PR.
Also, the removed precompiles instances should be repalced by the RevertPrecompiletype to behave properly with the precompile registry.
@nanocryk It's the first time that with remove a precompile with "instances", how cna we implement the RevertPrecompileto all the addresses with a prefix?

@nanocryk
Copy link
Contributor

nanocryk commented Feb 1, 2024

To properly revert for all previously "used" addresses of the assets precompile set, you should implement a new precompile set, which the "discriminant" function filtering on the list of addresses (probably hardcoded if they aren't many), and that will always revert. Something like the following:

#[precompile_utils::precompile]
#[precompile::precompile_set]
impl<Filter> BlacklistPrecompileSet<Filter>
where
    Filter: InstanceFilter<H160>
{
    #[precompile::discriminant]
    fn discriminant(address: H160, _gas: u64) -> DiscriminantResult<()> {
        if Filter::filter(&address) {
            return DiscriminantResult::Some((), 0);
        }

        DiscriminantResult::None(0)
    }

    #[precompile::fallback]
    #[precompile::payable]
    fn fallback(_handle: &mut impl PrecompileHandle) -> EvmResult {
        Err(revert("disabled precompile"))
    }
}

Avoid filtering all addresses with the prefix, as it may ban users.

@RomarQ
Copy link
Contributor Author

RomarQ commented Feb 1, 2024

I will give it a try, thanks for the suggestion @nanocryk 👍

@RomarQ
Copy link
Contributor Author

RomarQ commented Feb 2, 2024

Hi @nanocryk,

Is the blacklist really necessary?

After digging a bit into it, my understanding is that "destroyed local assets" are already not considered part of the PrecompileSet by this check in the discriminant verification:

        if pallet_assets::Pallet::<Runtime, Instance>::maybe_total_supply(asset_id.clone())
            .is_some()
        {
            DiscriminantResult::Some(asset_id, extra_cost)
        } else {
            DiscriminantResult::None(extra_cost) // <------ Excludes the address as being part of the PrecompileSet
        }

If my understanding is correct, shouldn't be enough to just remove all storage keys prefixed by LocalAssets and all "revert codes" of local assets stored in pallet_evm::AccountCodes?

All local xc-20 addresses that were not destroyed yet would be considered as simple codeless accounts I suspect.

@librelois
Copy link
Collaborator

my understanding is that "destroyed local assets" are already not considered part of the PrecompileSet by this check in the discriminant verification:

If it's true it's a bug, destroyed local asset should still revert, otherwise it might open an attack vector to existing contracts that call old xc-20s (by deploying a new contract to the same address)

shouldn't be enough to just remove all storage keys prefixed by LocalAssets and all "revert codes" of local assets stored in pallet_evm::AccountCodes?

All local xc-20 addresses that were not destroyed yet would be considered as simple codeless accounts I suspect.

I think we should still revert for all the addresses that had an xc-20 in the past.

@albertov19 do we have a list of all the old xc-20s for moonriver and moonbeam?

@albertov19
Copy link
Contributor

@albertov19 do we have a list of all the old xc-20s for moonriver and moonbeam?

In what sense? The only non-destroyed Mintable XC-20 right now is on Moonbeam with AssetID 270,195,117,769,614,861,929,703,564,202,131,636,628

The other three were destroyed successfully -> https://moonbeam.subscan.io/event?page=1&time_dimension=date&module=localassets&event_id=destroyed

@librelois
Copy link
Collaborator

In what sense?

because it seem's that we have a bug, so we should list all destroyed local assets to replace them by explicit RevertPrecompile, to not break the precompile registry

@albertov19
Copy link
Contributor

albertov19 commented Feb 2, 2024

In what sense?

because it seem's that we have a bug, so we should list all destroyed local assets to replace them by explicit RevertPrecompile, to not break the precompile registry

Ok cool so the destroyed assets ID are:

Asset 1:

  • ID 337110116006454532607322340792629567158
  • Precompile Address 0xfffffffeFD9D0BF45A2947A519A741C4B9E99EB6

Asset 2:

  • ID 278750993613512357835566279094880339619
  • Precompile Address 0xfffffffeD1B57D12738E41EAE0124E285E5E86A3

Asset 3:

  • ID 228256396637196286254896220398224702687
  • Precompile Address 0xfffffffeABB8953AC77EDECE4A6E251A849C7CDF

Asset not yet destroyed (soon to be, allegedly):

  • ID 270195117769614861929703564202131636628
  • Precompile Address 0xfffffffeCB45AFD30A637967995394CC88C0C194

@RomarQ RomarQ marked this pull request as ready for review February 5, 2024 13:05
Copy link
Collaborator

@librelois librelois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the code that remove pallet assets storage?
Probably a migration is not safe, it might be better to create a temporary extrinsic to lazily remove batchs of storage items that start with the removed pallet prefix

@RomarQ
Copy link
Contributor Author

RomarQ commented Feb 6, 2024

@librelois
Copy link
Collaborator

The code that removed the pallet storage is here: https://github.com/paritytech/polkadot-sdk/blob/7df1ae3b8111d534cce108b2b405b6a33fcdedc3/substrate/frame/support/src/migrations.rs#L299

We should not use that directly, this RemovePallet type doesn't account for proof size, it's very dangerous because if the migration produce a tto big PoV the parachain will be stall and we will need to rely on the polkadot governance to force a runtime upgrade, it would be far too dramatic.
We need to implement an extrinsic that removes a certain number of items based on a proof_size_limit given as an extrinsic parameter.

@RomarQ
Copy link
Contributor Author

RomarQ commented Feb 6, 2024

The code that removed the pallet storage is here: https://github.com/paritytech/polkadot-sdk/blob/7df1ae3b8111d534cce108b2b405b6a33fcdedc3/substrate/frame/support/src/migrations.rs#L299

We should not use that directly, this RemovePallet type doesn't account for proof size, it's very dangerous because if the migration produce a tto big PoV the parachain will be stall and we will need to rely on the polkadot governance to force a runtime upgrade, it would be far too dramatic. We need to implement an extrinsic that removes a certain number of items based on a proof_size_limit given as an extrinsic parameter.

Ok, will add a new extrinsic to pallet_migration in a different PR.

@RomarQ RomarQ requested a review from librelois February 6, 2024 14:56
@librelois
Copy link
Collaborator

@RomarQ with the merge of the Pr that mvoe all tes tests files the changes become very unreadable :/

Can you exceptionally rebase your branch against master and force push?
Force push is usually forbidden because it forces us to reread all changes, but in this case I'd rather reread all changes than be polluted by 500 moved files.

@RomarQ
Copy link
Contributor Author

RomarQ commented Feb 13, 2024

@RomarQ with the merge of the Pr that mvoe all tes tests files the changes become very unreadable :/

Can you exceptionally rebase your branch against master and force push? Force push is usually forbidden because it forces us to reread all changes, but in this case I'd rather reread all changes than be polluted by 500 moved files.

Those moved test files come from this PR (I ended up merging it to this branch since I needed the changes, it is not on master yet): #2636

I can move the tests to another PR if necessary.

Copy link
Collaborator

@librelois librelois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, but overall it's very good now :)

RomarQ and others added 2 commits February 14, 2024 10:56
Co-authored-by: Éloïs <c@elo.tf>
Co-authored-by: Éloïs <c@elo.tf>
@librelois librelois added D9-needsaudit👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited and removed D5-nicetohaveaudit⚠️ PR contains trivial changes to logic that should be properly reviewed. labels Feb 14, 2024
@librelois
Copy link
Collaborator

@ahmadkaouk can you review this PR?

@RomarQ RomarQ added the A8-mergeoncegreen Pull request is reviewed well. label Feb 15, 2024
@RomarQ RomarQ merged commit 7a73691 into master Feb 15, 2024
29 checks passed
@RomarQ RomarQ deleted the rq/deprecate-mintable-xc20s branch February 15, 2024 14:34
@noandrea noandrea removed the D9-needsaudit👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited label Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A8-mergeoncegreen Pull request is reviewed well. B7-runtimenoteworthy Changes should be noted in any runtime-upgrade release notes breaking Needs to be mentioned in breaking changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants