Metadata difference command #1015

tadeohepperle · 2023-06-14T11:32:40Z

Explore the differences between two nodes

Example

subxt diff ./artifacts/polkadot_metadata_small.scale ./artifacts/polkadot_metadata_tiny.scale
subxt diff ./artifacts/polkadot_metadata_small.scale wss://rpc.polkadot.io:443

Example output (colored in cli):

Pallets:
    ~ Balances
        Calls:
            - transfer_allow_death
            - force_set_balance
            + set_balance
            - upgrade_accounts
            - set_balance_deprecated
        Constants:
            - MaxHolds
            - MaxFreezes
        Storage Entries:
            - Freezes
            ~ Account (Changed: value type, default value)
            - Holds
    + Scheduler
    + Session
    + Democracy
    + Claims
    + Crowdloan
    + Registrar
    + FastUnstake
    + Initializer
    + Offences
    + Bounties
    + Treasury
    ~ System
        Storage Entries:
            ~ Events (Changed: value type)
            ~ Account (Changed: value type, default value)
    + ChildBounties
    + Babe
    + ElectionProviderMultiPhase
    + Configuration
    + ParaInherent
    + VoterList
    + Hrmp
    + Ump
    + ParasShared
    ~ Multisig
        Calls:
            ~ as_multi
            ~ as_multi_threshold_1
    + PhragmenElection
    + Utility
    + Proxy
    + ParachainsOrigin
    + Paras
    + NominationPools
    + Auctions
    + Timestamp
    + Indices
    + Grandpa
    + AuthorityDiscovery
    + Slots
    + TransactionPayment
    + Historical
    + TechnicalMembership
    + ImOnline
    + ParaSessionInfo
    + Authorship
    + Dmp
    + ParasDisputes
    + Preimage
    + ParaScheduler
    + TechnicalCommittee
    + Vesting
    + Identity
    + XcmPallet
    + Council
    + Tips
    + ParaInclusion
Runtime APIs:
    - TransactionPaymentCallApi
    - Core
    - MmrApi
    - BeefyApi
    - GrandpaApi
    - AccountNonceApi
    - BlockBuilder
    - TaggedTransactionQueue
    - StakingApi
    - AuthorityDiscoveryApi
    - SessionKeys
    - BabeApi
    - Metadata
    - NominationPoolsApi
    - ParachainHost
    - OffchainWorkerApi
    - TransactionPaymentApi

cli/src/commands/diff.rs

niklasad1 · 2023-06-15T09:01:58Z

cli/src/commands/diff.rs

+    node1: String,
+    node2: String,
+    #[clap(long, short)]
+    version: Option<MetadataVersion>,


I don't get why version is a CLI parameter, how would one insert --version here and why?

Better to read that from the metadata itself?

The version is which version of metadata is being requested when a url is given (eg are we downloading V14 metadatas or unstable-v15 metadatas or "whatever is latest stable"; something like that)

niklasad1 · 2023-06-15T09:07:46Z

cli/src/commands/diff.rs

+    #[clap(long, short)]
+    version: Option<MetadataVersion>,
+    #[clap(long, short)]
+    pallet: Option<String>,


This isn't used AFAIU, what's intention behind it? Just compare a single pallet in the metadata?

If we want this, it would be better with Vec to compare a one or more pallets

xlc · 2023-06-15T21:43:52Z

doesn't subwasm already have something like this?

cli/src/commands/diff.rs

niklasad1 · 2023-06-16T08:52:50Z

doesn't subwasm already have something like this?

Yes, subwasm has something similar but it compares wasm files and this/subxt operates on metadata either by passing an URL or a path to the metadata. Perhaps the subxt output might be a little easier to read because of the scale encode/decode stuff better to try it out :)

Both are useful but if you are already a user of subxt the diff command is more ergonomic to use, most likely one has already embedded a static metadata file in the binary.

jsdw · 2023-06-16T10:59:15Z

Some small feedback bits re the output:

wonder whether <ENTRY1> and <ENTRY2> could be followed by help like "metadata file or node URL" in the following?:

error: the following required arguments were not provided:
  <ENTRY1>
  <ENTRY2>

Usage: subxt diff <ENTRY1> <ENTRY2>

For more information, try '--help'.

It appears (and please correct me if I'm wrong) that subxt diff A B will show changes relative to "A", ie which pallets were added or removed to get to A. I'd normally expect a diff command to show changes with respect to "B". Could this be swapped around?

Otherwise, the output looks fantastic and I love the use of colours :)

jsdw · 2023-06-16T14:38:48Z

cli/src/commands/diff.rs

+    let mut calls: CallsHashmap = HashMap::new();
+    if let Some(call_variants) = pallet_metadata_1.call_variants() {
+        for call_variant in call_variants {
+            let (e1, _) = calls.entry(&call_variant.name).or_default();
+            *e1 = Some(call_variant);
+        }
+    }
+    if let Some(call_variants) = pallet_metadata_2.call_variants() {
+        for call_variant in call_variants {
+            let (e1, e2) = calls.entry(&call_variant.name).or_default();
+            // skip all entries with the same hash:
+            if let Some(e1_inner) = e1 {
+                let e1_hash = pallet_metadata_1
+                    .call_hash(&e1_inner.name)
+                    .expect("call should be present");
+                let e2_hash = pallet_metadata_2
+                    .call_hash(&call_variant.name)
+                    .expect("call should be present");
+                if e1_hash == e2_hash {
+                    calls.remove(call_variant.name.as_str());
+                    continue;
+                }
+            }
+            *e2 = Some(call_variant);
+        }
+    }


The general pattern of "give me a list of the differences between two things" feels like something that could be abstracted here since it's repeated several times; the only logic that changes each time is "how do I turn a thing into something comparable".

I'd propose it'd be a fun challenge to have a go at doing this abstracting; I could imagine ultimately ending with a signature something like this:

fn diff<T, C: PartialEq>( items_a: impl IntoIterator<Item = T>, comparable_a: impl Fn(&T) -> C items_b: impl IntoIterator<Item = T>, comparable_b: impl Fn(&T) -> C ) -> Vec<Diff<T>>

Doing this might also negate the need for a slightly arbitrary TryFrom<(a,b)> type thing from tuples to Diff, since it would all be done in this one place, and would make it easier to check the diffing logic is all good in one place rather than several :)

jsdw · 2023-06-16T14:42:40Z

metadata/src/lib.rs

@@ -509,6 +519,11 @@ impl<'a> RuntimeApiMetadata<'a> {
    pub fn method_hash(&self, method_name: &str) -> Option<[u8; 32]> {
        crate::utils::validation::get_runtime_api_hash(self, method_name)
    }
+
+    /// Return a hash for the entire pallet.


Suggested change

/// Return a hash for the entire pallet.

/// Return a hash for the entire runtime API trait.

The comment on 519 is also wrong, whoops! It should be "Return a hash for the method, or None if it was not found"

jsdw · 2023-06-16T14:45:46Z

cli/src/commands/diff.rs

+    entry1: FileOrUrl,
+    entry2: FileOrUrl,


Perhaps a slightly more descriptive name for these like metadata_or_url1 and metadata_or_url2 would make the output a little clearer.

I wonder whether adding comments above these would lead to nicer help being shown in the output too :)

jsdw · 2023-06-16T14:46:38Z

Looks great; just a few suggestions :)

tadeohepperle · 2023-06-19T10:48:42Z

it appears (and please correct me if I'm wrong) that subxt diff A B will show changes relative to "A", ie which pallets were added or removed to get to A.

@jsdw I think maybe you understood it wrong. It shows things with a + that are present in B but not in A, so things that were added to get from A to B. And vice versa. For example consider this:
subxt diff ./artifacts/polkadot_metadata_small.scale ./artifacts/polkadot_metadata_tiny.scale
gives as output

Pallets:
    - Staking
    - Balances
    - Multisig
    - System
Runtime APIs:
    ~ TransactionPaymentCallApi

indicating that some pallets are removed going from small to tiny.
Or maybe I am understanding it wrong?

jsdw · 2023-06-19T10:51:41Z

Oh interesting; hopefully I just got it wrong then, lemme check! I compared kuama to polkadot iirc and it seemed as if it was the otherway round but I'll try again to confirm :)

Edit: yup my mistake; I played around some more and must have just confused myself!

…fference

jsdw · 2023-06-19T13:32:57Z

metadata/src/lib.rs

@@ -303,8 +313,8 @@ impl StorageMetadata {
    }

    /// An iterator over the storage entries.
-    pub fn entries(&self) -> impl ExactSizeIterator<Item = &StorageEntryMetadata> {


Not really a big deal, but I was wodnering whether there was a reason for this change?

IIRC the reason I tended to iterate over other metadata structs exposed here rather than give back slices was just to give a little more wiggle room in case we wanted to change the internals a bit in the future. There are a couple of other places in this API that return the ExactSizeIterator even though they could return slices too for this reason :)

I did it because I needed to do an unwrap_or_default() and I could not get a default impl ExactSizeIterator working properly that is the exact same type as the self.entries.values().iter().
Hmm, do you have an idea how I could solve this?

pallet_metadata_1 .storage() .map(|s| s.entries()) .unwrap_or_default()

jsdw · 2023-06-19T13:35:53Z

cli/src/commands/diff.rs

+    Removed(T),
+}
+
+fn diff<T, C: PartialEq, I: Hash + PartialEq + Eq>(


Awesome! Much prefer this :)

(and with this logic now separated, you could also have a quick unit test or two that diffing things using this function works as expected :))

I added a unit test. This also made me realize that the output order was random because of the random nature of hash maps. So I added alphabetical ordering as well.

jsdw · 2023-06-21T11:45:35Z

Awesome stuff; good job @tadeohepperle!

lexnv

Lovely PR! Amazing job here 🙏

tadeohepperle added 10 commits June 12, 2023 19:28

diffing pallets and runtime apis

b635538

print diff

e67e16e

clippy fix and format

3adcf3d

change formatting

c21e41d

fmt

de2ed82

diff working with storage details

d192808

Merge branch 'master' into tadeo-hepperle-metadata-difference

4a74eb7

fix diff

1e0019f

cargo fmt

3134d86

remove printing of node

ed8595d

tadeohepperle marked this pull request as ready for review June 14, 2023 12:21

tadeohepperle requested a review from a team as a code owner June 14, 2023 12:21

change strings

37b4ec5

niklasad1 reviewed Jun 15, 2023

View reviewed changes

cli/src/commands/diff.rs Outdated Show resolved Hide resolved

niklasad1 reviewed Jun 15, 2023

View reviewed changes

cli/src/commands/diff.rs Outdated Show resolved Hide resolved

niklasad1 reviewed Jun 15, 2023

View reviewed changes

jsdw mentioned this pull request Jun 15, 2023

Implement stable type id paritytech/scale-info#182

Open

tadeohepperle added 3 commits June 15, 2023 13:42

Merge branch 'master' into tadeo-hepperle-metadata-difference

ee93467

handle parsing differently

42648f1

clippy fix

624a9cb

cargo fmt

f1a6c70

niklasad1 reviewed Jun 16, 2023

View reviewed changes

cli/src/commands/diff.rs Outdated Show resolved Hide resolved

niklasad1 reviewed Jun 16, 2023

View reviewed changes

cli/src/commands/diff.rs Outdated Show resolved Hide resolved

niklasad1 reviewed Jun 16, 2023

View reviewed changes

cli/src/commands/diff.rs Outdated Show resolved Hide resolved

niklasad1 approved these changes Jun 16, 2023

View reviewed changes

jsdw reviewed Jun 16, 2023

View reviewed changes

more abstraction

cbd2437

tadeohepperle added 2 commits June 19, 2023 12:52

clippy fix and fmt

07281d1

Merge remote-tracking branch 'origin' into tadeo-hepperle-metadata-di…

523f6b9

…fference

jsdw reviewed Jun 19, 2023

View reviewed changes

tadeohepperle added 2 commits June 19, 2023 18:45

add unit test and ordering

df3e59f

fix small issue

f0f14ab

jsdw approved these changes Jun 21, 2023

View reviewed changes

lexnv approved these changes Jun 21, 2023

View reviewed changes

tadeohepperle merged commit 2a990ed into master Jun 21, 2023
6 checks passed

tadeohepperle deleted the tadeo-hepperle-metadata-difference branch June 21, 2023 12:33

jsdw mentioned this pull request Jul 24, 2023

Prep for 0.30 release #1089

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Metadata difference command #1015

Metadata difference command #1015

tadeohepperle commented Jun 14, 2023 •

edited

Loading

niklasad1 Jun 15, 2023

jsdw Jun 15, 2023

niklasad1 Jun 15, 2023

xlc commented Jun 15, 2023

niklasad1 commented Jun 16, 2023 •

edited

Loading

jsdw commented Jun 16, 2023 •

edited

Loading

jsdw Jun 16, 2023 •

edited

Loading

jsdw Jun 16, 2023 •

edited

Loading

jsdw Jun 16, 2023

jsdw commented Jun 16, 2023

tadeohepperle commented Jun 19, 2023

jsdw commented Jun 19, 2023 •

edited

Loading

jsdw Jun 19, 2023

tadeohepperle Jun 19, 2023

jsdw Jun 19, 2023

jsdw Jun 19, 2023

tadeohepperle Jun 19, 2023

jsdw commented Jun 21, 2023

lexnv left a comment

	/// Return a hash for the entire pallet.
	/// Return a hash for the entire runtime API trait.

Metadata difference command #1015

Metadata difference command #1015

Conversation

tadeohepperle commented Jun 14, 2023 • edited Loading

Example

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xlc commented Jun 15, 2023

niklasad1 commented Jun 16, 2023 • edited Loading

jsdw commented Jun 16, 2023 • edited Loading

jsdw Jun 16, 2023 • edited Loading

Choose a reason for hiding this comment

jsdw Jun 16, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsdw commented Jun 16, 2023

tadeohepperle commented Jun 19, 2023

jsdw commented Jun 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsdw commented Jun 21, 2023

lexnv left a comment

Choose a reason for hiding this comment

tadeohepperle commented Jun 14, 2023 •

edited

Loading

niklasad1 commented Jun 16, 2023 •

edited

Loading

jsdw commented Jun 16, 2023 •

edited

Loading

jsdw Jun 16, 2023 •

edited

Loading

jsdw Jun 16, 2023 •

edited

Loading

jsdw commented Jun 19, 2023 •

edited

Loading