fix!: pallet storage metadataV14 #710
Conversation
| const blockNumber = blockHeader.number.toNumber(); | ||
|
|
||
| let adjustedMetadata; | ||
| if (blockNumber < upgradeBlocks[specName.toString()]) { |
There was a problem hiding this comment.
In case specName.toString() is not present in upgradeBlocks you will be comparing a number to undefined which, which AFAIU will be always false, hence always default to V13. Might want to make this a bit more robust?
There was a problem hiding this comment.
It will actually default to V14 which is what we want. I think you just got the two mixed up. For chains that aren't specified (unless they specify it themselves), we want it to default to V14
There was a problem hiding this comment.
This brings up a good point though, and ill test it against a outside network and see how it responds
TarikGul
changed the title
|
@dvdplm .... Didnt push my commit with the changes to the local |
| description: This gives the option to set historic blocks to use V13 metadata as oppose to V14. There are some | ||
| type differences between V14 and V13 when it comes to `StorageEntryType`. Specifally the 'map' key. | ||
| In turn this gives the ability to receive both older and newer storage responses while transitioning to V14. |
There was a problem hiding this comment.
| description: This gives the option to set historic blocks to use V13 metadata as oppose to V14. There are some | |
| type differences between V14 and V13 when it comes to `StorageEntryType`. Specifally the 'map' key. | |
| In turn this gives the ability to receive both older and newer storage responses while transitioning to V14. | |
| description: Instruct sidecar to return `StorageEntryType` in the V13 metdata format rather than V14. This is a **temporary** flag to allow existing systems to migrate. It will be deprecated and then removed in the future. |
| in: query | ||
| description: This gives the option to set historic blocks to use V13 metadata as oppose to V14. There are some | ||
| type differences between V14 and V13 when it comes to `StorageEntryType`. Specifally the 'map' key. | ||
| In turn this gives the ability to receive both older and newer storage responses while transitioning to V14. |
There was a problem hiding this comment.
I'd adjust along the same lines as above.
| linked: false | ||
| description: If the query parameter 'adjustMetadataV13' is set to true, all historic blocks that are | ||
| pre v9110 will have the return type `StorageEntryTypeV13`, and all present and post v9110 blocks will | ||
| have a return type of `StorageEntryTypeV14`. Please check those types to see potential responses. |
There was a problem hiding this comment.
Add note about the temporary nature of adjustMetadataV13?
|
rel: #698 |
closes: #678
closes: #712
Since the v9110 runtime upgrade, pallets storage now has a bug when querying the endpoint. This PR takes care of that issue.
/pallets/{storageId}/*[Added]:
query param:
adjustMetadataV13: This allows users to receive historic responses from pallets for pre-v9110 runtimes that match sidecar's old responses forPalletMetadata. Specifically it usesStorageEntryTypeV13whereas V14 now returnsStorageEntryTypeV14which has a differentasMap.Example query:
/pallets/democracy/storage?at=700000&adjustMetadataV13=true[Changed]
The response for
typeinside ofISanitizedStorageItemMetadata, will now useStorageEntryTypeV14as oppose to previous runtimes. But backwards historic compatibility will be an option, as stated above.ISanitizedStorageItemMetadatastates it's type as unknown because all previousStorageEntryType's shared the samemapkeys, but there were many types. This should have beenStorageEntryTypeLatestbefore this PR, but now it makes sense to keep it unknown.Example queries:
To demonstrate the differences between responses for historic v9110 blocks.
With
adjustMetadataV13=false/pallets/democracy/storage?at=9625125&adjustMetadataV13=false{ "at": { "hash": "0x2813b8eb5fd296bc0869ab3bdf092fdb6c0a9a5e016e1592b94bae699329feaf", "height": "9625125" }, "pallet": "democracy", "palletIndex": "13", "items": [ { "name": "PublicPropCount", "modifier": "Default", "type": { "plain": "145" }, "fallback": "0x00000000", "docs": " The number of (public) proposals that have been made so far." }, { "name": "PublicProps", "modifier": "Default", "type": { "plain": "149" }, "fallback": "0x00", "docs": " The public proposals. Unsorted. The second item is the proposal's hash." }, { "name": "DepositOf", "modifier": "Optional", "type": { "map": { "hashers": [ "Twox64Concat" ], "key": "145", "value": "150" } }, "fallback": "0x00", "docs": " Those who have locked a deposit.\n\n TWOX-NOTE: Safe, as increasing integer keys are safe." }, { "name": "Preimages", "modifier": "Optional", "type": { "map": { "hashers": [ "Identity" ], "key": "20", "value": "151" } }, "fallback": "0x00", "docs": " Map of hashes to the proposal preimage, along with who registered it and their deposit.\n The block number is the block at which it was deposited." }, { "name": "ReferendumCount", "modifier": "Default", "type": { "plain": "140" }, "fallback": "0x00000000", "docs": " The next free referendum index, aka the number of referenda started so far." }, { "name": "LowestUnbaked", "modifier": "Default", "type": { "plain": "140" }, "fallback": "0x00000000", "docs": " The lowest referendum index representing an unbaked referendum. Equal to\n `ReferendumCount` if there isn't a unbaked referendum." }, { "name": "ReferendumInfoOf", "modifier": "Optional", "type": { "map": { "hashers": [ "Twox64Concat" ], "key": "140", "value": "152" } }, "fallback": "0x00", "docs": " Information concerning any given referendum.\n\n TWOX-NOTE: SAFE as indexes are not under an attacker’s control." }, { "name": "VotingOf", "modifier": "Default", "type": { "map": { "hashers": [ "Twox64Concat" ], "key": "19", "value": "153" } }, "fallback": "0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "docs": " All votes for a particular voter. We store the balance for the number of votes that we\n have recorded. The second item is the total amount of delegations, that will be added.\n\n TWOX-NOTE: SAFE as `AccountId`s are crypto hashes anyway." }, { "name": "Locks", "modifier": "Optional", "type": { "map": { "hashers": [ "Twox64Concat" ], "key": "19", "value": "12" } }, "fallback": "0x00", "docs": " Accounts for which there are locks in action which may be removed at some point in the\n future. The value is the block number at which the lock expires and may be removed.\n\n TWOX-NOTE: OK ― `AccountId` is a secure hash." }, { "name": "LastTabledWasExternal", "modifier": "Default", "type": { "plain": "29" }, "fallback": "0x00", "docs": " True if the last referendum tabled was submitted externally. False if it was a public\n proposal." }, { "name": "NextExternal", "modifier": "Optional", "type": { "plain": "154" }, "fallback": "0x00", "docs": " The referendum to be tabled whenever it would be valid to table an external proposal.\n This happens when a referendum needs to be tabled and one of two conditions are met:\n - `LastTabledWasExternal` is `false`; or\n - `PublicProps` is empty." }, { "name": "Blacklist", "modifier": "Optional", "type": { "map": { "hashers": [ "Identity" ], "key": "20", "value": "155" } }, "fallback": "0x00", "docs": " A record of who vetoed what. Maps proposal hash to a possible existent block number\n (until when it may not be resubmitted) and who vetoed it." }, { "name": "Cancellations", "modifier": "Default", "type": { "map": { "hashers": [ "Identity" ], "key": "20", "value": "29" } }, "fallback": "0x00", "docs": " Record of all proposals that have been subject to emergency cancellation." }, { "name": "StorageVersion", "modifier": "Optional", "type": { "plain": "62" }, "fallback": "0x00", "docs": " Storage version of the pallet.\n\n New networks start with last version." } ] }With
adjustMetadataV13=true/pallets/democracy/storage?at=9625125&adjustMetadataV13=true{ "at": { "hash": "0x2813b8eb5fd296bc0869ab3bdf092fdb6c0a9a5e016e1592b94bae699329feaf", "height": "9625125" }, "pallet": "democracy", "palletIndex": "13", "items": [ { "name": "PublicPropCount", "modifier": "Default", "type": { "plain": "PropIndex" }, "fallback": "0x00000000", "docs": " The number of (public) proposals that have been made so far." }, { "name": "PublicProps", "modifier": "Default", "type": { "plain": "Vec<(PropIndex,Hash,AccountId)>" }, "fallback": "0x00", "docs": " The public proposals. Unsorted. The second item is the proposal's hash." }, { "name": "DepositOf", "modifier": "Optional", "type": { "map": { "hasher": "Twox64Concat", "key": "PropIndex", "value": "(Vec<AccountId>,BalanceOf)", "linked": false } }, "fallback": "0x00", "docs": " Those who have locked a deposit.\n\n TWOX-NOTE: Safe, as increasing integer keys are safe." }, { "name": "Preimages", "modifier": "Optional", "type": { "map": { "hasher": "Identity", "key": "Hash", "value": "PreimageStatus", "linked": false } }, "fallback": "0x00", "docs": " Map of hashes to the proposal preimage, along with who registered it and their deposit.\n The block number is the block at which it was deposited." }, { "name": "ReferendumCount", "modifier": "Default", "type": { "plain": "ReferendumIndex" }, "fallback": "0x00000000", "docs": " The next free referendum index, aka the number of referenda started so far." }, { "name": "LowestUnbaked", "modifier": "Default", "type": { "plain": "ReferendumIndex" }, "fallback": "0x00000000", "docs": " The lowest referendum index representing an unbaked referendum. Equal to\n `ReferendumCount` if there isn't a unbaked referendum." }, { "name": "ReferendumInfoOf", "modifier": "Optional", "type": { "map": { "hasher": "Twox64Concat", "key": "ReferendumIndex", "value": "ReferendumInfo", "linked": false } }, "fallback": "0x00", "docs": " Information concerning any given referendum.\n\n TWOX-NOTE: SAFE as indexes are not under an attacker’s control." }, { "name": "VotingOf", "modifier": "Default", "type": { "map": { "hasher": "Twox64Concat", "key": "AccountId", "value": "Voting", "linked": false } }, "fallback": "0x000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "docs": " All votes for a particular voter. We store the balance for the number of votes that we\n have recorded. The second item is the total amount of delegations, that will be added.\n\n TWOX-NOTE: SAFE as `AccountId`s are crypto hashes anyway." }, { "name": "Locks", "modifier": "Optional", "type": { "map": { "hasher": "Twox64Concat", "key": "AccountId", "value": "BlockNumber", "linked": false } }, "fallback": "0x00", "docs": " Accounts for which there are locks in action which may be removed at some point in the\n future. The value is the block number at which the lock expires and may be removed.\n\n TWOX-NOTE: OK ― `AccountId` is a secure hash." }, { "name": "LastTabledWasExternal", "modifier": "Default", "type": { "plain": "bool" }, "fallback": "0x00", "docs": " True if the last referendum tabled was submitted externally. False if it was a public\n proposal." }, { "name": "NextExternal", "modifier": "Optional", "type": { "plain": "(Hash,VoteThreshold)" }, "fallback": "0x00", "docs": " The referendum to be tabled whenever it would be valid to table an external proposal.\n This happens when a referendum needs to be tabled and one of two conditions are met:\n - `LastTabledWasExternal` is `false`; or\n - `PublicProps` is empty." }, { "name": "Blacklist", "modifier": "Optional", "type": { "map": { "hasher": "Identity", "key": "Hash", "value": "(BlockNumber,Vec<AccountId>)", "linked": false } }, "fallback": "0x00", "docs": " A record of who vetoed what. Maps proposal hash to a possible existent block number\n (until when it may not be resubmitted) and who vetoed it." }, { "name": "Cancellations", "modifier": "Default", "type": { "map": { "hasher": "Identity", "key": "Hash", "value": "bool", "linked": false } }, "fallback": "0x00", "docs": " Record of all proposals that have been subject to emergency cancellation." }, { "name": "StorageVersion", "modifier": "Optional", "type": { "plain": "Releases" }, "fallback": "0x00", "docs": " Storage version of the pallet.\n\n New networks start with last version." } ] }