Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve TryDecodeEntireState output #2724

Merged
merged 10 commits into from
Dec 23, 2023
Merged

Improve TryDecodeEntireState output #2724

Changes from 7 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c57775d
improve decode err output
liamaharon Dec 15, 2023
d1de7ac
? syntax
liamaharon Dec 15, 2023
587c57b
clean up
liamaharon Dec 15, 2023
06df60d
Merge branch 'master' into liam-improve-decode-err-output
liamaharon Dec 18, 2023
2840411
fix missing first key
liamaharon Dec 18, 2023
01dc683
Merge branch 'liam-improve-decode-err-output' of github.com:paritytec…
liamaharon Dec 18, 2023
cd673c5
remove hex and add test
liamaharon Dec 18, 2023
048fe64
improve log
liamaharon Dec 20, 2023
7dae7a0
fix
liamaharon Dec 20, 2023
8d3ceba
Merge branch 'master' into liam-improve-decode-err-output
liamaharon Dec 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
50 changes: 39 additions & 11 deletions substrate/frame/support/src/traits/try_runtime/decode_entire_state.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ impl TryDecodeEntireStorage for Tuple {
}

/// A value could not be decoded.
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Clone, PartialEq, Eq)]
pub struct TryDecodeEntireStorageError {
/// The key of the undecodable value.
pub key: Vec<u8>,
Expand All @@ -81,9 +81,22 @@ impl core::fmt::Display for TryDecodeEntireStorageError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!(
f,
"Failed to decode storage item `{}::{}`",
"`{}::{}` key `{}` is undecodable",
&sp_std::str::from_utf8(&self.info.pallet_name).unwrap_or("<invalid>"),
&sp_std::str::from_utf8(&self.info.storage_name).unwrap_or("<invalid>"),
array_bytes::bytes2hex("0x", &self.key)
)
}
}

impl core::fmt::Debug for TryDecodeEntireStorageError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!(
f,
"key: {} value: {} info: {:?}",
array_bytes::bytes2hex("0x", &self.key),
array_bytes::bytes2hex("0x", self.raw.clone().unwrap_or_default()),
self.info
)
}
}
Expand All @@ -96,37 +109,46 @@ impl core::fmt::Display for TryDecodeEntireStorageError {
fn decode_storage_info<V: Decode>(
info: StorageInfo,
) -> Result<usize, Vec<TryDecodeEntireStorageError>> {
let mut next_key = info.prefix.clone();
let mut decoded = 0;

let decode_key = |key: &[u8]| match sp_io::storage::get(key) {
None => Ok(0),
Some(bytes) => {
let len = bytes.len();
let _ = <V as DecodeAll>::decode_all(&mut bytes.as_ref()).map_err(|_| {
vec![TryDecodeEntireStorageError {
TryDecodeEntireStorageError {
key: key.to_vec(),
raw: Some(bytes.to_vec()),
info: info.clone(),
}]
}
})?;

Ok::<usize, Vec<_>>(len)
Ok::<usize, _>(len)
},
};

decoded += decode_key(&next_key)?;
liamaharon marked this conversation as resolved.
Show resolved Hide resolved
let mut errors = vec![];
let mut next_key = Some(info.prefix.clone());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should just use PrefixKeyIterator here. Not sure why we have written this here "manually" :P

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disadvantage of KeyPrefixIterator is that it "skips" keys that error decoding. We want to handle the Result instead.

loop {
match sp_io::storage::next_key(&next_key) {
match next_key {
Some(key) if key.starts_with(&info.prefix) => {
decoded += decode_key(&key)?;
next_key = key;
match decode_key(&key) {
Ok(bytes) => {
decoded += bytes;
},
Err(e) => errors.push(e),
};
next_key = sp_io::storage::next_key(&key);
},
_ => break,
}
}

Ok(decoded)
if errors.is_empty() {
Ok(decoded)
} else {
Err(errors)
}
}

impl<Prefix, Value, QueryKind, OnEmpty> TryDecodeEntireStorage
Expand Down Expand Up @@ -353,6 +375,12 @@ mod tests {
// two bytes, cannot be decoded into u32.
sp_io::storage::set(&Map::hashed_key_for(2), &[0u8, 1]);
assert!(Map::try_decode_entire_state().is_err());
assert_eq!(Map::try_decode_entire_state().unwrap_err().len(), 1);

// multiple errs in the same map are be detected
sp_io::storage::set(&Map::hashed_key_for(3), &[0u8, 1]);
assert!(Map::try_decode_entire_state().is_err());
assert_eq!(Map::try_decode_entire_state().unwrap_err().len(), 2);
})
}

Expand Down