Accessing the raw unvalidated blocks?

[wilwade]

I recently worked on a tool (PAPI, TypeScript based) that was investigating some network issues. At first I used smoldot, but I was unable to locate a way to gain access to the unvalidated blocks.

I could see them when logging, and even thought about trying to scan the log for the information, but the logger doesn't have the complete hash, so that wasn't an option either.

Is there perhaps another way? If not, is this a feature that would be welcome for others?

[josepot]

I recently worked on a tool (PAPI, TypeScript based) that was investigating some network issues. At first I used smoldot, but I was unable to locate a way to gain access to the unvalidated blocks.

I'm not sure what it is that you mean by "unvalidated blocks". However, if this means that you just want to get all the new blocks that are being observed by the client (regardless of whether they are finalized, best or whatever), then you can get those from the blocks$ observable that the client exposes, eg:

import { CompatibilityLevel, createClient } from "polkadot-api"
import { getSmProvider } from "polkadot-api/sm-provider"
import { start } from "polkadot-api/smoldot"
import { chainSpec } from "polkadot-api/chains/polkadot"

const smoldot = start()
const client = createClient(getSmProvider(smoldot.addChain({ chainSpec })))

client.blocks$.subscribe((block) => {
  console.log(block.hash)
})

If by "unvalidated blocks" you refer to blocks that have not been validated to be part of the chain... Then, I'm afraid that there is no way to have access to those blocks, and IMO it doesn't make a whole lot of sense for smoldot to expose them...

[wilwade]

I tried using blocks$, but didn't receive any of the forks and the only blocks that came through were 3-4 blocks behind what an rpc node was showing. It was as if the smoldot was only passing through finalized blocks.

This was a parachain if it matters. Perhaps it is different for relay chains vs parachains?

[josepot]

I just used the PAPI console to check this, and yeah... There seems to be something off, here. I put side by side the same explorer on Polkadot AssetHub, one was using smoldot, and the other one was using an IBP RPC. These are some screenshots, on the left smoldot, on the right the RPC ws:

For some reason smoldot is always one block behind, and when there is a block with a lot of forks (like block 9,481,229) then smoldot takes a long time before reporting anything and when it reports it is b/c it has already received another block on top, like if it was waiting for a block to figure-out which one is going to be the best block or something...

I mean, I have seen smoldot reporting forks in the past, for sure! However, it seems like it has some sort of logic to delay reporting blocks until it has figured out which one is the best candidate to be the best-block...

IMO this is worth opening an issue, b/c I would expect to always report all the known blocks over the chainHead subscription.

[tomaka]

If you pass false to chainHead_follow, then smoldot should report blocks as soon as it is receives a block announce. There shouldn't be any delay.
If you pass true, then it first does an additional request to check whether the runtime has changed in the block. This adds a small delay, but it shouldn't be more than a few hundred milliseconds.

[josepot]

If you pass true, then it first does an additional request to check whether the runtime has changed in the block

I must be missing something here, but I was under the impression that the additional request was only be necessary whenever smoldot has determined (by looking into the header digest) that there is, indeed, a runtime upgrade, in order to obtain the details of the new runtime... I.e if a digest-item of type 8 is found.

Anyhow, this doesn't seem to explain the amount of missing blocks reported and/or the fact that with smoldot the tip of the chainHead subscription is always one block behind. So, I think that there must be a bug here. I will gather some logs and I will open an issue providing more context.

[tomaka]

I must be missing something here, but I was under the impression that the additional request was only be necessary whenever smoldot has determined (by looking into the header digest) that there is, indeed, a runtime upgrade, in order to obtain the details of the new runtime... I.e if a digest-item of type 8 is found.

Ah yes, that's correct, I forgot about this.