PVF should accept data based on preimages

[rphmeier]

A common issue is that parachains need to include data which is large and takes up PoV space. We can use the Preimage pallet to have validators load this data from the relay chain state and pass it into the PVF instead. An initial use-case is smart contract code. PVF upgrades may also be optimized with this method cc #971

Summary of changes:

• CandidateReceipts should have a data_dependencies: Vec<Hash> which are the hashes of all the data that should be passed into the PVF
• The relay-chain runtime should reject candidates which either have too many (governance configurable) hashes or candidates which have any hashes where the preimage pallet doesn't have the preimage for.
• Preimages should last for T (~24 hours) after being 'removed'. During this 'removal' phase they cannot be referenced by candidate receipts
• The PVF should accept a parameter Vec<(Hash, Vec<u8>)> which are all the preimages for the candidate receipt.
• ValidateFromExhaustive should accept a HashMap<Hash, Vec<u8>> for candidate execution
• Validators, before executing a block (in backing, approval, or disputes), should pull the preimages from the state with a runtime API called at any recent relay-chain block. If any preimages in a candidate are unavailable, then the block is invalid

We don't want to allow too many preimages because it adds runtime, network, and execution overhead for every candidate because of the added checks and candidate receipt size.

This also allows data (e.g. smart contract code) to be reused across parachains in a way that doesn't add to the PoV size. Collator logic and parachain runtimes can be customized to detect data at block authoring time which is available through the preimages pallet and opt to include it that way as opposed to the PoV state. Users will need to pay to store data on the relay-chain.

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/altering-polkadots-fork-choice-to-reduce-da-load/3389/4

[burdges]

An example of a pre-image would be a block meant to be committed to now, but executed somewhat later?

[shawntabrizi]

@burdges I think common preimages would be, for example, popular smart contracts.

Or any other data which stays mostly the same, but is read often. I could imagine multisigs would be another candidate for example, although impact to DA may be small in this example.

But I do think it would be cool if, for an on-demand parachain, they can just upload the whole block beforehand, as a part of the collation, and then it would not involve the DA limits at all. I could imagine such a block could be many times the maximum block size, since multiple transactions could provide parts of the final block which is eventually concatenated together.

Stuff like that.