Explicit versioning of PVF execution environment

[pepyakin]

Currently, there is no explicit versioning of the PVF execution environment.

The execution environment (EE) includes (non-exhaustive probably):

• the set of available host functions for the PVF during execution, as well as their behavior.
• the shape of the ValidationParams and ValidationOutputs.
• the validity constraints on those, including time constraints
• the set of available features for wasm. (i.e. extensions like multi-value, multiple memories, simd, etc).

As soon as we release a new version of node with a different EE we will have a window of time, where there is previous and last version of EE is available at the same time. In case there is a candidate that is valid on newer version of EE and invalid on the previous version, it can lead to disputes.

One potential way to avoid that is to introduce explicit versioning of EE. A version of an EE is an integer. This integer is stored on-chain and can be retrieved via runtime API. To simplify matters, we can put this version into the HostConfiguration which would tie changing this version to session boundaries: all candidates produced within the session will have to be validated with EE of the same version.

Whenever the node version with newer EE has been deployed widely, we can bump the EE version on chain. Validators that do not support the new EE version should abstain in backing, approvals and pre-checking. In that case, they will not receive rewards which should incentivise them to update ASAP.

[rphmeier]

Yeah, that's a good idea! The candidate receipt should commit to the EE somehow - PersistedValidationData? If you're a node in session N and you have a candidate to execute from N-K, you'd want a way to check which EE to use

[pepyakin]

Yes, that's one way to do this.

However, PersistedValidationData is primarily used as input to a PVF. Also, PersistedValidationData is unique for each relay-parent¹. It is also stored in the availability data. On the other hand, the execution environment data² stays the same within the session and is shared between parachain.

That observation made me wonder if storing the execution environment SessionInfo is a good idea. This way, the execution environment data does not clutter the availability data and is stored only once on-chain. That is a generally valuable construction that allows us to create a side table with relevant information for the historical candidate validation.

To pull this off, the runtime API that provides the execution environment parametrization needs to know the session index to extract the configuration. An obvious way to solve this problem is to stash the session index at which the candidate was included in the candidate receipt. The candidate receipt already stores the relay parent, so maybe it is already possible to extract the session index?

Footnotes

1. One thing that stands out in PersistedValidationData is max_pov_size. It is only changed with the configuration, thus each session boundary. Maybe a candidate to pull it out? Although cumulus might be using this field. ↩

2. A little aside: I figured the execution environment should be more expressive than just a version number. That has to do with the fact that we have several networks. We may have different parameter changes for various features, similar to a schedule in parity-ethereum. Those features can graduate throughout the networks at a different pace. In other words, we cannot say that e.g., Kusama will have a less version number than rococo. ↩

[s0me0ne-unkn0wn]

It's not the time to close it yet, botty. We've got parametrization but not versioning yet.

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/ux-implications-of-pvf-executor-environment-versioning/2519/1