Extrinsic Horizon

[gavofyork]

Goals

• Clean up the mess of types and impractical trait architecture used in transactions.
• Ensure that it is easy to introduce and integrate new transaction crypto.
• Facilitate use-cases which depend on altering the Origin prior to fee payment including proxied-pays-fee and multisig-pays-fee (eg. Allow transaction payment from proxy target account #348).
• Unify ValidateUnsigned and SignedExtension (fixes Switch ValidateUnsigned over to SignedExtension #365).

Explanation

Stage 1: TransactionExtension

Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have General for the former and Bare for the latter.

Types of extrinsic are now therefore:

• Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned")
  • Bare transactions (DEPRECATED): Gossiped, validated with ValidateUnsigned (DEPRECATED) and the _unsigned bits of TransactionExtension (DEPRECATED).
  • Inherents: Not gossiped, validated with ProvideInherent.
• Extended (with a TxExtension (ne Extra) data): Gossiped, validated via TransactionExtension type.
  • Signed transactions (with a hardcoded signature).
  • General transactions (without a hardcoded signature).

Three extrinsic version discriminators ("versions") are now permissible:

• 0b000000100: Bare (used to be called "Unsigned"): contains neither a Signature nor Extension (i.e. Extra data). After Bare transactions are no longer supported, this will strictly identify Inherents only and will be renamed as such.
• 0b100000100: Old-school "Signed" Transaction: contains Signature and Extension (Extra data).
• 0b010000100: New-school "General" Transaction: contains Extension (Extra data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above.

• FRAME: Create TransactionExtension as a replacement for SignedExtension #2280

Stage 2: Begin removal of concept of Unsigned transactions:

• Deprecate ValidateUnsigned & #[validate_unsigned] (logic should be moved to TransactionExtension).
• Remove (if unused) or migrate uses of validate_unsigned:
  • Frontier: migrate.
  • Claims: remove (in favour of feeless_if).
• Stop using Extrinsic as much as possible internally, just use concrete types in tests.

Stage 2.5: Integrated semantic description

Transactions should include a metadata extension VerifyMetadata

• Extension places optional hash in the signed payload which corresponds to the hash of the metadata for the chain.
• Online signers simply hash the metadata and place it in the transaction.
• Offline signers should generally be provided with witness data to be able to construct a sparse Merkle tree of the metadata and display the transactions meaning to the user at the time of signing.

Stage 3 PR: New APIs, deprecate old-school.

All extrinsics are now validated either by ProvideInherent (which is an aggregated type of the runtime) or TransactionExtension (which is an explicitly configured type of the runtime).

Bare transactions are removed, so:

• "Extended transactions" are synonymous with "transactions" and we drop the "Extended" nomenclature.
• Similarly, "Bare extrinsics" will be synonymous with "Inherents" and we will drop the "Bare" nomenclature.

This leaves us with three types/versions of Extrinsics:

• 0b000000100: Inherent: contains neither a Signature nor Extension (i.e. Extra data). Validated by ProvideInherent.
• 0b100000100: Old-school "Signed" Transaction: contains Signature and Extension.
• 0b010000100: New-school "General" Transaction: contains Extension, but no Signature.

Action items:

Remove deprecated items:

• Remove ValidateUnsigned and #[validate_unsigned]; Bare extrinsics are always Inherents and therefore always fail validation/pre-dispatch in UncheckedExtrinsic.

Move test code to regular types:

• Remove old test transaction types and just use the regular CheckedExtrinsic and UncheckedExtrinsic.
• Remove ExtrinsicWrapper and use the regular CheckedExtrinsic and UncheckedExtrinsic.

Remove all needless reliance on traits with a defunct model of transactions:

• Stop using and deprecate old items in Extrinsic trait:
  • Extrinsic::is_signed, Extrinsic::new, Extrinsic::SignaturePayload.
• Deprecate trait SignaturePayload, trait CreateSignedTransaction.
• Introduce new CreateTransaction traits, simplifying transaction creation for tests & OCW.
  • fn createTransaction(from: Signer, call: Call) -> UncheckedExtrinsic;
    • called inside a state-queryable environment (so pallet storage can be gotten).
  • Implementing this is the requirement of the runtime, but can be mostly automated with utility structs.

Tidy up language:

• Rename Signed to Account and ensure_signed to ensure_account.

Stage 4: Remove rest of deprecated API

• Remove SignaturePayload, CreateSignedTransaction and deprecated items in Extrinsic.

Other notes:

• This isn't really what we mean - we really want a tx.is_transaction, since bare transactions may be gossipped as in the case of Frontier txs or claims. This will be sorted once we dispense with the concept of bare transactions and make inherents the only possible type of extrinsics which are bare. At this point we can change this to tx.is_transaction(). - substrate/client/transaction-pool/src/lib.rs

[bkchr]

Transactions should include a metadata extension VerifyMetadata

Wouldn't this also not just be a TransactionExtension?

[ggwpez]

Remove or migrate uses of validate_unsigned:
Frontier: migrate.
Claims: remove.

What is the criteria for migration vs removal in the other pallets that use it?

[gavofyork]

If the feature is out of use (as I expected claims was), then they can be deprecated and removed. But since claim was actually used only 5 days ago last, I guess we ought to keep it around 😕.

[bkchr]

We should also change SignedPayload to just always hash the data that we sign/verify. I don't really see the advantage of doing it this way.

[bkchr]

	fn validate(
		&self,
		origin: OriginOf<Call>,
		call: &Call,
		info: &DispatchInfoOf<Call>,
		len: usize,
		context: &mut Context,
		self_implicit: Self::Implicit,
		inherited_implication: &impl Encode,
	) -> ValidateResult<Self::Val, Call>;

	pub type ValidateResult<Val, Call> =
		Result<(ValidTransaction, Val, OriginOf<Call>), TransactionValidityError>;

Instead of this, I think it would be better to change the interface to:

	fn validate(
		&self,
		origin: Option<OriginOf<Call>>,
                signer: Option<Account>,
		call: &Call,
		info: &DispatchInfoOf<Call>,
		len: usize,
		context: &mut Context,
		self_implicit: Self::Implicit,
		inherited_implication: &impl Encode,
	) -> ValidateResult<Self::Val, Call>;

	pub type ValidateResult<Val, Call> =
		Result<(ValidTransaction, Val, Option<OriginOf<Call>>), TransactionValidityError>;

So, basically we would make the origin an Option and go back to passing the signer. This would bring us the advantage of chaining logic, but first let's look at the current structure. If I want to use the SkipFeelessIf with the TransactionPayment extension, I need to nest them. Aka SkipFeelessIf<TransactionPayment>. Now let's assume we actually want to have General transactions that are created by an offchain worker for example. The problem would be that we need to skip at least TransactionPayment. So, we would need to nest again CustomExt<SkipFeelessIf<TransactionPayment>>>. (We can probably continue this for multiple different extensions).

IMO it would be better to do this like:

Exts = (
     CustomExt,
     SkipFeelessIf,
     TransactionPayment,
);

At the beginning origin would be None. The extensions can set the origin to Some, which would signal the following extensions like TransactionPayment to not try to deduct any fees. If at the end of executing all extensions, origin would still be None, we would reject the transaction. This way we would also get back the old behavior of rejecting unsigned/general transactions "by default".

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/2024-04-23-technical-fellowship-opendev-call/7592/1