diff --git a/text/0017-tangle-message/0017-tangle-message.md b/text/0017-tangle-message/0017-tangle-message.md index 6a048792f..44dfd8b74 100644 --- a/text/0017-tangle-message/0017-tangle-message.md +++ b/text/0017-tangle-message/0017-tangle-message.md @@ -141,11 +141,11 @@ The following criteria defines whether the message passes the syntactical valida While messages without a payload, i.e. `Payload Length` set to zero, are valid, such messages do not contain any information. As such, messages usually contain a payload. The detailed specification of each payload type is out of scope of this RFC. The following table lists all currently specified payloads that can be part of a message and links to their specification. The _indexation payload_ will be specified here as an example: -| Payload Name | Type Value | RFC | -| ------------ | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Transaction | 0 | [RFC-0018 (draft)](https://github.com/luca-moser/protocol-rfcs/blob/signed-tx-payload/text/0000-signed-transaction-payload/0000-signed-transaction-payload.md) | -| Milestone | 1 | [RFC-0019](https://iotaledger.github.io/protocol-rfcs/0019-milestone-payload/0019-milestone-payload.html) | -| Indexation | 2 | [RFC-0017](#indexation-payload) | +| Payload Name | Type Value | RFC | +| ------------ | ---------- | ------------------------------------------------------------------------------------------------------------- | +| Transaction | 0 | [RFC-0018](https://iotaledger.github.io/protocol-rfcs/0018-transaction-payload/0018-transaction-payload.html) | +| Milestone | 1 | [RFC-0019](https://iotaledger.github.io/protocol-rfcs/0019-milestone-payload/0019-milestone-payload.html) | +| Indexation | 2 | [RFC-0017](#indexation-payload) | ### Indexation payload diff --git a/text/0018-transaction-payload/0018-transaction-payload.md b/text/0018-transaction-payload/0018-transaction-payload.md new file mode 100644 index 000000000..f3d2d19c4 --- /dev/null +++ b/text/0018-transaction-payload/0018-transaction-payload.md @@ -0,0 +1,543 @@ ++ Feature name: `transaction-payload` ++ Start date: 2020-07-10 ++ RFC PR: [iotaledger/protocol-rfcs#18](https://github.com/iotaledger/protocol-rfcs/pull/18) + +# Summary + +In the current IOTA protocol, transactions are grouped into so-called bundles to assure that they can only be confirmed as one unit. This RFC proposes a new UTXO-based transaction structure containing all the inputs and outputs of a transfer. Specifically, this RFC defines a transaction payload for the _messages_ described in the IOTA protocol [RFC-0017](https://iotaledger.github.io/protocol-rfcs/0017-tangle-message/0017-tangle-message.html). + +# Motivation + +Currently, the vertices of the Tangle are represented by transactions, where each transaction defines either an input or output. A grouping of those input/output transaction vertices makes up a bundle which transfers the given values as an atomic unit (the entire bundle is applied or none of it). An applied bundle consumes the input transactions' funds and creates the corresponding deposits into the output transactions' target addresses. Furthermore, additional meta transactions can be part of the bundle to carry parts of the signature which do not fit into a single input transaction. + +The bundle concept has proven to be very challenging in practice because of the following issues: +* Since the data making up the bundle is split across multiple vertices, it complicates the validation of the entire transfer. Instead of being able to immediately tell whether a bundle is valid or not, a node implementation must first collect all parts of the bundle before any actual validation can happen. This increases the complexity of the node implementation. +* Reattaching the tail transaction of a bundle causes the entire transfer to be reapplied. +* Due to the split across multiple transaction vertices and having to do PoW for each of them, a bundle might already be lazy in terms of where it attaches, reducing its chances to be confirmed. + +To fix the problems mentioned above and to create a more flexible transaction structure, the goal is to achieve a self-contained transaction structure defining the data of the entire transfer as a payload to be embedded into a message. + +The new transaction structure should fulfil the following criteria: +* Support for Ed25519 (and thus reusable addresses). +* Support for adding new types of signature schemes, addresses, inputs, and outputs as part of protocol upgrades. +* Self-contained, as in being able to validate the transaction immediately after receiving it. +* Enable unspent transaction outputs (UTXO) as inputs instead of an account based model. + +# Detailed design + +## UTXO + +The *unspent transaction output* (UTXO) model defines a ledger state where balances are not directly associated to addresses but to the outputs of transactions. In this model, transactions reference outputs of previous transactions as inputs, which are consumed (removed) to create new outputs. A transaction must consume all the funds of the referenced inputs. + +Using a UTXO based model provides several benefits: +* Parallel validation of transactions. +* Easier double-spend detection, since conflicting transactions would reference the same UTXO. +* Replay-protection which is important when having reusable addresses. Replaying the same transaction would manifest itself as already being applied or existent and thus not have any impact. +* Technically seen, balances are no longer associated to addresses which raises the level of abstraction and thus enables other types of outputs with particular unlock criteria. + +Within a transaction using UTXOs, inputs and outputs make up the to-be-signed data of the transaction. The section unlocking the inputs is called the *unlock block*. An unlock block may contain a signature proving ownership of a given input's address and/or other unlock criteria. + +The following image depicts the flow of funds using UTXO: + +![UTXO flow](img/utxo.png) + +## Structure + +### Serialized layout + +A _Transaction Payload_ is made up of two parts: +1. The _Transaction Essence_ part which contains the inputs, outputs and an optional embedded payload. +2. The _Unlock Blocks_ which unlock the inputs of the _Transaction Essence_. When an unlock block contains a signature, it signs the entire _Transaction Essence_ part. + +All values are serialized in little-endian encoding. The serialized form of the transaction is deterministic, meaning the same logical transaction always results in the same serialized byte sequence. + +The *Transaction ID* is the [BLAKE2b-256](https://tools.ietf.org/html/rfc7693) hash of the entire serialized payload data including signatures. + +The following table structure describes the entirety of a _Transaction Payload_ in its serialized form: +* Data Type Notation, see [RFC-0017](https://iotaledger.github.io/protocol-rfcs/0017-tangle-message/0017-tangle-message.html#data-types) +*
+ Subschema Notation + + + + + + + + + + + + + + + + + +
NameDescription
oneOfOne of the listed subschemas.
optOneOfOptionally one of the listed subschemas.
anyOfAny (one or more) of the listed subschemas.
+
+ +

+ + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Payload Typeuint32 + Set to value 0 to denote a Transaction Payload. +
Essence oneOf +
+ Transaction Essence +
+ Describes the essence data making up a transaction by defining its inputs, outputs and an optional payload. +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Transaction Typeuint8 + Set to value 0 to denote a Transaction Essence. +
Inputs Countuint16The number of input entries.
Inputs anyOf +
+ UTXO Input +
+ Describes an input which references an unspent transaction output to consume. +
+ + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Input Typeuint8 + Set to value 0 to denote an UTXO Input. +
Transaction IDByteArray[32]The BLAKE2b-256 hash of the transaction payload containing the referenced output.
Transaction Output Indexuint16The output index of the referenced output.
+
+
Outputs Countuint16The number of output entries.
Outputs anyOf +
+ SigLockedSingleOutput +
+ Describes a deposit to a single address which is unlocked via a signature. +
+ + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Output Typeuint8 + Set to value 0 to denote a SigLockedSingleOutput. +
Address oneOf +
+ Ed25519 Address + + + + + + + + + + + + + + + + +
NameTypeDescription
Address Typeuint8 + Set to value 0 to denote an Ed25519 Address. +
AddressByteArray[32]The raw bytes of the Ed25519 address which is the BLAKE2b-256 hash of the public key.
+
+
Amountuint64The amount of tokens to deposit.
+
+
+ SigLockedDustAllowanceOutput +
+ Describes a deposit which as a special property also alters the dust allowance of the target address. +
+ + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Output Typeuint8 + Set to value 1 to denote a SigLockedDustAllowanceOutput. +
Address oneOf +
+ Ed25519 Address + + + + + + + + + + + + + + + + +
NameTypeDescription
Address Typeuint8 + Set to value 0 to denote an Ed25519 Address. +
AddressByteArray[32]The raw bytes of the Ed25519 address which is the BLAKE2b-256 hash of the public key.
+
+
Amountuint64The amount of tokens to deposit.
+
+
Payload Lengthuint32The length in bytes of the optional payload.
Payload optOneOf +
+ Generic Payload +
+ An outline of a generic payload. +
+ + + + + + + + + + + + + + + + +
NameTypeDescription
Payload Typeuint32 + The type of the payload. It will instruct the node how to parse the fields that follow. +
Data FieldsANYA sequence of fields, where the structure depends on Payload Type.
+
+
+
+
Unlock Blocks Countuint16The number of unlock block entries. It must match the field Inputs Count.
Unlock Blocks anyOf +
+ Signature Unlock Block +
+ Defines an unlock block containing a signature. +
+ + + + + + + + + + + + + + + +
NameTypeDescription
Unlock Typeuint8 + Set to value 0 to denote a Signature Unlock Block. +
Signature oneOf +
+ Ed25519 Signature + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescription
Signature Typeuint8 + Set to value 0 to denote an Ed25519 Signature. +
Public keyByteArray[32]The Ed25519 public key of the signature.
SignatureByteArray[64]The Ed25519 signature signing the Blake2b-256 hash of the serialized Transaction Essence.
+
+
+
+
+ Reference Unlock Block +
+ References a previous unlock block, where the same unlock block can be used for multiple inputs. +
+ + + + + + + + + + + + + + + + +
NameTypeDescription
Unlock Typeuint8 + Set to value 1 to denote a Reference Unlock Block. +
Referenceuint16Represents the index of a previous unlock block.
+
+
+ +### Transaction parts + +In general, all parts of a Transaction Payload begin with a byte describing the type of the given part. This improves the flexibility to introduce new types/versions of the given part in the future. + +#### Transaction Essence data + +The Transaction Essence of a Transaction Payload carries the inputs, outputs, and an optional payload. The Transaction Essence is an explicit type and therefore starts with its own Transaction Essence Type byte which is of value 0. + +##### Inputs + +The Inputs part holds the inputs to consume in order to fund the outputs of the Transaction Payload. Currently, there is only one type of input, the UTXO Input. In the future, more types of inputs may be specified as part of protocol upgrades. + +Each input must be accompanied by a corresponding Unlock Block at the same index in the Unlock Blocks part of the Transaction Payload. + +###### UTXO Input + +A UTXO Input is an input which references an unspent output of a previous transaction. This UTXO is uniquely defined by the _Transaction ID_ of that transaction together with corresponding output index. Each UTXO Input must be accompanied by an Unlock Block that is allowed to unlock the output the UTXO Input is referencing. + +Example: +If the input references an output to an Ed25519 address, then the corresponding unlock block must be of type Signature Unlock Block holding an Ed25519 signature. + +##### Outputs + +The Outputs part holds the outputs that are created by this Transaction Payload. The following output types are supported: + +###### SigLockedSingleOutput + +The SigLockedSingleOutput defines an output (with a certain amount) to a single target address which is unlocked via a signature proving ownership over the given address. This output supports addresses of different types. + +###### SigLockedDustAllowanceOutput + +The SigLockedDustAllowanceOutput works in the same way as a SigLockedSingleOutput but additionally controls the dust allowance on the target address. See [Dust Protection RFC-0032 (draft)](https://github.com/iotaledger/protocol-rfcs/pull/32) for further information. + +##### Payload + +The _Transaction Essence_ itself can contain another payload as described in general in [RFC-0017](https://iotaledger.github.io/protocol-rfcs/0017-tangle-message/0017-tangle-message.html). The [semantic validity](#semantic-validation) of the encapsulating _Transaction Payload_ does not have any impact on the payload. + +The following table lists all the payload types that can be nested inside a _Transaction Essence_ as well as links to the corresponding specification: + +| Name | Type Value | RFC | +| ---------- | ---------- | ---------------------------------------------------------------------------------------------------------------------- | +| Indexation | 2 | [RFC-0017](https://iotaledger.github.io/protocol-rfcs/0017-tangle-message/0017-tangle-message.html#indexation-payload) | + +#### Unlock Blocks + +The Unlock Blocks part holds the unlock blocks unlocking inputs within a Transaction Essence. The following types of unlock blocks are supported: + +##### Signature Unlock Block + +A Signature Unlock Block defines an Unlock Block which holds a signature signing the BLAKE2b-256 hash of the Transaction Essence (including the optional payload). + +##### Reference Unlock block + +A Reference Unlock Block defines an Unlock Block which references a previous Unlock Block (which must not be another Reference Unlock Block). It **must** be used if multiple inputs can be unlocked via the same Unlock Block. + +Example: +Consider a Transaction Essence containing the UTXO Inputs 0, 1 and 2, where 0 and 2 are both spending outputs belonging to the same Ed25519 address A and 1 is spending from a different address B. This results in the following structure of the Unlock Blocks part: + +| Index | Unlock Block | +| ----- | ---------------------------------------------------------------------------------------- | +| 0 | A _Signature Unlock Block_ holding the Ed25519 signature for address A. | +| 1 | A _Signature Unlock Block_ holding the Ed25519 signature for address B. | +| 2 | A _Reference Unlock Block_ which references 0, as both require the same signature for A. | + +## Validation + +A Transaction Payload has different validation stages, since some validation steps can only be executed when certain information has (or has not) been received. We therefore distinguish between syntactic and semantic validation: + +### Syntactic validation + +Syntactic validation is checked as soon as the transaction data has been received in its entirety. It validates the structure but not the signatures of the transaction. If the transaction does not pass this stage, it must not be broadcasted further and can be discarded right away. + +The following criteria defines whether a payload passes the syntactical validation: +* Essence: + * `Transaction Type` value must denote a _Transaction Essence_. + * Inputs: + * `Inputs Count` must be 0 < x ≤ 127. + * For each input the following must be true: + * `Input Type` must denote a _UTXO Input_. + * `Transaction Output Index` must be 0 ≤ x < 127. + * `Inputs` must be sorted in lexicographical order of their serialized form.1 + * Each pair of `Transaction ID` and `Transaction Output Index` must be unique in the inputs set. + * Outputs: + * `Outputs Count` must be 0 < x ≤ 127. + * For each input the following must be true: + * `Output Type` must denote a _SigLockedSingleOutput_ or a _SigLockedDustAllowanceOutput_. + * `Address Type` must denote an _Ed25519 Address_. + * `Amount` must be larger than zero. + * `Outputs` must be sorted in lexicographical order of their serialized form.1 + * Each `Address` must be unique per output type. For example, a _SigLockedSingleOutput_ and a _SigLockedDustAllowanceOutput_ can have the same address, but not two _SigLockedSingleOutputs_. + * The sum of all `Amount` fields must not exceed the total IOTA supply of 2,779,530,283,277,761. + * Payload (if present): + * `Payload Type` must match one of the values described under [Payload](#payload). + * `Data fields` must be correctly parsable in the context of the `Payload Type`. + * The payload itself must pass syntactic validation. +* Unlock Blocks: + * `Unlock Blocks Count` must match `Inputs Count` of the _Transaction Essence_. + * Each `Unlock Type` must denote a _Signature Unlock Block_ or a _Reference Unlock Block_. + * Each _Signature Unlock Block_ must contain an _Ed25519 Signature_. + * Each _Signature Unlock Block_ must be unique. + * A _Reference Unlock Block_ at index i must have `Reference` < i and the unlock block at index `Reference` must be a _Signature Unlock Block_. +* Given the type and length information, the _Transaction Payload_ must consume the entire byte array of the `Payload` field of the encapsulating object. + +1 ensures that serialization of the transaction becomes deterministic, meaning that libraries always produce the same bytes given the logical transaction. + +### Semantic validation + +The Semantic validation of a _Transaction Payload_ is performed when its encapsulating message is confirmed by a milestone. The semantic validity of transactions depends on the order in which they are processed. Thus, it is necessary that all the nodes in the network perform the checks in the same order, no matter the order in which the transactions are received. This is assured by using the White-Flag ordering as described in [RFC-005](https://iotaledger.github.io/protocol-rfcs/0005-white-flag/0005-white-flag.html#deterministically-ordering-the-tangle). + +Processing transactions according to the White-Flag ordering enables users to spend UTXOs which are created in the same milestone confirmation cone, as long as the spending transaction comes after the funding transaction in the aforementioned White-Flag order. In this case, it is recommended that users include the _Message ID_ of the funding transaction as a parent of the message containing the spending transaction. + +The following criteria defines whether a payload passes the semantic validation: +* Each input must reference a valid UTXO, i.e. the output referenced by the input's `Transaction ID` and `Transaction Output Index` is known (booked) and unspent. +* The transaction must spend the entire balance, i.e. the sum of the `Amount` fields of all the UTXOs referenced by inputs must match the sum of the `Amount` fields of all outputs. +* Each unlock block must be valid with respect to the UTXO referenced by the input of the same index: + * If it is a _Signature Unlock Block_: + * The `Signature Type` must match the `Address Type` of the UTXO, + * the BLAKE2b-256 hash of `Public Key` must match the `Address` of the UTXO and + * the `Signature` field must contain a valid signature for `Public Key`. + * If it is a _Reference Unlock Block_, the referenced _Signature Unlock Block_ must be valid with respect to the UTXO. + +If a _Transaction Payload_ passes the semantic validation, its referenced UTXOs must be marked as spent and its new outputs must be created/booked in the ledger. The _Message ID_ of the message encapsulating the processed payload then also becomes part of the input for the White-Flag Merkle tree hash of the confirming milestone ([RFC-0012](https://iotaledger.github.io/protocol-rfcs/0012-milestone-merkle-validation/0012-milestone-merkle-validation.html)). + +Transactions that do not pass semantic validation are ignored. Their UTXOs are not marked as spent and their outputs are not booked in the ledger. + +## Miscellaneous + +### Transaction timestamps + +Since transaction timestamps – whether they are signed or not – do not provide any guarantee of correctness, they have been left out of the _Transaction Payload_. Applications relying on some notion of time for transactions can use the local solidification time or the global timestamp of the confirming milestone ([RFC-0019](https://iotaledger.github.io/protocol-rfcs/0019-milestone-payload/0019-milestone-payload.html)). + +### Address reuse + +While, in contrast to Winternitz one-time signatures (W-OTS), producing multiple Ed25519 signatures for the same private key and address does not decrease its security, it still drastically reduces the privacy of users. It is thus considered best practice that applications and services create a new address per deposit to circumvent these privacy issues. + +In essence, Ed25519 support allows for smaller transaction sizes and to safely spend funds which were sent to an already used deposit address. Ed25519 addresses are not meant to be used like email addresses. See this [Bitcoin wiki article](https://en.bitcoin.it/wiki/Address_reuse) for further information. + +# Drawbacks + +* The new transaction format is the core data type within the IOTA ecosystem. Changing it means that all projects need to accommodate it, including wallets, web services, client libraries and applications using IOTA in general. It is not possible to keep these changes backwards compatible, meaning that all nodes must upgrade to further participate in the network. +* Additionally, local snapshots can no longer be represented by a list of addresses and their balances, since the ledger is now made up of the UTXOs on which the actual funds reside. Therefore, local snapshot file schemes have to be adjusted to incorporate the transaction hashes, output indices, and then the destination addresses including the balances. + +# Rationale and alternatives + +* Introducing this new transaction structure allows for extensions in the future, to accommodate new requirements. With the support for Ed25519 addresses/signatures, transaction size is drastically reduced and allows for safe re-signing in case of address reuse. Due to the switch to a complete binary transaction, the transaction size is reduced even further, saving network bandwidth and processing time. +* Other transaction structures have been considered but they would have misused existing transaction fields to accommodate for new features, instead of putting them into a proper descriptive structure. Additionally, those ideas would not have been safe against replay attacks, which deems reusing the old transaction structure, for example for Ed25519 addresses/signatures, as infeasible. +* Not switching to the new transaction structure described in this RFC would have led to more people losing funds because of W-OTS address reuse and it would prevent extending the IOTA protocol further down the line. diff --git a/text/0018-transaction-payload/img/utxo.png b/text/0018-transaction-payload/img/utxo.png new file mode 100644 index 000000000..b6f9f1988 Binary files /dev/null and b/text/0018-transaction-payload/img/utxo.png differ