From 2be08dd9ec88e8c9ea917a95aa49283f44c031ec Mon Sep 17 00:00:00 2001 From: Muhammet Kara Date: Mon, 21 Jul 2025 17:29:27 +0300 Subject: [PATCH] Reapply "New fields; added more detailed explanation for variant discriminator; replaced "enum" nomenclature to "union-type" in "calltable" serialization docs; Added a comprehensive example of both serializing and deserializing Transaction::Version1" This reverts commit 260acdf773e31f2225018a6f2a7792203521a407. --- .../serialization/calltable-serialization.md | 33 +- .../transaction-deserialization-example.md | 386 ++++++++++++++++++ .../transaction-serialization-example.md | 315 ++++++++++++++ .../concepts/serialization/transaction.md | 48 +-- .../concepts/serialization/types.md | 8 + 5 files changed, 751 insertions(+), 39 deletions(-) create mode 100644 versioned_docs/version-2.0.0/concepts/serialization/transaction-deserialization-example.md create mode 100644 versioned_docs/version-2.0.0/concepts/serialization/transaction-serialization-example.md diff --git a/versioned_docs/version-2.0.0/concepts/serialization/calltable-serialization.md b/versioned_docs/version-2.0.0/concepts/serialization/calltable-serialization.md index 2ccf167bb..d65d84667 100644 --- a/versioned_docs/version-2.0.0/concepts/serialization/calltable-serialization.md +++ b/versioned_docs/version-2.0.0/concepts/serialization/calltable-serialization.md @@ -9,7 +9,7 @@ As mentioned before, the `calltable serialization approach` requires serializing - `fields` - field which is an ordered collection of [Field](#field). This collection is serialized as a Vec<Field> in the "regular" `Binary Serialization Standard` schema. The invariants for `fields` are: - for each fi, fj given `i` < `j` implies fi.index < fj.index - for each fi, fj given `i` < `j` implies fi.offset < fj.offset -- `bytes` - field which is an amorphous blob of bytes serialized as `Bytes` in the "regular" `Binary Serialization Standard` schema. +- `bytes` - field which is an amorphous blob of bytes serialized as `Bytes` in the "regular" `Binary Serialization Standard` schema. Bear in mind that this field will be serialized by appending first a `u32` integer (number of bytes) than the raw bytes payload. A `Field` consists of: @@ -79,6 +79,9 @@ Once we have the `fields` and `bytes` we can proceed to serialize those in the " for i in range(0, len(fields)): bytes_arr += fields[i]["index"].to_bytes(2, byteorder = 'little') bytes_arr += fields[i]["offset"].to_bytes(4, byteorder = 'little') + concatenated_serialized_fields_len = len(concatenated_serialized_fields) + print(f"{concatenated_serialized_fields_len}") + bytes_arr += concatenated_serialized_fields_len.to_bytes(4, byteorder = 'little') bytes_arr += concatenated_serialized_fields return bytes_arr ``` @@ -88,7 +91,7 @@ and once we have all that we can apply the scripts to example: ```python I = [0, 1, 3, 5] B = [bytes([0, 1, 255]), bytes([55, 12, 110, 60, 15]), bytes([7, 149, 1]), bytes([55])] - envelope = buildCalltableData(I, B) + envelope = build_calltable_data(I, B) serialized_calltable_representation = serialize_calltable_representation(envelope["fields"], envelope["bytes"]) print(f"{serialized_calltable_representation.hex()}") ``` @@ -96,15 +99,15 @@ and once we have all that we can apply the scripts to example: which produces: ``` -0400000000000000000001000300000003000800000005000b0000000001ff370c6e3c0f07950137 +0400000000000000000001000300000003000800000005000b0000000c0000000001ff370c6e3c0f07950137 ``` In the above hex: -| Serialized length of `fields` collection | field[0].index | field[0].offset | field[1].index | field[1].offset | field[2].index | field[2].offset | field[3].index | field[3].offset | bytes | -| ---------------------------------------- | -------------- | --------------- | -------------- | --------------- | -------------- | --------------- | -------------- | --------------- | ----- | -| 04000000 | 0000 | 00000000 | 0100 | 03000000 | 0300 | 08000000 | 0500 | 0b000000 | 0001ff370c6e3c0f07950137 +| Serialized length of `fields` collection | field[0].index | field[0].offset | field[1].index | field[1].offset | field[2].index | field[2].offset | field[3].index | field[3].offset | number of bytes in `bytes` field | raw bytes of `bytes` field | +| ---------------------------------------- | -------------- | --------------- | -------------- | --------------- | -------------- | --------------- | -------------- | --------------- | ----- | -- | +| 04000000 | 0000 | 00000000 | 0100 | 03000000 | 0300 | 08000000 | 0500 | 0b000000 | 0c000000 | 0001ff370c6e3c0f07950137 -This concludes how we construct and byte-serialize an `envelope`. In the next paragraphs we will explain what are the assumptions and conventions when dealing with `struct`s and `enum`s. +This concludes how we construct and byte-serialize an `envelope`. In the next paragraphs we will explain what are the assumptions and conventions when dealing with `struct`s and `tagged-union`s. ## Serializing uniform data structures @@ -120,15 +123,15 @@ struct A { In the above example we could assign `a` index `0`, `b` index `1` and `c` index `2`. Knowing this and assuming that we know how to byte-serialize `OtherStruct` we should be able to create an `envelope` for this struct and serialize it in the `calltable` scheme. -## Serializing enums +## Serializing tagged-unions -By `enums` we understand polymorphic, but limited (an instance of an enum can be only one of N known `variants`) data structures that are unions of structures and/or other enums. An enum variant can be: +By `tagged-union` we understand polymorphic, but limited (an instance of an tagged-union can be only one of N known `variants`) data structures that are unions of structures and/or other tagged-unions. An tagged-union variant can be: - empty (tag variant) - a struct -- a nested enum +- a nested tagged-union -As mentioned, there is a polymorphic aspect to these kinds of enums and we handle them by convention - `serialization index` `0` is always reserved for a 1 byte discriminator number which defines which enum variant is being serialized, the next indices are used to serialize the fields of specific variants (for empty tag variants there will be no more indices). So, given an enum: +As mentioned, there is a polymorphic aspect to these kinds of tagged-union and we handle them by convention - `serialization index` `0` is always reserved for a 1 byte discriminator number which defines which tagged-union variant is being serialized. The value of this specific pseudo-field will be called `variant discriminator`. Subsequent indices are used to serialize the fields of specific variants (for empty tag variants there will be no more indices). So, given an example tagged-union (implemented in rust, rust equivalent of tagged-union is `enum`): ```rust enum X { @@ -138,13 +141,13 @@ As mentioned, there is a polymorphic aspect to these kinds of enums and we handl } ``` -First we need to chose variant discriminator values for each of the enum variants, let's select: +First we need to chose variant discriminator values for each of the tagged-union variants, let's select: - if variant `A` - the variant discriminator will be `0` - if variant `B` - the variant discriminator will be `1` - if variant `C` - the variant discriminator will be `2` -Again, as with fields in `Serializing enums` the variant discriminator values don't need to start from `0` and don't need to be contiguous, but that is our convention and any "holes" in the definition would indicate a retired enum variant. +Again, as with fields in `Serializing tagged-unions` the variant discriminator values does not need to start from `0` and does not need to be contiguous, but that is our convention and any "discontinuities" in the value set for variant disciminator would indicate a retired tagged-union variant or variants. Next we need to assign field `serialization indices` for each variant: @@ -157,8 +160,8 @@ Next we need to assign field `serialization indices` for each variant: - `2` for the second tuple element (of type u32), - `3` for the second tuple element (of type u64), -As you can see, `serialization indices` for fields need to be unique in scope of a particular enum variant. -Knowing the above, let's see how the `I` and `B` collections would look like for different instances of this enum: +As you can see, `serialization indices` for fields need to be unique in scope of a particular tagged-union variant. +Knowing the above, let's see how the `I` and `B` collections would look like for different instances of this tagged-union: - when serializing variant `X::A` (assuming python notation): ```python diff --git a/versioned_docs/version-2.0.0/concepts/serialization/transaction-deserialization-example.md b/versioned_docs/version-2.0.0/concepts/serialization/transaction-deserialization-example.md new file mode 100644 index 000000000..33ec19208 --- /dev/null +++ b/versioned_docs/version-2.0.0/concepts/serialization/transaction-deserialization-example.md @@ -0,0 +1,386 @@ +# Transaction deserialization example + +``` +**PLEASE NOTE** In this document long snippets representing hex-encoded bytes are split into 64 character lines. Putting such a text in multiple lines does not mean that the newline character is part of the binary payload - newlines are used purely as formatting. +``` + +In this document we will go through byte-deserialization of a full Transaction example. We will assume an examble binary payload and we will step-by-step deserialize it. The goal here is to show a real-life example of how calltable serialization works. To achieve that we will have a Transaction::Version1 variant since Transaction::Deploy doesn't use calltable serialization. + +Here is an example hex-encoded byte array payload: + +``` +0x01030000000000000000000100200000000200170100007d01000013891c67 +a803d1803c932c6a9c342a44e8adb7be3f287cd69a8cab8b2a446fa606000000 +00000000000001003600000002003e0000000300460000000400520000000500 +7d000000cb00000002000000000000000000010001000000220000000001d9bf +2148748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2900c7b00 +00000000000080ee360000000000080000006d792d636861696e040000000000 +0000000001000100000002000900000003000a0000000b0000000053aa080000 +000000010104000000000005000000000000000001000f000000010000000000 +00000000010000000002000f0000000100000000000000000001000000050300 +0f0000000100000000000000000001000000000100000001d9bf2148748a85c8 +9da5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2900c013104d575cdd9a8 +65a54eae377586dd2ea9912fde9511d3471f63a73c1162b7f6c7a290611a527f +c14247e4eb86c308c27f45d2ece4abc506291c600c54ee720b +``` + +## Deserializing Transaction + +The first byte of the payload is `01`, by which we know that we need to attempt to treat it as a Transaction::Version1. We will drop the first byte and proceed deserializing the rest as `TransactionV1` - since that is the internal payload of Transaction::Version1 + +## Deserializing TransactionV1 + +After unpacking the discriminator byte for `Transaction` we are left with the following bytes: + +``` +0x030000000000000000000100200000000200170100007d01000013891c67a8 +03d1803c932c6a9c342a44e8adb7be3f287cd69a8cab8b2a446fa60600000000 +000000000001003600000002003e00000003004600000004005200000005007d +000000cb00000002000000000000000000010001000000220000000001d9bf21 +48748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2900c7b0000 +000000000080ee360000000000080000006d792d636861696e04000000000000 +00000001000100000002000900000003000a0000000b0000000053aa08000000 +0000010104000000000005000000000000000001000f00000001000000000000 +000000010000000002000f00000001000000000000000000010000000503000f +0000000100000000000000000001000000000100000001d9bf2148748a85c89d +a5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2900c013104d575cdd9a865 +a54eae377586dd2ea9912fde9511d3471f63a73c1162b7f6c7a290611a527fc1 +4247e4eb86c308c27f45d2ece4abc506291c600c54ee720b +``` + +TransactionV1 is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x03000000 | le-encoded number of entries in the calltable header for TransactionV1 | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means `hash` | +| 0x00000000 | le encoded `offset` of the first calltable entry | +| 0x0100 | le encoded `index` of the second calltable entry: `1` means `payload` | +| 0x20000000 | le encoded `offset` of the second calltable entry: `32` means that the bytes of `payload` start at byte 32 of the binary payload of the calltable envelope | +| 0x0200 | le encoded `index` of the third calltable entry: `2` means `approvals` | +| 0x17010000 | le encoded `offset` of the third calltable entry: `279` means that the bytes of `payload` start at byte 279 of the binary payload of the calltable envelope | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| 0x7d010000 | 381 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x13891c67a803d1803c932c6a9c342a44e8adb7be3f287cd69a8cab8b2a446fa6 | `hash` field of type Digest, should be interpreted as [here](./types.md#digest-digest) | +| 0x0600000000000000000001003600000002003e000000030046000000040052
00000005007d000000cb00000002000000000000000000010001000000220000
000001d9bf2148748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536354f0a
e2900c7b0000000000000080ee360000000000080000006d792d636861696e04
00000000000000000001000100000002000900000003000a0000000b00000000
53aa080000000000010104000000000005000000000000000001000f00000001
000000000000000000010000000002000f000000010000000000000000000100
00000503000f000000010000000000000000000100000000 | binary representation of `payload` deserialization drill-down [here](#payload-field-deserialization) | +| 0x0100000001d9bf2148748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536
354f0ae2900c013104d575cdd9a865a54eae377586dd2ea9912fde9511d3471f
63a73c1162b7f6c7a290611a527fc14247e4eb86c308c27f45d2ece4abc50629
1c600c54ee720b | binary representation of a collection [Approvals](#deserializing-approvals) | + +### `payload` field deserialization + +Previously we established that the `payload` fields raw bytes are: + +``` +0x0600000000000000000001003600000002003e000000030046000000040052 +00000005007d000000cb00000002000000000000000000010001000000220000 +000001d9bf2148748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536354f0a +e2900c7b0000000000000080ee360000000000080000006d792d636861696e04 +00000000000000000001000100000002000900000003000a0000000b00000000 +53aa080000000000010104000000000005000000000000000001000f00000001 +000000000000000000010000000002000f000000010000000000000000000100 +00000503000f000000010000000000000000000100000000 +``` + +now we will attempt to deserialize it as an instance of `TransactionV1Payload`. + +TransactionV1Payload is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| 0x06000000 | le-encoded number of entries in the calltable header for TransactionV1Payload | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means `initiator_addr` | +| 0x00000000 | le encoded `offset` of the first calltable entry - means that the initiator_addr starts at index 0 of the envelope payload | +| 0x0100 | le encoded `index` of the second calltable entry: `1` means `timestamp` | +| 0x36000000 | le encoded `offset` of the second calltable entry - means that the timestamp starts at index 54 of the envelope payload | +| 0x0200 | le encoded `index` of the third calltable entry: `2` means `ttl` | +| 0x3e000000 | le encoded `offset` of the third calltable entry - means that the ttl starts at index 62 of the envelope payload | +| 0x0300 | le encoded `index` of the fourth calltable entry: `3` means `chain_name` | +| 0x46000000 | le encoded `offset` of the fourth calltable entry - means that the chain_name starts at index 70 of the envelope payload | +| 0x0400 | le encoded `index` of the fifth calltable entry: `4` means `pricing_mode` | +| 0x52000000 | le encoded `offset` of the fifth calltable entry - means that the pricing_mode starts at index 82 of the envelope payload | +| 0x0500 | le encoded `index` of the sixth calltable entry: `5` means `fields` | +| 0x7d000000 | le encoded `offset` of the sixth calltable entry - means that the fields starts at index 125 of the envelope payload | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| 0xcb000000  | 203 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x02000000000000000000010001000000220000000001d9bf2148748a85c89d
a5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2900c | bytes which should be interpreted as `InitiatorAddr` | +| 0x7b00000000000000 | bytes which should be interpreted as `Timestamp` (le unsigned 8 bytes unix-style milliseconds value which translates to `123`) | +| 0x80ee360000000000 | bytes which should be interpreted as `TTL` (le unsigned 8 bytes value which translates to `3600000`) | +| 0x080000006d792d636861696e | bytes which should be interpreted as [`String`](./primitives.md#string-clvalue-string) | +| 0x0400000000000000000001000100000002000900000003000a0000000b0000
000053aa0800000000000101 | bytes which should be interpreted as `PricingMode` | +| 0x04000000000005000000000000000001000f00000001000000000000000000
010000000002000f00000001000000000000000000010000000503000f000000
010000000000000000000100000000 | bytes which should be interpreted as a map of field id -> field mapping. Please see [this paragraph](#payloadfields-field-deserialization) for details | + +### `payload.initiator_addr` field deserialization + +Previously we established that the `initiator_addr` fields raw bytes were: + +``` +0x02000000000000000000010001000000220000000001d9bf2148748a85c89 +da5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2900c +``` + +now we will attempt to deserialize it as an instance of `InitiatorAddr`. + +InitiatorAddr is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x02000000 | le-encoded number of entries in the calltable header for InitiatorAddr | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means variant discriminator of the union type (since InitiatorAddr is a union type) | +| 0x00000000 | le encoded `offset` of the first calltable entry - means that the variant discriminator starts at byte 0 of the envelope payload. | +| 0x0100 | le encoded `index` of the second calltable entry: `1` - we don't really know what field it is at this moment since we don't know what the variant is | +| 0x01000000 | le encoded `offset` of the second calltable entry - means that the payload of the first field of the strucure starts at position 1 of the envelope payload | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x22000000 | 34 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x00 | value of the discriminator - deserializes to `PublicKey` variant. We now know that `index = 1` from the header points to an instance of `PublicKey` | +| 0x01d9bf2148748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536354f0ae2
900c | `PublicKey` whish should be interpreted as [`explained here`](./types.md#publickey-publickey) | + +### `payload.pricing_mode` field deserialization + +Previously we established that the `pricing_mode` fields raw bytes were: + +``` +0x0400000000000000000001000100000002000900000003000a0000000b000 +0000053aa0800000000000101 +``` + +now we will attempt to deserialize it as an instance of `PricingMode`. + +PricingMode is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x04000000 | le-encoded number of entries in the calltable header for PricingMode | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means variant discriminator of the union type (since PricingMode is a union type) | +| 0x00000000 | le encoded `offset` of the first calltable entry - means that the variant discriminator starts at byte 0 of the envelope payload. | +| 0x0100 | le encoded `index` of the second calltable entry: `1` - we don't really know what field it is at this moment since we don't know what the variant is | +| 0x01000000 | le encoded `offset` of the second calltable entry - means that the payload of the first field of the strucure starts at index 1 of the envelope payload | +| 0x0200 | le encoded `index` of the third calltable entry: `2` - we don't really know what field it is at this moment since we don't know what the variant is | +| 0x09000000 | le encoded `offset` of the third calltable entry - means that the payload of the second field of the strucure starts at index 9 of the envelope payload | +| 0x0300 | le encoded `index` of the fourth calltable entry: `3` - we don't really know what field it is at this moment since we don't know what the variant is | +| 0x0a000000 | le encoded `offset` of the fourth calltable entry - means that the payload of the second field of the strucure starts at index 10 of the envelope payload | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x0b000000 | 11 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x00 | value of the discriminator - deserializes to `PaymentLimited` variant. We now know that: `index = 1` from the header points to a 8 bytes unsigned number `payment_amount` field; `index = 2` from the header points to a 1 byte unsigned number `gas_price_tolerance` field; `index = 3` from the header points to a 1 byte bool `standard_payment` field | +| 0x53aa080000000000 | 567891 (8-bytes unsigned LE `payment_amount` ) | +| 0x01 | 1 (1-bytes unsigned LE) `gas_price_tolerance` | +| 0x01 | true `standard_payment` | + +### `payload.fields` field deserialization + +Previously we established that the `fields` fields raw bytes were: + +``` +0x04000000000005000000000000000001000f00000001000000000000000000 +010000000002000f00000001000000000000000000010000000503000f000000 +010000000000000000000100000000 +``` + +Payload of this field is not serialized using calltable. It is serialized as a map (for details, see [here](#version1payloadfields)). We will attempt to deconstruct the payload: + +| hex formatted bytes | Interpretation | +| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 0x04000000 | `4` - number of entries in the map (4 bytes LE encoded unsigned number) | +| 0x0000 | `0` - key of first entry (2 bytes encoded unsigned number) | +| 0x05000000 | `5` - number of bytes of the payload of the value under key `0` (4 bytes encoded unsigned number) | +| 0x0000000000 | 5 bytes read which is the raw payload for key `0`. Based on the fields index table [here](#version1payloadfields) we see that it should be interpreted as `TransactionArgs`. To deserialize `TransactionArgs` see [here](./types.md#transactionargs-transaction-args) | +| 0x0100 | `1` - key of second entry (2 bytes encoded unsigned number) | +| 0x0f000000 | `15` - number of bytes of the payload of the value under key `1` (4 bytes encoded unsigned number) | +| 0x010000000000000000000100000000 | 15 bytes read which is the raw payload for key `1`. Based on the fields index table [here](#version1payloadfields) we see that it should be interpreted as `TransactionTarget`. Deserialization explanation is [here](#payloadfields1-field-deserialization) | +| 0x0200 | `2` - key of third entry (2 bytes encoded unsigned number) | +| 0x0f000000 | `15` - number of bytes of the payload of the value under key `2` (4 bytes encoded unsigned number) | +| 0x010000000000000000000100000005 | 15 bytes read which is the raw payload for key `2`. Based on the fields index table [here](#version1payloadfields) we see that it should be interpreted as `TransactionEntryPoint`. Deserialization explanation is [here](#payloadfields2-field-deserialization) | +| 0x0300 | `3` - key of third entry (2 bytes encoded unsigned number) | +| 0x0f000000 | `15` - number of bytes of the payload of the value under key `2` (4 bytes encoded unsigned number) | +| 0x010000000000000000000100000000 | 15 bytes read which is the raw payload for key `2`. Based on the fields index table [here](#version1payloadfields) we see that it should be interpreted as `TransactionScheduling`. Deserialization explanation is [here](#payloadfields3-field-deserialization) | + +### `payload.fields.1` field deserialization + +Previously we established that the TransactionTarget fields map entry raw bytes are: + +``` +0x010000000000000000000100000000 +``` + +TransactionTarget is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| 0x01000000 | le-encoded number of entries in the calltable header for TransactionTarget | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means variant discriminator of the union type (since PricingMode is a union type) | +| 0x00000000 | le encoded `offset` of the first calltable entry - means that the variant discriminator starts at byte 0 of the envelope payload. | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | --------------------------------------------------------------------------------------------- | +| 0x01000000 | 1 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x00 | `0` value of the discriminator - deserializes to `Native` variant. This variant has no fields | + +### `payload.fields.2` field deserialization + +Previously we established that the TransactionEntryPoint fields map entry raw bytes are: + +``` +0x010000000000000000000100000005 +``` + +TransactionEntryPoint is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | +| 0x01000000 | le-encoded number of entries in the calltable header for TransactionEntryPoint | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means variant discriminator of the union type (since TransactionEntryPoint is a union type) | +| 0x00000000 | le encoded `offset` of the first calltable entry - means that the variant discriminator starts at byte 0 of the envelope payload. | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | ----------------------------------------------------------------------------------------------- | +| 0x01000000 | 1 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x05 | `5` value of the discriminator - deserializes to `Delegate` variant. This variant has no fields | + +### `payload.fields.3` field deserialization + +Previously we established that the TransactionScheduling fields map entry raw bytes are: + +``` +0x010000000000000000000100000000 +``` + +TransactionScheduling is serialized using calltable representation. From the [calltable serialization](./calltable-serialization.md) document we know that we need to split the bytes into a header and payload: + +Interpretation of the bytes representing header of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | +| 0x01000000 | le-encoded number of entries in the calltable header for TransactionScheduling | +| 0x0000 | le encoded `index` of the first calltable entry: `0` means variant discriminator of the union type (since TransactionScheduling is a union type) | +| 0x00000000 | le encoded `offset` of the first calltable entry - means that the variant discriminator starts at byte 0 of the envelope payload. | + +Interpretation of the bytes representing payload of the calltable envelope: + +| hex formatted bytes | Interpretation | +| ------------------- | ----------------------------------------------------------------------------------------------- | +| 0x01000000 | 1 (4-bytes unsigned LE) - number of bytes in the paylod | +| 0x00 | `0` value of the discriminator - deserializes to `Standard` variant. This variant has no fields | + +### deserializing Approvals + +Previously we established that the `approvals` raw bytes are: + +``` +0x0100000001d9bf2148748a85c89da5aad8ee0b0fc2d105fd39d41a4c796536 +354f0ae2900c013104d575cdd9a865a54eae377586dd2ea9912fde9511d3471f +63a73c1162b7f6c7a290611a527fc14247e4eb86c308c27f45d2ece4abc50629 +1c600c54ee720b +``` + +We can deserialize the above as [collection](./primitives.md#clvalue-list) of [approvals](./types.md#approval-approval) + +## Helper script + +The binary payload used in this example can be reproduced using the following script written in rust and facilitating the reference implementation [casper-types](https://crates.io/crates/casper-types) library. + +```rust + use casper_types::{ + Approval, Digest, InitiatorAddr, PricingMode, PublicKey, RuntimeArgs, SecretKey, TimeDiff, + Timestamp, Transaction, TransactionArgs, TransactionEntryPoint, TransactionHash, + TransactionScheduling, TransactionTarget, TransactionV1, TransactionV1Hash, + TransactionV1Payload, bytesrepr::ToBytes, + }; + use std::collections::{BTreeMap, BTreeSet}; + + let signer_secret_key = + SecretKey::ed25519_from_bytes([15u8; SecretKey::ED25519_LENGTH]).unwrap(); + let signer_public_key = PublicKey::from(&signer_secret_key); + let timestamp = Timestamp::from(123); + let ttl = TimeDiff::from_seconds(3600); + let pricing_mode = PricingMode::PaymentLimited { + payment_amount: 567_891_u64, + gas_price_tolerance: 1, + standard_payment: true, + }; + let initiator_addr = InitiatorAddr::PublicKey(signer_public_key); + let mut fields = BTreeMap::new(); + + let arg = TransactionArgs::Named(RuntimeArgs::new()); + let arg_bytes = arg + .to_bytes() + .expect("should be able to serialize transaction args to bytes"); + fields.insert(0_u16, arg_bytes.into()); + + let transaction_target = TransactionTarget::Native; + let target_bytes = transaction_target + .to_bytes() + .expect("should be able to serialize transaction_target to bytes"); + fields.insert(1_u16, target_bytes.into()); + + let entry_point = TransactionEntryPoint::Delegate; + let entry_point_bytes = entry_point + .to_bytes() + .expect("should be able to serialize entry_point to bytes"); + fields.insert(2_u16, entry_point_bytes.into()); + + let scheduling = TransactionScheduling::Standard; + let scheduling_bytes = scheduling + .to_bytes() + .expect("should be able to serialize scheduling to bytes"); + fields.insert(3_u16, scheduling_bytes.into()); + + let payload = TransactionV1Payload::new( + "my-chain".to_owned(), + timestamp, + ttl, + pricing_mode, + initiator_addr, + fields, + ); + let payload_bytes = payload + .to_bytes() + .expect("It should be possible to turn payload into bytes"); + let hash = Digest::hash(payload_bytes); + let approval = Approval::create( + &TransactionHash::V1(TransactionV1Hash::from(hash)), + &signer_secret_key, + ); + let mut approvals = BTreeSet::new(); + approvals.insert(approval); + + let transaction_v1 = TransactionV1::new(hash.into(), payload, approvals); + let transaction = Transaction::V1(transaction_v1); + println!( + "{}", + hex::encode( + transaction + .to_bytes() + .expect("Expected transaction to correctly serialize") + ) + ); +``` diff --git a/versioned_docs/version-2.0.0/concepts/serialization/transaction-serialization-example.md b/versioned_docs/version-2.0.0/concepts/serialization/transaction-serialization-example.md new file mode 100644 index 000000000..9b9418322 --- /dev/null +++ b/versioned_docs/version-2.0.0/concepts/serialization/transaction-serialization-example.md @@ -0,0 +1,315 @@ +# Transaction serialization example + +``` +**PLEASE NOTE** In this document long snippets representing hex-encoded bytes are split into 64 character lines. Putting such a text in multiple lines does not mean that the newline character is part of the binary payload - newlines are used purely as formatting. +``` + +In this document we will go through byte-serialization of a full Transaction example. We will first describe an example Transaction of variant Version1 and afterwards serialize it's parts to assemble a binary representation of the transaction. + +In this document when we do signing of the hash it has beed done with a secret keys which `der` encoding is: +`0x302e020100300506032b6570042204209090909090909090909090909090909090909090909090909090909090909090` + +## Structure of the example transaction + +- Transaction + + - Transaction of variant Version1 + + - `hash`: 0xde78f579a9afb234cb6ee3ab989a44f44a1544caa991a470dfa0eccde2937337 (hex encoded bytes of the Digest) + - `payload`: + + - `initiator_addr`: `PublicKey` case with internal key of `Ed25519` variant and the actual key bytes as: `0x011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9` + - `timestamp`: timestamp with 1752478672515 milliseconds in unix epoch time + - `ttl`: time period of 2 hours. + - `chain_name`: string with value `abc` + - `pricing_mode`: `PricingMode` of case `Fixed` with values: + - `additional_computation_factor`: `2` + - `gas_price_tolerance`: `3` + - `fields`: A [map](./primitives.md#map-clvalue-map). This `fields` instance will have the following key-value mappings: + - key `0`; => binary-serialized [`TransactionArgs`](./types.md#transactionargs-transaction-args) of variant `Named` with the inner RuntimeArgs being a map of "a" => "xyz". When serialized, this transaction args structure is 0x1600000000010000000100000061070000000300000078797a0a. An explanation how it was binary serialized see [`serializing transaction args`](#serializing-transaction-args) + - key `1`; value as binary serialized [`TransactionTarget`](./transaction.md#transactiontarget) of variant `Native`. When serialized, this transaction args structure is 0x0d00000001000000000000000100000000. An explanation how it was binary serialized see [`serializing transaction target`](#serializing-transaction-target) + - key `2`; value as binary serialized [`TransactionEntryPoint`](./transaction.md#transactionentrypoint) of variant `AddBid`. When serialized, this transaction args structure is 0x0f000000010000000000000000000100000003. An explanation how it was binary serialized see [`serializing transaction entry point`](#serializing-transaction-target) + - key `3`; value as binary serialized [`TransactionScheduling`](./transaction.md#transactionentrypoint) of variant `Standard`. When serialized, this transaction args structure is 0x0d00000001000000000000000100000000. An explanation how it was binary serialized see [`serializing transaction scheduling`](#serializing-transaction-scheduling) + + - `approvals`: + - `0`: + - `signer`: 0x011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9 (hex encoded bytes of a public key paired with the secret key mentioned a the start of the document) + - `signature`: 0x01bcc0ee36a4bb5d8d7b8934bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 (hex encoded bytes of the signature) + +## Serialization of the example Transaction + +The first byte of the serialized pyalod will be `0x01` (based on [this](./transaction.md#version1), knowing that we are serializing `Version1`). +The next bytes are binary representation of [`TransactionV1`](./transaction.md#transactionv1). Binary representation of the `TransactionV1` from the example is: + +``` +0x030000000000000000000100200000000200150100007b010000de78f579a9 +afb234cb6ee3ab989a44f44a1544caa991a470dfa0eccde29373370600000000 +000000000001003600000002003e00000003004600000004004d00000005006a +000000c9000000020000000000000000000100010000002200000000011a3818 +79f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d98366de +079801000000dd6d000000000003000000616263030000000000000000000100 +0100000002000200000003000000010302040000000000160000000001000000 +0100000061070000000300000078797a0a01000f000000010000000000000000 +00010000000002000f00000001000000000000000000010000000303000f0000 +0001000000000000000000010000000001000000011a381879f8a8dc97361d01 +2e3b472207cc7313ed1a81c918eebfa872b93414d901bcc0ee36a4bb5d8d7b89 +34bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9 +bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 +``` + +You can find the step-by-step explanation [here](#serialization-of-transactionv1) + +The final binary representation of the transaction is: + +``` +0x01030000000000000000000100200000000200150100007b010000de78f579 +a9afb234cb6ee3ab989a44f44a1544caa991a470dfa0eccde293733706000000 +00000000000001003600000002003e00000003004600000004004d0000000500 +6a000000c9000000020000000000000000000100010000002200000000011a38 +1879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d98366 +de079801000000dd6d0000000000030000006162630300000000000000000001 +0001000000020002000000030000000103020400000000001600000000010000 +000100000061070000000300000078797a0a01000f0000000100000000000000 +0000010000000002000f00000001000000000000000000010000000303000f00 +000001000000000000000000010000000001000000011a381879f8a8dc97361d +012e3b472207cc7313ed1a81c918eebfa872b93414d901bcc0ee36a4bb5d8d7b +8934bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541 +e9bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 +``` + +## Serialization Of TransactionV1 + +To serialize `TransactionV1` we will use the calltable schema (see [here](./transaction.md#transactionv1)). The exaplanation of the individual bytes are below: + +This particular `TransactionV1` consists of three fields: + +- `hash`: `0xde78f579a9afb234cb6ee3ab989a44f44a1544caa991a470dfa0eccde2937337` (32 bytes) +- `payload`: `0x0600000000000000000001003600000002003e00000003004600000004004d00000005006a000000c9000000020000000000000000000100010000002200000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d98366de079801000000dd6d00000000000300000061626303000000000000000000010001000000020002000000030000000103020400000000001600000000010000000100000061070000000300000078797a0a01000f00000001000000000000000000010000000002000f00000001000000000000000000010000000303000f000000010000000000000000000100000000` +- `approvals` : `0x01000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d901bcc0ee36a4bb5d8d7b8934bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000` + + | Calltable segment | Hex-fmt representation | Description | + | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- | + | HEADER | 0x03000000 | u32 encoded number of calltable header entries (TransactionV1 has 3 fields) | + | HEADER | 0x0000 | u16 encoded field index of the first field (`hash`) | + | HEADER | 0x00000000 | u32 encoded bytes offset where the `hash` starts in the raw payload | + | HEADER | 0x0100 | u16 encoded field index of the second field (`payload`) | + | HEADER | 0x20000000 | u32 encoded bytes offset where the `payload` starts in the raw payload | + | HEADER | 0x0200 | u16 encoded field index of the third field (`approvals`) | + | HEADER | 0x15010000 | u32 encoded bytes offset where the `approvals` starts in the raw payload | + | PAYLOAD | 0x7b010000 | u32 encoded number of bytes of the raw payload | + | PAYLOAD | 0xde78f579a9afb234cb6ee3ab989a44f44a1544caa991a470dfa0eccde2937337 | bytes of `hash` | + | PAYLOAD | 0x0600000000000000000001003600000002003e00000003004600000004004d00000005006a000000c9000000020000000000000000000100010000002200000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d98366de079801000000dd6d00000000000300000061626303000000000000000000010001000000020002000000030000000103020400000000001600000000010000000100000061070000000300000078797a0a01000f00000001000000000000000000010000000002000f00000001000000000000000000010000000303000f000000010000000000000000000100000000 | bytes of `payload` | + | PAYLOAD | 0x01000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d901bcc0ee36a4bb5d8d7b8934bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 | bytes of `approvals` | + +Which concatenates to: + +``` +0x030000000000000000000100200000000200150100007b010000de78f579a9 +afb234cb6ee3ab989a44f44a1544caa991a470dfa0eccde29373370600000000 +000000000001003600000002003e00000003004600000004004d00000005006a +000000c9000000020000000000000000000100010000002200000000011a3818 +79f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d98366de +079801000000dd6d000000000003000000616263030000000000000000000100 +0100000002000200000003000000010302040000000000160000000001000000 +0100000061070000000300000078797a0a01000f000000010000000000000000 +00010000000002000f00000001000000000000000000010000000303000f0000 +0001000000000000000000010000000001000000011a381879f8a8dc97361d01 +2e3b472207cc7313ed1a81c918eebfa872b93414d901bcc0ee36a4bb5d8d7b89 +34bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9 +bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 +``` + +### Serializing Approvals + +To recap, the approvals structure is as follows: + +- `approvals`: + - `0`: + - `signer`: 0x011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9 (hex encoded bytes of the public key) + - `signature`: 0x01bcc0ee36a4bb5d8d7b8934bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 (hex encoded bytes of the signature) + +The binary representation "blob" will be: + +- 0x01000000 - `u32` encoded number of entries (1) +- 0x011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9 - the signer bytes +- 0x01bcc0ee36a4bb5d8d7b8934bbd2e09270456eb201db15f6288490879c9a8e44d5a40514600952e6a541e9bf8f0bcc7f9dcec55533c0faec0b8674c979c8c86000 - the signature bytes + +Concatenated: + +``` +0x01000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebf +a872b93414d901bcc0ee36a4bb5d8d7b8934bbd2e09270456eb201db15f62884 +90879c9a8e44d5a40514600952e6a541e9bf8f0bcc7f9dcec55533c0faec0b86 +74c979c8c86000 +``` + +### Serializing payload + +To serialize `TransactionV1Payload` we will use the calltable schema (see [here](./transaction.md#transactionv1payload)). The exaplanation of the individual bytes are below: + +This particular `TransactionV1Payload` consists of six fields: + +- `initiator_addr`: it's binary serialization is `0x020000000000000000000100010000002200000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9` (see [here](#serializing-initiatoraddr)) +- `timestamp`: it's binary serialization is: `0x8366de0798010000` (see [here](#serializing-timestamp)) +- `ttl`: it's binary serialization is: `0x00dd6d0000000000` (see [here](#serializing-ttl)) +- `chain_name`: it's binary serialization is: `0x03000000616263` , (see [here](./primitives.md#clvalue-string)) +- `pricing_mode`: it's binary serialization is: `0x0300000000000000000001000100000002000200000003000000010302` , (see [here](#serializing-pricing-mode)) +- `fields`: it's binary serialization is: `0x0400000000001600000000010000000100000061070000000300000078797a0a01000f00000001000000000000000000010000000002000f00000001000000000000000000010000000303000f000000010000000000000000000100000000` , (see [here](#serializing-fields)) + +| Calltable segment | Hex-fmt representation | Description | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | +| HEADER | 0x06000000 | u32 encoded number of calltable header entries (this PricingMode binary representation will only have one field) | +| HEADER | 0x0000 | index of the first field (`initiator_addr`) | +| HEADER | 0x00000000 | bytes offset where the field with index `0` starts in the payload (we have no previous fields, so there is no offset) | +| HEADER | 0x0100 | u16 encoded field index of the second field (`timestamp`) | +| HEADER | 0x36000000 | u32 encoded bytes offset where the `timestamp` starts in the raw payload | +| HEADER | 0x0200 | u16 encoded field index of the second field (`ttl`) | +| HEADER | 0x3e000000 | u32 encoded bytes offset where the `ttl` starts in the raw payload | +| HEADER | 0x0300 | u16 encoded field index of the second field (`chain_name`) | +| HEADER | 0x46000000 | u32 encoded bytes offset where the `chain_name` starts in the raw payload | +| HEADER | 0x0400 | u16 encoded field index of the second field (`pricing_mode`) | +| HEADER | 0x4d000000 | u32 encoded bytes offset where the `pricing_mode` starts in the raw payload | +| HEADER | 0x0500 | u16 encoded field index of the second field (`fields`) | +| HEADER | 0x6a000000 | u32 encoded bytes offset where the `fields` starts in the raw payload | +| PAYLOAD | 0xc9000000 | u32 encoded number of bytes of the raw payload | +| PAYLOAD | 0x020000000000000000000100010000002200000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9 | bytes of `initiator_addr` value | +| PAYLOAD | 0x8366de0798010000 | bytes of `timestamp` value | +| PAYLOAD | 0x00dd6d0000000000 | bytes of `ttl` value | +| PAYLOAD | 0x03000000616263 | bytes of `chain_name` value | +| PAYLOAD | 0x0300000000000000000001000100000002000200000003000000010302 | bytes of `pricing_mode` value | +| PAYLOAD | 0x0400000000001600000000010000000100000061070000000300000078797a0a01000f00000001000000000000000000010000000002000f00000001000000000000000000010000000303000f000000010000000000000000000100000000 | bytes of `fields` value | + +Which concatenates to: + +``` + 0x0600000000000000000001003600000002003e00000003004600000004004d00000005006a000000c9000000020000000000000000000100010000002200000000011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d98366de079801000000dd6d00000000000300000061626303000000000000000000010001000000020002000000030000000103020400000000001600000000010000000100000061070000000300000078797a0a01000f00000001000000000000000000010000000002000f00000001000000000000000000010000000303000f000000010000000000000000000100000000 +``` + +#### Serializing Pricing Mode + +To serialize `PricingMode` we will use the calltable schema (see [here](./transaction.md#pricingmode)). This particular PricingMode is of type `Fixed` The exaplanation of the individual bytes are below: + +| Calltable segment | Hex-fmt representation | Description | +| ----------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------- | +| HEADER | 0x03000000 | u32 encoded number of calltable header entries (this PricingMode binary representation will only have one field) | +| HEADER | 0x0000 | index of the first field (for PricingMode, this is the `variant discriminator`) | +| HEADER | 0x00000000 | bytes offset where the field with index `0` starts in the payload (we have no previous fields, so there is no offset) | +| HEADER | 0x0100 | u16 encoded field index of the second field (`additional_computation_factor`) | +| HEADER | 0x01000000 | u32 encoded bytes offset where the `gas_price_tolerance` starts in the raw payload | +| HEADER | 0x0200 | u16 encoded field index of the second field (`gas_price_tolerance`) | +| HEADER | 0x02000000 | u32 encoded bytes offset where the `additional_computation_factor` starts in the raw payload | +| PAYLOAD | 0x03000000 | u32 encoded number of bytes of the raw payload | +| PAYLOAD | 0x01 | u8 encoded variant discriminator value for `Fixed` (1) | +| PAYLOAD | 0x03 | u8 encoded value of `gas_price_tolerance` | +| PAYLOAD | 0x02 | u8 encoded value of `additional_computation_factor` | + +Which concatenates to: + +``` + 0x0300000000000000000001000100000002000200000003000000010302 +``` + +#### Serializing payload.fields + +Fields is a [map](./primitives.md#map-clvalue-map) which keys are u16 and values are collections of bytes. We can assemble the binary serialization as follows: + +| hex-encoded bytes | Explanation | +| ------------------------------------------------------ | ------------------------------------------------------------------------------------ | +| 0x04000000 | u32 encoded number of elements in the map (4) | +| 0x0000 | u16 encoded key: `0` | +| 0x1600000000010000000100000061070000000300000078797a0a | binary payload of the first value (see [here](#serializing-transaction-args)) | +| 0x0100 | u16 encoded key: `1` | +| 0x0f000000010000000000000000000100000000 | binary payload of the first value (see [here](#serializing-transaction-target)) | +| 0x0200 | u16 encoded key: `2` | +| 0x0f000000010000000000000000000100000003 | binary payload of the first value (see [here](#serializing-transaction-entry-point)) | +| 0x0300 | u16 encoded key: `2` | +| 0x0f000000010000000000000000000100000000 | binary payload of the first value (see [here](#serializing-transaction-scheduling)) | + +Which concatenates to: + +``` +0x0400000000001600000000010000000100000061070000000300000078797a +0a01000f00000001000000000000000000010000000002000f000000010000 +00000000000000010000000303000f00000001000000000000000000010000 +0000 +``` + +#### Serializing Timestamp + +Value of this field is serialized as explained [here](./types.md#timestamp-timestamp). It's binary serialization is: `0x8366de0798010000` + +#### Serializing Ttl + +Value of this field is serialized as explained [here](./types.md#timediff-timediff). It's binary serialization is: `0x00dd6d0000000000` + +#### Serializing InitiatorAddr + +To serialize `InitiatorAddr` we will use the calltable schema (see [here](./transaction.md#initiatoraddr)). This particular InitiatorAddr is of type `PublicKey` The exaplanation of the individual bytes are below: + +| Calltable segment | Hex-fmt representation | Description | +| ----------------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | +| HEADER | 0x02000000 | u32 encoded number of calltable header entries (this InitiatorAddr binary representation will only have one field) | +| HEADER | 0x0000 | index of the first field (for InitiatorAddr, this is the `variant discriminator`) | +| HEADER | 0x00000000 | bytes offset where the field with index `0` starts in the payload (we have no previous fields, so there is no offset) | +| HEADER | 0x0100 | index of the second field (for InitiatorAddr, this is the public key) | +| HEADER | 0x01000000 | bytes offset where the field with index `0` starts in the payload | +| PAYLOAD | 0x22000000 | u32 encoded number of bytes of the raw payload | +| PAYLOAD | 0x00 | value for the variant discriminator (`0` means "PublicKey") | +| PAYLOAD | 0x011a381879f8a8dc97361d012e3b472207cc7313ed1a81c918eebfa872b93414d9 | raw bytes of the public key | + +And the concatenated version is: + +``` +0x020000000000000000000100010000002200000000011a381879f8a8dc9736 +1d012e3b472207cc7313ed1a81c918eebfa872b93414d9 +``` + +### Serializing Transaction Args + +To recap, the structure of `TransactionArgs` which we want to serialize is `TransactionArgs` of variant `Named`, the inner `RuntimeArgs`being a mapping of `"a" => "xyz"`. +Firstly let's serialize `RuntimeArgs`. It will be a collection of [NamedArg](./types.md#namedarg-namedarg). The collection has one element, so the first 4 bytes will be `0x010000`, followed by the first argumentname ("a") which binary reprentation is `0x0100000061`, followed by cl-value representation of `xyz` string which is `0x070000000300000078797a0a`. This gives binary representation of `RuntimeArgs`: `0x010000000100000061070000000300000078797a0a` By looking at [TransactionArgs doc](./types.md#transactionargs-transaction-args) we see that the binary represenation will be `0x00` followed by serialization of the RuntimeArgs, which gives `0x00010000000100000061070000000300000078797a0a`. We also should notice that the value that we will be putting into the `fields` map is going to be a binary representation of a collection of bytes. We will need to prepend the raw payload with the bytes-length of the payload (as described [here](./types.md#bytes-bytes)). The payload consists of 22 bytes, so the u32 representation of it's length is going to be `0x16000000` So the final "blob" of bytes that we will be putting into the `fields` map will be: `0x1600000000010000000100000061070000000300000078797a0a`. + +### Serializing Transaction Target + +To recap, the TransactionTarget which we want to serialize is a `Native` one. To serialize this we will be using the calltable scheme. To construct the binary representation we will need: + +| Calltable segment | Hex-fmt representation | Description | +| ----------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| HEADER | 0x01000000 | u32 encoded number of calltable header entries (this TransactionTarget binary representation will only have one field) | +| HEADER | 0x0000 | index of the first field (for TransactionTarget, this is the `variant discriminator`) | +| HEADER | 0x00000000 | bytes offset where the field with index `0` starts in the payload (we have no previous fields, so there is no offset) | +| PAYLOAD | 0x01000000 | u32 encoded number of bytes of the raw payload | +| PAYLOAD | 0x00 | first field (variant discriminator) value serialized (`TransactionTarget::Native`) serialized as u8 | + +Which gives a binary representation of: `0x010000000000000000000100000000` +We also should notice that the value that we will be putting into the `fields` map is going to be a binary representation of a collection of bytes. We will need to prepend the raw payload with the bytes-length of the payload (as described [here](./types.md#bytes-bytes)). The payload consists of 15 bytes, so the u32 representation of it's length is going to be `0x0f000000` So the final "blob" of bytes that we will be putting into the `fields` map will be: `0x0f000000010000000000000000000100000000`. + +### Serializing Transaction Entry Point + +To recap, the TransactionEntryPoint which we want to serialize is a `AddBid` one. To serialize this we will be using the calltable scheme. To construct the binary representation we will need: + +| Calltable segment | Hex-fmt representation | Description | +| ----------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| HEADER | 0x01000000 | u32 encoded number of calltable header entries (this TransactionEntryPoint binary representation will only have one field) | +| HEADER | 0x0000 | index of the first field (for TransactionEntryPoint, this is the `variant discriminator`) | +| HEADER | 0x00000000 | bytes offset where the field with index `0` starts in the payload (we have no previous fields, so there is no offset) | +| PAYLOAD | 0x01000000 | u32 encoded number of bytes of the raw payload | +| PAYLOAD | 0x03 | first field (variant discriminator) value serialized (`TransactionEntryPoint::AddBid`) serialized as u8 | + +Which gives a binary representation of: `0x010000000000000000000100000003` +We also should notice that the value that we will be putting into the `fields` map is going to be a binary representation of a collection of bytes. We will need to prepend the raw payload with the bytes-length of the payload (as described [here](./types.md#bytes-bytes)). The payload consists of 15 bytes, so the u32 representation of it's length is going to be `0x0f000000` So the final "blob" of bytes that we will be putting into the `fields` map will be: `0x0f000000010000000000000000000100000003`. + +### Serializing Transaction Scheduling + +To recap, the TransactionScheduling which we want to serialize is a `Standard` one. To serialize this we will be using the calltable scheme. To construct the binary representation we will need: + +| Calltable segment | Hex-fmt representation | Description | +| ----------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| HEADER | 0x01000000 | u32 encoded number of calltable header entries (this TransactionScheduling binary representation will only have one field) | +| HEADER | 0x0000 | index of the first field (for TransactionScheduling, this is the `variant discriminator`) | +| HEADER | 0x00000000 | bytes offset where the field with index `0` starts in the payload (we have no previous fields, so there is no offset) | +| PAYLOAD | 0x01000000 | u32 encoded number of bytes of the raw payload | +| PAYLOAD |  0x00 | raw payload | + +Which gives a binary representation of: `0x010000000000000000000100000000` +We also should notice that the value that we will be putting into the `fields` map is going to be a binary representation of a collection of bytes. We will need to prepend the raw payload with the bytes-length of the payload (as described [here](./types.md#bytes-bytes)). The payload consists of 15 bytes, so the u32 representation of it's length is going to be `0x0f000000` So the final "blob" of bytes that we will be putting into the `fields` map will be: `0x0f000000010000000000000000000100000000`. diff --git a/versioned_docs/version-2.0.0/concepts/serialization/transaction.md b/versioned_docs/version-2.0.0/concepts/serialization/transaction.md index a066be930..5bdf5a803 100644 --- a/versioned_docs/version-2.0.0/concepts/serialization/transaction.md +++ b/versioned_docs/version-2.0.0/concepts/serialization/transaction.md @@ -5,7 +5,7 @@ | [Deploy](#deploy) | | [Version1](#version1) | -A transaction is an enum-type whose variants represent atomic units of work that can be sent to the node for execution. +A transaction is an tagged-union type whose variants represent atomic units of work that can be sent to the node for execution. Currently it consists of two possible variants: @@ -23,7 +23,8 @@ Serializing a `Transaction` of variant `Deploy`: ## Version1 -Transaction::Version1 is the new way that work can be proposed to a node. It's nature is more amorphic than Transaction::Deploy. This type of Transaction uses a self-describing binary serialization scheme (comparing to other [Binary Serialization Standard](./index.md) types). The self-describing scheme is called `Calltable serialization` in this document. An in-depth description of how this self-describing standard works is in the [calltable serialization](./calltable-serialization.md) document. We will be using `serialization_index` and `variant discriminator` terms, their interpretation is explained in the [calltable serialization](./calltable-serialization.md) document also. +Transaction::Version1 is the new way that work can be proposed to a node. It's nature is more amorphic than Transaction::Deploy. This type of Transaction uses a self-describing binary serialization scheme (comparing to other [Binary Serialization Standard](./index.md) types). The self-describing scheme is called `Calltable serialization` in this document. In general, the layout of this new style of transaction is ["tagged-union"](https://en.wikipedia.org/wiki/Tagged_union) oriented. An in-depth description of how this self-describing standard works is in the [calltable serialization](./calltable-serialization.md) document. +In the scope of this documentation, for abbreviation, we will call these "tagged-unions". Tagged unions are data structures that can define very different internal fields depending on which variant is currently being used. For serialization we denote the variant (effectively - "which variant of the union was created for this instance of the union type") and serialize it alongside the fields. This "variant marking" is stored as a one byte unsigned [number](./primitives.md#numeric-clvalue-numeric) and will be referred to in this document as `variant discriminator`. The terms `serialization_index` and `variant discriminator` (which we use in this document) are also explained in the [calltable serialization](./calltable-serialization.md) document also. Serializing a `Transaction` of variant `Version1`: @@ -38,7 +39,7 @@ Serializing a `Transaction` of variant `Version1`: - `payload`- of type [`TransactionV1Payload`](#transactionv1payload), contains all the actual data that the transaction sends to node. It's `serialization_index` is `1` - `approvals` - collection of [Approvals](./structures.md#approval) being signatures over `hash`. It's `serialization_index` is `2` -### TransactionV1Payload +#### TransactionV1Payload `TransactionV1Payload` (serialized using calltable scheme) consists of: @@ -49,7 +50,7 @@ Serializing a `Transaction` of variant `Version1`: - `pricing_mode` - [PricingMode](#pricingmode) declaration of how the transaction should be payed for. It's `serialization_index` is `4` - `fields` - please see [v1.payload.fields](#v1payloadfields) section. It's `serialization_index` is `5` -#### InitiatorAddr +##### InitiatorAddr `InitiatorAddr` (serialized using calltable scheme) consists of: @@ -60,7 +61,7 @@ Serializing a `Transaction` of variant `Version1`: - variant discriminator of value `1` - [`AccountHash`](./types.md#account-hash-account-hash) serialized under serialization key `1` -#### PricingMode +##### PricingMode `PricingMode` (serialized using calltable scheme) consists of: @@ -77,9 +78,9 @@ Serializing a `Transaction` of variant `Version1`: - variant discriminator of value `2` - field `receipt` of type [Digest](./types.md#digest-digest) with serialization index `1` -### V1.payload.fields +#### Version1.payload.fields -This is an amorphous data holder. It is serialized as an [ordered collection](./primitives.md#list-clvalue-list) of [tuples](./primitives.md#tuple-clvalue-tuple) holding (u16, [Bytes](./types.md#bytes-bytes)). By u16 we understand an unsigned 2 byte number (see [numeric](./primitives.md#numeric-clvalue-numeric)). +This is an amorphous data holder. It is serialized as a [map](./primitives.md#map) of key `u16` and value `Byte`. By u16 we understand an unsigned 2 byte number (see [numeric](./primitives.md#numeric-clvalue-numeric)). `Byte` a [collection of bytes](./primitives.md#list-clvalue-list). The invariants for this field are: - keys (first tuple entries) are unique @@ -91,23 +92,12 @@ This field is design in this amorphous way to facilitate the possibility of resh _Currently_ the "keys" (first tuple entries) of the `fields` map are interpreted as: -- payload with key `0`: an instance of [`TransactionArgs`](#transactionargs) +- payload with key `0`: an instance of [`TransactionArgs`](./types.md#transactionargs-transaction-args) - payload with key `1`: an instance of [`TransactionTarget`](#transactiontarget) - payload with key `2`: an instance of [`TransactionEntryPoint`](#transactionentrypoint) - payload with key `3`: an instance of [`TransactionScheduling`](#transactionscheduling) -#### TransactionArgs - -`TransactionArgs` (serialized using calltable scheme) consists of: - -- in variant `Named` - - variant discriminator of value `0` - - [RuntimeArgs](./types.md#runtimeargs-runtimeargs) with serialization index `1` -- in variant `Bytesrepr` - - variant discriminator of value `1` - - [Bytes](./types.md#bytes-bytes) with serialization index `1` - -#### TransactionTarget +##### TransactionTarget `TransactionTarget` (serialized using calltable scheme) consists of: @@ -123,7 +113,7 @@ _Currently_ the "keys" (first tuple entries) of the `fields` map are interpreted - field `module_bytes` of type `Bytes` with serialization index `2` - field `runtime` of type [TransactionRuntimeParams](#transactionruntimeparams) with serialization index `2` -#### TransactionInvocationTarget +##### TransactionInvocationTarget `TransactionInvocationTarget` (serialized using calltable scheme) consists of: @@ -137,12 +127,14 @@ _Currently_ the "keys" (first tuple entries) of the `fields` map are interpreted - variant discriminator of value `2` - field `addr` which is a 32 bytes hash digest with serialization index `1` - field `version` of type `Option` ([option](./primitives.md#option-clvalue-option) of 4 bytes unsigned [number](./primitives.md#numeric-clvalue-numeric)) with serialization index `2` + - field `protocol_version_major` of type `u32` (4 bytes unsigned [number](./primitives.md#numeric-clvalue-numeric)). It's serialization index is `3`. This field is not mandatory - in fact in the reference rust implementation it is defined as `Option`. If it is present it should be serialized as `u32` under index `3`. But if it's not present, there will be no entry for it at all (both in the calltable section of `TransactionInvocationTarget` or payload) - in variant `ByPackageName` - variant discriminator of value `3` - field `name` of type `String` with serialization index `1` - field `version` of type `Option` ([option](./primitives.md#option-clvalue-option) of 4 bytes unsigned [number](./primitives.md#numeric-clvalue-numeric)) with serialization index `2` + - field `protocol_version_major` of type `u32` (4 bytes unsigned [number](./primitives.md#numeric-clvalue-numeric)). It's serialization index is `3`. This field is not mandatory - in fact in the reference rust implementation it is defined as `Option`. If it is present it should be serialized as `u32` under index `3`. But if it's not present, there will be no entry for it at all (both in the calltable section of `TransactionInvocationTarget` or payload) -#### TransactionRuntimeParams +##### TransactionRuntimeParams `TransactionRuntimeParams` (serialized using calltable scheme) consists of: @@ -153,7 +145,7 @@ _Currently_ the "keys" (first tuple entries) of the `fields` map are interpreted - field `transferred_value` of type `u64` (8 bytes unsigned [number](./primitives.md#numeric-clvalue-numeric)) with serialization index `1` - field `seed` of type `Option<[u8; 32]>` ([option](./primitives.md#option-clvalue-option) of 32 bytes [Byte array](./primitives.md#bytearray-clvalue-bytearray)) with serialization index `2` -#### TransactionEntryPoint +##### TransactionEntryPoint `TransactionEntryPoint` (serialized using calltable scheme) consists of: @@ -185,9 +177,17 @@ _Currently_ the "keys" (first tuple entries) of the `fields` map are interpreted - in variant `Burn`: - variant discriminator of value `12` -#### TransactionScheduling +##### TransactionScheduling `TransactionScheduling` (serialized using calltable scheme) consists of: - in variant `Standard`: - variant discriminator of value `0` + +## Serialization example + +Please see [this document](./transaction-serialization-example.md) to see an in-depth step-by-step example of deserializing a transaction + +## Deserialization example + +Please see [this document](./transaction-deserialization-example.md) to see an in-depth step-by-step example of deserializing a transaction diff --git a/versioned_docs/version-2.0.0/concepts/serialization/types.md b/versioned_docs/version-2.0.0/concepts/serialization/types.md index f038c7242..71e65aebd 100644 --- a/versioned_docs/version-2.0.0/concepts/serialization/types.md +++ b/versioned_docs/version-2.0.0/concepts/serialization/types.md @@ -898,3 +898,11 @@ A purse used for unbonding, replaced in 1.5 by [UnbondingPurse](#unbondingpurse) - `amount` The unbonding amount, serialized as a [`U512`](./primitives.md#clvalue-numeric) value. + +## TransactionArgs {#transaction-args} + +Arguments passed to execution of a `Transaction::Version1`, serialized as a `u8` identifying tag followed by additional bytes as follows: + +- `Named`: Serializes as a `u8` tag of 0 followed by [`RuntimeArgs`](#runtimeargs). + +- `Bytesrepr`: Serializes as a `u8` tag of 1 followed by [`bytes`](#bytes). \ No newline at end of file