Move `UtxoId` from `fuel-tx` to `fuel-types` #51

xgreenx · 2022-09-22T15:49:39Z

In the fuel-core we have custom AsRef<[u8]> for UtxoId. We represent UtxoId as 33 bytes and use it as a key for data.

fuel-tx serialize it as 40 bytes where 32 bytes it TxId and 8 bytes are output_index: u8. It is because each separate field in the struct should be WORD_SIZE during serialization in the fuel-tx.

We can represent UtxoId as 33 bytes and know that the last byte is an output_index. We don't need to make it a separate field. UtxoId can provide two getter methods for tx_id and output_index.

It is breaking change but:

It reduces the size of all transactions (instead of 40 bytes, we need 33 bytes).
Makes UtxoId the same as other ids(ContractId, MessageId).
We have one serialization model in fuel-tx and fuel-core.

Voxelot · 2022-09-22T16:42:32Z

This is 40 bytes when doing canonical serialization in the vm so that all types are 64bit aligned. Word alignment isn't required when storing to disk though.

xgreenx · 2022-09-22T16:51:26Z

Where exactly do we need 64-alignment? (Maybe you can put the link on the code)
We also have Base4 and Base20 that are not 64-bit aligned.

Voxelot · 2022-09-22T17:56:42Z

The word alignment is required here when initializing the VM memory: https://github.com/FuelLabs/fuel-vm/blob/v0.18.2/src/interpreter/initialization.rs#L36-L47

vlopes11 · 2022-09-22T17:55:02Z

src/types.rs

+                    *i = rng.gen();
+                }
+
+                default


Why would we want to generate bytes individually instead of delegating the implementation to R so he is aware we are in fact generating n bytes?

Because Distribution is not implemented for unusual arrays as [u8; 33].

vlopes11 · 2022-09-22T17:55:44Z

src/types.rs

        /// FuelVM atomic type.
        pub struct $i([u8; $s]);

+        impl Default for $i {
+            fn default() -> $i {
+                $i([0u8; $s])


Why remove the proc macro default implementation to add more code that will do the same - but manually instead of relying in the language defaults?

Because Default is not implemented for unusual arrays as [u8; 33].

vlopes11 · 2022-09-22T17:55:57Z

src/types.rs

@@ -24,16 +24,29 @@ const fn hex_val(c: u8) -> Option<u8> {

 macro_rules! key {
    ($i:ident, $s:expr) => {
-        #[derive(Default, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
+        #[repr(transparent)]


why adding transparent here is required?

Because UtxoId actually has two parts, 32 bytes and 1 byte, it can be helpful if someone wants to mem::transmute part of the 33 bytes array into TxId or output_index.

I don't think its a good idea to mix the output index with the bytes representation. If its a matter of encoding / decoding, we can surely optimize this there but then not in the type representation and memory layout

It is not a matter of encoding/decoding. UtxoId is an identifier as MessageId and ContractId, so it should have all methods available for other identifiers.

If we use representation:

pub struct UtxoId([u8; 32], u8);

We need to implement all traits manually. But UtxoId(Bytes32, u8) equevalent to UtxoId([u8; 33]). AsRef<[u8]> implementation for UtxoId(Bytes32, u8) requires usage of unsafe { mem::transmute } when it is natively available for UtxoId([u8; 33]).

Also, information about the output index matters during the creation of the UtxoId, but after it is used not often.

I checked our implementation of "unsafe" methods, and it is required for memory layout correctness in our unsafe code

vlopes11 · 2022-09-22T17:56:26Z

src/types.rs

@@ -271,6 +284,9 @@ key!(Bytes20, 20);
 key!(Bytes32, 32);
 key!(MessageId, 32);
 key!(Salt, 32);
+key!(UtxoId, 33);
+
+pub type TxId = Bytes32;


Why not follow the pattern of the other types instead of aliasing?

I also think so, but in the fuel-tx it is an alias. I moved it from fuel-tx here. But I also for the creation of a separate type. If you are okay with it(it is a breaking change), I will do it=)

vlopes11 · 2022-09-22T17:57:33Z

src/types.rs

+            r[8], r[9], r[10], r[11], r[12], r[13], r[14], r[15],
+            r[16], r[17], r[18], r[19], r[20], r[21], r[22], r[23],
+            r[24], r[25], r[26], r[27], r[28], r[29], r[30], r[31],
+        ];


I'm not sure why to do it like this. Its harder to read/understand without a clear gain

Because slice_copy is not const operation=)
So I can do copy only manually byte by byte=(

vlopes11 · 2022-09-22T18:00:07Z

src/types.rs

+    }
+
+    pub const fn output_index(&self) -> u8 {
+        self.0[TxId::LEN]


This whole behavior looks like some magic. Instead of having hardcoded positions in the array, maybe we want to create a field with the output index?

The purpose of this change is to represent the UtxoId as an array of 33 bytes=) Because on the fuel-core level, we work with arrays and slices.

It can be a struct with two fields, but then in the AsRef<[u8]> implementation, we need to use unsafe{ mem::transmute } to convert &UtxoId into &[u8; 33] or &[u8].

I don't think it's a good approach to let serialization concerns affect how our core domain types are structured. It's a bit like the tail wagging the dog tbh.

On the other hand, copying these utxoIds every time we go to storage isn't great either.

Maybe we should consider using a vetted zerocopy lib instead of implementing transmute ourselves? e.g. https://docs.rs/zerocopy/latest/zerocopy/trait.AsBytes.html

Did an alter implementation #52

For me, the approach with [u8; 33] looks cleaner=)

vlopes11 · 2022-09-23T12:12:06Z

src/types.rs

@@ -24,16 +24,29 @@ const fn hex_val(c: u8) -> Option<u8> {

 macro_rules! key {
    ($i:ident, $s:expr) => {
-        #[derive(Default, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
+        #[repr(transparent)]


I don't think its a good idea to mix the output index with the bytes representation. If its a matter of encoding / decoding, we can surely optimize this there but then not in the type representation and memory layout

Added UtxoId

72554e9

xgreenx added the breaking A breaking api change label Sep 22, 2022

xgreenx requested review from Voxelot, vlopes11 and kayagokalp September 22, 2022 15:49

xgreenx self-assigned this Sep 22, 2022

Fix fmt

c83a394

xgreenx mentioned this pull request Sep 22, 2022

Moved UtxoId to fuel-types FuelLabs/fuel-tx#187

Open

vlopes11 suggested changes Sep 22, 2022

View reviewed changes

vlopes11 suggested changes Sep 23, 2022

View reviewed changes

This was referenced Sep 24, 2022

Introduced two new traits for spec's serialization and deserialization FuelLabs/fuel-tx#190

Draft

Some improvements during work with the database FuelLabs/fuel-core#643

Closed

It is an alternative to the implementation of the UtxoId #52

Open

xgreenx mentioned this pull request Feb 3, 2023

Move UtxoId from fuel-tx to fuel-type and update to be the same with other types FuelLabs/fuel-vm#338

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move `UtxoId` from `fuel-tx` to `fuel-types` #51

Move `UtxoId` from `fuel-tx` to `fuel-types` #51

xgreenx commented Sep 22, 2022

Voxelot commented Sep 22, 2022

xgreenx commented Sep 22, 2022

Voxelot commented Sep 22, 2022 •

edited

Loading

vlopes11 Sep 22, 2022

xgreenx Sep 22, 2022

vlopes11 Sep 22, 2022

xgreenx Sep 22, 2022

vlopes11 Sep 22, 2022

xgreenx Sep 22, 2022

vlopes11 Sep 23, 2022

xgreenx Sep 23, 2022

xgreenx Sep 26, 2022

vlopes11 Sep 22, 2022

xgreenx Sep 22, 2022

vlopes11 Sep 22, 2022

xgreenx Sep 22, 2022

vlopes11 Sep 22, 2022

xgreenx Sep 22, 2022

Voxelot Sep 23, 2022

xgreenx Sep 26, 2022

vlopes11 Sep 23, 2022

Move UtxoId from fuel-tx to fuel-types #51

Are you sure you want to change the base?

Move UtxoId from fuel-tx to fuel-types #51

Conversation

xgreenx commented Sep 22, 2022

Voxelot commented Sep 22, 2022

xgreenx commented Sep 22, 2022

Voxelot commented Sep 22, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Move `UtxoId` from `fuel-tx` to `fuel-types` #51

Move `UtxoId` from `fuel-tx` to `fuel-types` #51

Voxelot commented Sep 22, 2022 •

edited

Loading