Add transaction's content verification

[samuelmanzanera]

Description

This PR improves data validation for transfer & hosting tx
For the hosting, this is based on the changes from archethic-foundation/aeweb-cli#87

Fixes #409

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[bchamagne]

It does not work because it tries to validate the data transaction with the same schema.
AEWebCLI's data transaction format did not change and looks like this:

%{
  "dir1" => %{
    "file10.txt" => "H4sIAAAAAAAAA0vLzElVMDTgAgALt2T5CAAAAA",
    "file11.txt" => "H4sIAAAAAAAAA0vLzElVMDTkAgBKhn_gCAAAAA"
  },
  "dir2" => %{
    "hello.txt" => "H4sIAAAAAAAAA_NIzcnJ1yupKFHQyM_LqVRIy8xJVcjMU0jJLDLS5AIAt5R0vB4AAAA"
  },
  "dir3" => %{
    "index.html" => "H4sIAAAAAAAAA7PJMLRz8QwyVsjMS0mt0Msoyc2x0QeKcQEAL-DoARkAAAA"
  },
  "file1.txt" => "H4sIAAAAAAAAA0vLzElVMFTgAgBapaazCAAAAA",
  "file2.txt" => "H4sIAAAAAAAAA0vLzElVMOICAMGeIz8HAAAA",
  "file3.txt" => "H4sIAAAAAAAAA0vLzElVMOYCAICvOCYHAAAA"
}

2022-12-06 14:07:05.340 [debug] Invalid AEWeb format [{"Required properties aewebVersion, metaData were not present.", "#"}]

@samuelmanzanera , I'll deal with this

[apoorv-2204]

It does not work because it tries to validate the data transaction with the same schema. AEWebCLI's data transaction format did not change and looks like this:

%{
  "dir1" => %{
    "file10.txt" => "H4sIAAAAAAAAA0vLzElVMDTgAgALt2T5CAAAAA",
    "file11.txt" => "H4sIAAAAAAAAA0vLzElVMDTkAgBKhn_gCAAAAA"
  },
  "dir2" => %{
    "hello.txt" => "H4sIAAAAAAAAA_NIzcnJ1yupKFHQyM_LqVRIy8xJVcjMU0jJLDLS5AIAt5R0vB4AAAA"
  },
  "dir3" => %{
    "index.html" => "H4sIAAAAAAAAA7PJMLRz8QwyVsjMS0mt0Msoyc2x0QeKcQEAL-DoARkAAAA"
  },
  "file1.txt" => "H4sIAAAAAAAAA0vLzElVMFTgAgBapaazCAAAAA",
  "file2.txt" => "H4sIAAAAAAAAA0vLzElVMOICAMGeIz8HAAAA",
  "file3.txt" => "H4sIAAAAAAAAA0vLzElVMOYCAICvOCYHAAAA"
}

2022-12-06 14:07:05.340 [debug] Invalid AEWeb format [{"Required properties aewebVersion, metaData were not present.", "#"}]

@samuelmanzanera , I'll deal with this

we need to use branch on this pr
#742

and we need to use aeweb cli branch from this pr
archethic-foundation/aeweb-cli#87

[bchamagne]

The issue was because the json schema was created for this data transaction format:

%{
  "dir1/file10.txt" => "H4sIAAAAAAAAA0vLzElVMDTgAgALt2T5CAAAAA",
  "dir1/file11.txt" => "H4sIAAAAAAAAA0vLzElVMDTkAgBKhn_gCAAAAA",
  "dir2/hello.txt" => "H4sIAAAAAAAAA_NIzcnJ1yupKFHQyM_LqVRIy8xJVcjMU0jJLDLS5AIAt5R0vB4AAAA",
  "dir3/index.html" => "H4sIAAAAAAAAA7PJMLRz8QwyVsjMS0mt0Msoyc2x0QeKcQEAL-DoARkAAAA",
  "file1.txt" => "H4sIAAAAAAAAA0vLzElVMFTgAgBapaazCAAAAA",
  "file2.txt" => "H4sIAAAAAAAAA0vLzElVMOICAMGeIz8HAAAA",
  "file3.txt" => "H4sIAAAAAAAAA0vLzElVMOYCAICvOCYHAAAA"
}

This format has been discussed but it is not implemented like this yet. Current format is:

%{
  "dir1" => %{
    "file10.txt" => "H4sIAAAAAAAAA0vLzElVMDTgAgALt2T5CAAAAA",
    "file11.txt" => "H4sIAAAAAAAAA0vLzElVMDTkAgBKhn_gCAAAAA"
  },
  "dir2" => %{
    "hello.txt" => "H4sIAAAAAAAAA_NIzcnJ1yupKFHQyM_LqVRIy8xJVcjMU0jJLDLS5AIAt5R0vB4AAAA"
  },
  "dir3" => %{
    "index.html" => "H4sIAAAAAAAAA7PJMLRz8QwyVsjMS0mt0Msoyc2x0QeKcQEAL-DoARkAAAA"
  },
  "file1.txt" => "H4sIAAAAAAAAA0vLzElVMFTgAgBapaazCAAAAA",
  "file2.txt" => "H4sIAAAAAAAAA0vLzElVMOICAMGeIz8HAAAA",
  "file3.txt" => "H4sIAAAAAAAAA0vLzElVMOYCAICvOCYHAAAA"
}

Here's my suggestion to fix with current data transaction format: jsonschema.patch.tar.gz

[apoorv-2204]

The issue was because the json schema was created for this data transaction format:

%{
  "dir1/file10.txt" => "H4sIAAAAAAAAA0vLzElVMDTgAgALt2T5CAAAAA",
  "dir1/file11.txt" => "H4sIAAAAAAAAA0vLzElVMDTkAgBKhn_gCAAAAA",
  "dir2/hello.txt" => "H4sIAAAAAAAAA_NIzcnJ1yupKFHQyM_LqVRIy8xJVcjMU0jJLDLS5AIAt5R0vB4AAAA",
  "dir3/index.html" => "H4sIAAAAAAAAA7PJMLRz8QwyVsjMS0mt0Msoyc2x0QeKcQEAL-DoARkAAAA",
  "file1.txt" => "H4sIAAAAAAAAA0vLzElVMFTgAgBapaazCAAAAA",
  "file2.txt" => "H4sIAAAAAAAAA0vLzElVMOICAMGeIz8HAAAA",
  "file3.txt" => "H4sIAAAAAAAAA0vLzElVMOYCAICvOCYHAAAA"
}

This format has been discussed but it is not implemented like this yet. Current format is:

%{
  "dir1" => %{
    "file10.txt" => "H4sIAAAAAAAAA0vLzElVMDTgAgALt2T5CAAAAA",
    "file11.txt" => "H4sIAAAAAAAAA0vLzElVMDTkAgBKhn_gCAAAAA"
  },
  "dir2" => %{
    "hello.txt" => "H4sIAAAAAAAAA_NIzcnJ1yupKFHQyM_LqVRIy8xJVcjMU0jJLDLS5AIAt5R0vB4AAAA"
  },
  "dir3" => %{
    "index.html" => "H4sIAAAAAAAAA7PJMLRz8QwyVsjMS0mt0Msoyc2x0QeKcQEAL-DoARkAAAA"
  },
  "file1.txt" => "H4sIAAAAAAAAA0vLzElVMFTgAgBapaazCAAAAA",
  "file2.txt" => "H4sIAAAAAAAAA0vLzElVMOICAMGeIz8HAAAA",
  "file3.txt" => "H4sIAAAAAAAAA0vLzElVMOYCAICvOCYHAAAA"
}

Here's my suggestion to fix with current data transaction format: jsonschema.patch.tar.gz

we will proceed with nested json only.

[apoorv-2204]

archethic-foundation/aeweb-cli#87
This pr doesnt impl key value pair.It uses nested json struct with metaData props

[bchamagne]

Hello, I did the quick fix after the big merge of all AEWeb PRs.
This is ready to merge !

Tested with AEWEB CLI

[samuelmanzanera]

Hello, I did the quick fix after the big merge of all AEWeb PRs. This is ready to merge !

Tested with AEWEB CLI

Thanks 👍