feat: add in-transit XCM msgs in blocks endpoint #1412

Imod7 · 2024-03-12T08:26:05Z

Description

This pull request introduces the in transit horizontal messages within the Relay chain.

Closes issue #732 since this is the remaining part. First part was completed in the #1364 PR.

Sample Responses

Request
http://127.0.0.1:8080/blocks/17792973?decodedXcmMsgs=true

while connected to Kusama Relay Chain

Response
The item decodedXcmMsgs now shows also the in transit Horizontal Messages as shown below :

"decodedXcmMsgs": {
    "horizontalMessages": [
      {
        "originParaId": "2084",
        "destinationParaId": "2000",
        "data": [
          {
            "v2": [
              {
                "withdrawAsset": {
          ....
           },
      {
        "originParaId": "2110",
        "destinationParaId": "2114",
        "data": [
          {
            "v2": [
              {
                "withdrawAsset": [
          ....
             ]
      }
    ],
    "downwardMessages": [],
    "upwardMessages": []

This block has 2 in transit Horizontal Messages with different origin and different destination paraId.

Request
http://127.0.0.1:8080/blocks/21943991?decodedXcmMsgs=true

while connected to Kusama Relay Chain

Response
The item decodedXcmMsgs shows 2 in transit Horizontal Messages with same origin and same destination paraId:

"decodedXcmMsgs": {
    "horizontalMessages": [
      {
        "originParaId": "2001",
        "destinationParaId": "2023",
        "data": [
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                   ....
          },
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                    ....
      }
    ],
    "downwardMessages": [],
    "upwardMessages": []
  }

*** Same response structure when connected to a Parachain, e.g. block 5646401 from Polkadot Asset Hub

Request
http://127.0.0.1:8080/blocks/22238422?decodedXcmMsgs=true

while connected to Kusama Relay Chain

Response
The item decodedXcmMsgs shows 3 Upward Messages from same origin paraId :

 "decodedXcmMsgs": {
    "horizontalMessages": [],
    "downwardMessages": [],
    "upwardMessages": [
      {
        "originParaId": "2001",
        "data": [
          {
            "v3": [
              {
                "withdrawAsset": [
                 ....
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                   ....
          },
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                  ...
    ]
  }

Tested Cases with Example Blocks

Cases that were tested (while connected to a Kusama/Polkadot Relay chain):

Blocks with 1 Horizontal Message.
Blocks with 1 Horizontal and 1 Upward Message.
Blocks with 2 Horizontal Messages with same origin paraId and different destination paraId.
Blocks with 2 Horizontal Messages with different origin and different destination paraId.
Blocks with multiple Upward Messages with same origin paraId .

*** Above cases were also tested with the query param paraId

All examples tested can be found here

Changelog (with Fixes - Breaking Changes)

❗In Upward Messages of Relay, data is now an array since multiple XMC messages from same paraId can be retrieved.
❗The structure/shape (and consequently the associated interface) of Horizontal Messages in the Relay Chain differs from that of Horizontal Messages in Parachain. As a result:
- IHorizontalMessageInRelayChain interface was introduced.
- IHorizontalMessage was renamed to IHorizontalMessageInParachain to provide clearer distinction.
❗The field paraId was renamed to originParaId in the following interfaces / responses:
- IHorizontalMessageInParachain
- IUpwardMessage
  to explicitly indicate that it refers to the origin paraId rather than the destination. It also ensures consistency with the IHorizontalMessageInRelayChain interface.
The code has been refactored:
- when checking for Upward and Horizontal Messages in Relay and
- when checking for Horizontal Msgs in Parachain
  to avoid repetitive blocks of code
Corrected ISanitizedParachainInherentData so that it reflects how the information is retrieved.

Todos

Update Documentation

marshacb

Solid PR. LGTM!

IkerAlus

As discussed in the call, the output should something like:

When connected in Relay and I have 5 Horizontal Msgs like

4 from para 2010, 2 to para 2034, 1 to para 2031, 1 to para 2032.
1 from para 2020 to para 2032,

they should be shown as:

"decodedXcmMsgs": {
    "horizontalMessages": [
      {
        "sentAt": "blockNumber",
        "originId":"2010",
        "destinationId": "2034",
        "data":           {
            "v3": [
              {
                "withdrawAsset": [
                  {...
            ]
          },
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                 ...
           ]
        },
        {
        "sentAt": "blockNumber",
        "originId":"2010",
        "destinationId": "2031",
        "data": [
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                ...
        },
        {
        "sentAt": "blockNumber",
        "originId":"2010",
        "destinationId": "2032",
        "data": [
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                ...
        },
        {
        "sentAt": "blockNumber",
        "originId":"2020",
        "destinationId": "2032",
        "data": [
          {
            "v3": [
              {
                "withdrawAsset": [
                  {
                ...
          }
        ]
      }

Perhaps there are better choices for the naming of originId and destinationId but the structure should be like the one above.

TarikGul · 2024-03-13T12:31:36Z

src/services/blocks/XCMDecoder.ts

-									xcmMessages.upwardMessages?.push(msg_decoded);
-								}
+					data.backedCandidates.forEach((candidate) => {
+						if (paraId === undefined || candidate.candidate.descriptor.paraId.toString() === paraId.toString()) {


Suggested change

if (paraId === undefined || candidate.candidate.descriptor.paraId.toString() === paraId.toString()) {

if (!paraId || candidate.candidate.descriptor.paraId.toString() === paraId.toString()) {

TarikGul · 2024-03-13T12:32:05Z

src/services/blocks/XCMDecoder.ts

-								xcmMessages.horizontalMessages?.push(horizontalMessage);
-							} else if (paraId === undefined) {
-								horizontalMessage = {
+						if (paraId === undefined || index.toString() === paraId.toString()) {


Suggested change

if (paraId === undefined || index.toString() === paraId.toString()) {

if (!paraId || index.toString() === paraId.toString()) {

TarikGul · 2024-03-13T12:32:48Z

src/services/blocks/XCMDecoder.ts

+						: (msg as ISanitizedBackedCandidateHorizontalMessage).data.slice(1);
+				const xcmMessageDecoded: string = this.decodeMsg(api, msgData);
+
+				if (paraId === undefined || paraIdCandidate === paraId.toString()) {


Suggested change

if (paraId === undefined || paraIdCandidate === paraId.toString()) {

if (!paraId || paraIdCandidate === paraId.toString()) {

TarikGul

LGTM small nits

- added `IHorizontalMessageInRelayChain` interface - renamed `IHorizontalMessage` to `IHorizontalMessageInParachain`. - renamed `paraId` to `originParaId` in `IHorizontalMessageInParachain` and `IUpwardMessage` interface. - corrected `ISanitizedParachainInherentData` - updated tests

IkerAlus

LGTM, good job!

I see there is no "sentAt" for horizontal messages in the Relay chain but I think that in this context it is not so relevant, I am ok with leaving as it is now.

Also, consider the doc update so it is clear to the user what we are showing in each of the cases.

Imod7 · 2024-03-18T12:13:09Z

LGTM, good job!

I see there is no "sentAt" for horizontal messages in the Relay chain but I think that in this context it is not so relevant, I am ok with leaving as it is now.

Also, consider the doc update so it is clear to the user what we are showing in each of the cases.

Yes, there is no "sentAt" field available so that is why I didn't put it. Yes, definitely I will update the docs to make everything very clear and also the description of the PR with the latest changes. Thank you!

…originPara) - upward msgs response to be aligned with horizontal msgs response structure - added Tarik's suggestions

docs/src/openapi-v1.yaml

Co-authored-by: IkerAlus <34474035+IkerAlus@users.noreply.github.com>

feat: add in-transit XCM msgs in blocks endpoint

f6e401e

Imod7 requested a review from a team as a code owner March 12, 2024 08:26

marshacb approved these changes Mar 12, 2024

View reviewed changes

IkerAlus requested changes Mar 12, 2024

View reviewed changes

TarikGul reviewed Mar 13, 2024

View reviewed changes

TarikGul approved these changes Mar 13, 2024

View reviewed changes

IkerAlus approved these changes Mar 15, 2024

View reviewed changes

Imod7 added 3 commits March 19, 2024 15:25

changed response structure of upward msgs (multiple upward from same …

cf8ea13

…originPara) - upward msgs response to be aligned with horizontal msgs response structure - added Tarik's suggestions

updated docs

a4952b3

docs fix in schemas based on Iker's finding

25b3a52

IkerAlus reviewed Mar 21, 2024

View reviewed changes

docs/src/openapi-v1.yaml Outdated Show resolved Hide resolved

Imod7 and others added 2 commits March 21, 2024 10:53

Iker's grammar correction

f5effc9

Co-authored-by: IkerAlus <34474035+IkerAlus@users.noreply.github.com>

rebuilt docs

f8f260d

Imod7 merged commit 6028086 into master Mar 21, 2024
15 checks passed

Imod7 deleted the domi-xcm-transit branch March 21, 2024 10:08

Imod7 mentioned this pull request Mar 21, 2024

[Feature] Added query param to /blocks that will also include messages #732

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add in-transit XCM msgs in blocks endpoint #1412

feat: add in-transit XCM msgs in blocks endpoint #1412

Imod7 commented Mar 12, 2024 •

edited

marshacb left a comment

IkerAlus left a comment •

edited

TarikGul Mar 13, 2024

TarikGul Mar 13, 2024

TarikGul Mar 13, 2024

TarikGul left a comment

IkerAlus left a comment

Imod7 commented Mar 18, 2024

	if (paraId === undefined \|\| candidate.candidate.descriptor.paraId.toString() === paraId.toString()) {
	if (!paraId \|\| candidate.candidate.descriptor.paraId.toString() === paraId.toString()) {

	if (paraId === undefined \|\| index.toString() === paraId.toString()) {
	if (!paraId \|\| index.toString() === paraId.toString()) {

	if (paraId === undefined \|\| paraIdCandidate === paraId.toString()) {
	if (!paraId \|\| paraIdCandidate === paraId.toString()) {

feat: add in-transit XCM msgs in blocks endpoint #1412

feat: add in-transit XCM msgs in blocks endpoint #1412

Conversation

Imod7 commented Mar 12, 2024 • edited

Description

Sample Responses

Tested Cases with Example Blocks

Changelog (with Fixes - Breaking Changes)

Todos

marshacb left a comment

Choose a reason for hiding this comment

IkerAlus left a comment • edited

Choose a reason for hiding this comment

TarikGul Mar 13, 2024

Choose a reason for hiding this comment

TarikGul Mar 13, 2024

Choose a reason for hiding this comment

TarikGul Mar 13, 2024

Choose a reason for hiding this comment

TarikGul left a comment

Choose a reason for hiding this comment

IkerAlus left a comment

Choose a reason for hiding this comment

Imod7 commented Mar 18, 2024

Imod7 commented Mar 12, 2024 •

edited

IkerAlus left a comment •

edited