Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-hop packet routing in IBC #138

Open
cwgoes opened this issue Jun 25, 2019 · 18 comments

Comments

@cwgoes
Copy link
Collaborator

commented Jun 25, 2019

Followup from #126.

Think about multi-hop case, prepare IBC protocol for eventual usage as such.

  • Construct data structures accordingly(e.g. channels are end-to-end)
  • Write a trivial routing module which can handle loopback, and has the right interface for multi-hop
  • Currently fail on attempts to open multi-hop channels

@cwgoes cwgoes self-assigned this Jun 25, 2019

@cwgoes cwgoes referenced this issue Jun 25, 2019
0 of 10 tasks complete

@cwgoes cwgoes changed the title ICS 6: IBC routing module Multi-hop packet routing in IBC Jun 25, 2019

@cwgoes cwgoes added this to the Specification Clarification milestone Jun 25, 2019

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 25, 2019

Types of multi-hop routing:

IMG_20190611_152814969_HDR

  1. "Application-layer multihop"
    A talks to H, H talks to B
    No routing at IBC protocol layer, just two connections / channels.
    We can do this already.
  2. "Validity proxy multihop"
    A talks to B using H's validity predicate for B
    A talks to H • B , B talks to H • A
    A verifies the state of B using the root + block height validated by H.
    Connection and channel from A to B using proxy validity predicates.
    This is relatively easy, just a different type of client.
  3. "Routing layer multihop"
    A has a connection to H, H has a connection to B.
    A has a channel to B through H.
    A verifies the state of H, H verifies the state & block height of B, vice versa.
    This will require a routing module in the IBC core protocol.

Hops could be called "links".

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 25, 2019

What we need to think about immediately is just what data structure alterations will be required for (3).

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 25, 2019

  • Channel opening handshake will specify a list of connection hops instead of a single connection
  • Packets will specify a list of connection hops instead of single origination & destination connections
  • An "IBC routing" module will differentiate packets intended for this chain and packets intended for forwarding, and in the latter case pop the current hop off the hop list and forward the packet to the next hop
  • This routing module will also handle a special sentinel loopback connection from a chain to itself
  • Channel semantics are end-to-end, so timeouts will need to be conveyed through intermediary hops (this may require extra "packet types")
@zmanian

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

Some things confuse me about the multihop use case.

If the system is end to end, do I show validaty proofs in consensus for all the intermediary hops to the end hops or just the validity proofs?

Can we think of this as wrapping an IBC packet inside another IBC packet?

I can sort of imagine in two ways.

In one way, effectively Alice and Charlie have a channel but no connection. The connection is provided by Bob'ss connection to Alice and Bob's connection to Charlie.

Must packets in that channel be source routed so that they say they are only valid if they have originated over the specified connections?

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 25, 2019

If the system is end to end, do I show validaty proofs in consensus for all the intermediary hops to the end hops or just the validity proofs?

In model (3), the verification steps for a packet from A to B through H would be as follows:

  • A commits outgoing packet to H with routing information indicating a next-hop from H to B
  • H receives incoming packet, checks that A committed outgoing correctly
  • H parses packet routing metadata (but not the packet data itself), identifies next-hop to B
  • H strips the first hop from the routing metadata and commits the packet outgoing to B
  • B verifies that H committed the outgoing packet (which is tagged as being originally-from A)
  • B processes accordingly (e.g. mints token vouchers)

I believe this is the first way you are describing (though I am not quite sure which was the second way), and once the handshake takes place (the handshake would involve A, B, and H for such a channel, and create a routing entry in a routing table on H) that validity checking will take place.

@ancazamfir

This comment has been minimized.

Copy link

commented Jun 27, 2019

  • Channel opening handshake will specify a list of connection hops instead of a single connection

How does A know the connections/ hops to reach B?

  • Packets will specify a list of connection hops instead of single origination & destination connections

Is this needed if the channel is "pinned" to a sequence of connections?

  • An "IBC routing" module will differentiate packets intended for this chain and packets intended for forwarding, and in the latter case pop the current hop off the hop list and forward the packet to the next hop

When a channel is setup by A, the "IBC routing" module on H determines the next hop/ connection towards the destination B. At this point, if we follow a typical routing implementation, a "forwarding" entry is set so a simple lookup for future packets destined to B will be performed. In the presence of pre-established channels I don't understand why the packets need to be source routed.
I know this is not planned any time soon but I believe it would be useful to think over a more complex topology of blockchains/ hubs.

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 27, 2019

How does A know the connections/ hops to reach B?

A doesn't know anything in particular (although it may have existing connections). The user or module creating the channel must know the connections which will be used.

Is this needed if the channel is "pinned" to a sequence of connections?

I think so - just for lookup purposes, not semantically. Alternatively we could decide that channel names are unique on a single chain and avoid this.

When a channel is setup by A, the "IBC routing" module on H determines the next hop/ connection towards the destination B. At this point, if we follow a typical routing implementation, a "forwarding" entry is set so a simple lookup for future packets destined to B will be performed. In the presence of pre-established channels I don't understand why the packets need to be source routed.
I know this is not planned any time soon but I believe it would be useful to think over a more complex topology of blockchains/ hubs.

I am not quite sure I follow. Are you suggesting packet-switched routing? I don't think that makes sense for IBC because the security assumptions change depending on the route - source routing enables the sender to choose all the hops and thus all the security assumptions.

@ancazamfir

This comment has been minimized.

Copy link

commented Jun 27, 2019

I am not quite sure I follow. Are you suggesting packet-switched routing? I don't think that makes sense for IBC because the security assumptions change depending on the route - source routing enables the sender to choose all the hops and thus all the security assumptions.

Aren't packets over a channel following the same route as the channel itself? why do they need again the source route?

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 27, 2019

Aren't packets over a channel following the same route as the channel itself? why do they need again the source route?

They need the connection identifiers to distinguish between channels which might have the same name but use different connections.

@ancazamfir

This comment has been minimized.

Copy link

commented Jun 27, 2019

They need the connection identifiers to distinguish between channels which might have the same name but use different connections.

I understand this but I have a feeling the intermediate connection info is redundant and only source and destination connections are required; since the source route has been previously passed in during channel handshake the next hop/ connection should be known by intermediate nodes when they look up the destination channel ID+connection.

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 27, 2019

I understand this but I have a feeling the intermediate connection info is redundant and only source and destination connections are required; since the source route has been previously passed in during channel handshake the next hop/ connection should be known by intermediate nodes when they look up the destination channel ID+connection.

Agreed, that would probably work, the relevant routes could be stored in a routing table and should be unique by channel and source connection.

@ancazamfir

This comment has been minimized.

Copy link

commented Jun 28, 2019

Some thoughts/ questions on the channel handshake:
The Try (A->B), Ack (A<-B) and Confirm(A->B) messages must also include the list of connections, correct? And since they should show in reverse order in the Ack, maybe there shouldn't be any popping done when Try is sent over the sequence of connections (so B can construct the reverse path).
Or if there is, a reverse path should be constructed (push the popped connection to some reverse route metadata) and sent as part of the handshake message as it is forwarded by each hop.

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 7, 2019

Data structure changes for future compatibility:

  • Channels are created with list of connection "hops" instead of source / dest connection identifiers.
  • Channel opening handshake datagrams altered to contain these hops.
  • Currently, trying to open a channel with more than one hop fails.

Is anything else required? Packet metadata can change later without an issue.

@ancazamfir

This comment has been minimized.

Copy link

commented Jul 8, 2019

Loopback will be handled when multi-hop is supported?

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 8, 2019

Loopback will be handled when multi-hop is supported?

Ah, let's do that now (no reason not to), excellent point.

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 9, 2019

What needs to be in the store key? @mossid asked.

@cwgoes - I think we can keep it as is, since handshake will set up hops.

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 9, 2019

Alternatively, channel identifiers could be unique per-port - that seems to make more sense...

Then the channel key = port id || channel id

@cwgoes cwgoes removed this from the IBC "1.0" milestone Jul 16, 2019

@cwgoes cwgoes added the ibc-v1.1 label Aug 17, 2019

@cwgoes

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 17, 2019

Note also points raised in #65 (comment) (closed in favor of this issue).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.