Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Probing/exchange rates are not a STREAM concern #112

Open
kincaidoneil opened this issue Apr 8, 2019 · 9 comments

Comments

Projects
None yet
4 participants
@kincaidoneil
Copy link
Collaborator

commented Apr 8, 2019

Proposal: Allow logic enforcing minimum exchange rates to be completely disabled, including:

  1. Sending test packets before the stream is established
  2. Requiring packets to meet the minimum exchange rate that STREAM calculates

  1. Supporting receive-only mode.
    • Case 1: credit with connector is 0, which currently causes all test packets to fail and stream to be closed, even if the endpoint won't be sending money.
    • Case 2: there exists credit with the connector, but the probing temporarily exhausts it, which is unnecessary and can cause disruptions for other applications.
  2. Insecure. Exchange rates from probing can trivially be gamed unless there was a TON of liquidity and many connectors, so for the foreseeable future, most clients necessitate their own exchange rate oracle to ensure the rate is reasonable.
    • In practice, the current behavior only really ensures the exchange rate stays consistent/within the slippage margin throughout the stream, which isn't desirable since it should respond to exchange rate fluctuations. (By contrast, the price oracle approach can respond to fluctuations, if necessary).
    • Instead, a callback function could be passed to STREAM clients & servers, that given a source amount and a destination amount, could return true/false if that particular packet should be fulfilled. (If it returned false, STREAM would reject that packet). This would allow consumers of the module to integrate with their preferred price oracle or use their own method to enforce a minimum exchange rate on a per-packet basis.
  3. Longer connection establishment. Probing wastes time while establishing the connection (#44), when alternatively it could be used to start sending money immediately, since the test packets can trivially be identified and gamed.
  4. No support for large asset scale differences. Probing with no knowledge of the relative exchange rate can introduce problems with vastly different exchange rate scales. (And requires sending massive test packets, which as noted earlier, could temporarily exhaust bandwidth that other applications are using, if they're forwarded).

To determine the initial packet size to send, STREAM could send a single very large packet (e.g., maybe the maximum uint64 an ILP packet allows the total amount remaining to send). F08s (which connectors should be configured to trigger before T04s) should quickly reduce it to a reasonable amount. (Alternatively, STREAM could be more aware of settlement/how much credit exists with the upstream peer, but I'm guessing that'd be more contentious).

@emschwartz

This comment has been minimized.

Copy link
Member

commented Apr 8, 2019

Having a way to disable the probing and set the prepareAmount to 0 sounds good to me.

Instead, a callback function could be passed to STREAM clients & servers

The client doesn't know the amount that will actually arrive at the destination beforehand and the server would need another frame added to the protocol to know the source amount the sender sent. The original idea was to put this logic in the hands of the sender (so they could probe, use a price feed like you describe, or any other method) and make the receiver respect their wishes independent of the logic they use (with a simple check of the prepare amount against the desired amount configured in the stream packet).

@kincaidoneil

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 8, 2019

Instead, a callback function could be passed to STREAM clients & servers

The client doesn't know the amount that will actually arrive at the destination beforehand and the server would need another frame added to the protocol to know the source amount the sender sent. The original idea was to put this logic in the hands of the sender (so they could probe, use a price feed like you describe, or any other method) and make the receiver respect their wishes independent of the logic they use (with a simple check of the prepare amount against the desired amount configured in the stream packet).

Oh, right, right. Instead, the callback could return that minimum acceptable amount that the sender of the payment sets, e.g. minDestinationAmount: (sourceAmount: BigNumber) => BigNumber (it'd just allow the consumer to set that on a per-packet basis rather than Stream on a per-connection basis).

@traviscrist

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2019

To determine the initial packet size to send, STREAM could send a single very large packet (e.g. the total amount remaining to send). F08s (which connectors should be configured to trigger before T04s) should quickly reduce it to a reasonable amount. (Alternatively, STREAM could be more aware of settlement/how much credit exists with the upstream peer, but I'm guessing that'd be more contentious).

Won't this just result in the same issue we have with #44. Starting with a large real amount and then waiting for T04s to back off could lead to an even longer time till money is sent. It would just half each packet until it gets a small enough amount. This could take many round trips to establish.

If asset scales are different enough I will have to get to sending 1 so the receiver gets 100 or 1000.

What do you propose as the start size of this packet?

If that amount is too small say 1,000,000,000 and only 1 gets there how do you deal with that? (there is very little precision)

@kincaidoneil

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 9, 2019

Won't this just result in the same issue we have with #44. Starting with a large real amount and then waiting for T04s to back off could lead to an even longer time till money is sent. It would just half each packet until it gets a small enough amount. This could take many round trips to establish.

Good point. I guess I'd be fine keeping the volley of packets and just making them all fulfillable (and then just ensuring they total less than the amount remaining to send). However, that might require refactoring to support multiple packets in flight (?).

If that amount is too small say 1,000,000,000 and only 1 gets there how do you deal with that? (there is very little precision)

The nice thing about this approach is it lets the consumer deal with how much precision they're willing to accept. If the callback is called with 1,000,000,000 as the source amount, and the sender calculates 1.4056 as the minimum destination amount (for example), they could just round that up to 2. (If Stream could send a packet for larger than 10^9, presumably it would've already tried to send that, right?). In this case, the Stream would likely fail, because the receiver would probably receive packets of amount 1, to be rejected, and not 2 -- which should emulate the current behavior.

The one downside is it might take the entire timeout to fail, rather than fail pretty quickly.

@sentientwaffle

This comment has been minimized.

Copy link
Contributor

commented Apr 9, 2019

Exchange rates from probing can trivially be gamed unless there was a TON of liquidity and many connectors ...

Can you elaborate on this?


Also, how do you imagine the price oracle function would work if not with probe packets? Would it effectively be like a connector's rate backend with some slippage tacked on to account for the route?

@kincaidoneil

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 9, 2019

Exchange rates from probing can trivially be gamed unless there was a TON of liquidity and many connectors ...

Can you elaborate on this?

As a connector, I look for packets with an amount of 1,000,000,000,000 (the first test packet). If I see one, I know that it's more than likely a probing packet for a STREAM connection. I also know the specific connection tag from the destination address. If any subsequent packets are sent with that connection tag, I'll just take a much larger slippage margin than I typically would. If the sender & receiver have no other knowledge of the exchange rate other than the probing packets sent through me, they'll start fulfilling packets over it, enabling me to keep taking that higher slippage margin. Other packets would be unaffected, since I'm only doing it for their connection tag.

But, if the sender had multiple uplink plugins, Stream could probe using several of them to ensure the exchange rate was not manipulated, and then just use the uplink connector with the most competitive rate. (However, the receiver's direct downlink connector could still be manipulating the rate).

To make probing alone secure, I think it'd require:

  1. Enough liquidity and connectors so there are several wholly independent paths the packet could take from sender to receiver
  2. The ability for the receiver to have multiple downlink plugins (and therefore multiple destination addresses) linked to the same shared secret, so they could correlate packets for a single Stream connection received from different plugins

Also, how do you imagine the price oracle function would work if not with probe packets? Would it effectively be like a connector's rate backend with some slippage tacked on to account for the route?

Yep! Just like the connector's rate backend, minus slippage. In fact, in Switch/ilp-sdk, we're using the very same rate backend on the client as we do on our connector (using crypto-rate-utils, which uses CoinCap as its oracle).

@sentientwaffle

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2019

I'm not a fan of requiring a rate backend in the sender or receiver.
From interledger's design goals:

Interoperability - ILP should be usable across any type of ledger, even those that were not built for interoperability.

Requiring that the destination currency be known by the sender (in the rate backend) hinders interoperability.

btw, the stream Connection has the minimumAcceptableExchangeRate method that seems relevant. If the sender is aware of the destination currency, they could sanity-check the actual exchange rate against an expected one.

@kincaidoneil

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 10, 2019

Requiring that the destination currency be known by the sender (in the rate backend) hinders interoperability.

I kinda agree, but in any case, this wouldn't be the default. I like the probing model theoretically, but at this stage in the network, I just don't think it's secure in practice. (Also, didn't the ConnectionAssetDetails frame already kinda break that principle?)

btw, the stream Connection has the minimumAcceptableExchangeRate method that seems relevant. If the sender is aware of the destination currency, they could sanity-check the actual exchange rate against an expected one.

Yep. We've used that in the past, but what I'm proposing would give you control over that on a per-packet basis, rather than per-connection -- it exposes much more control to the consumer. And since probing has introduced issues for us with receive only mode (among the other grievances mentioned), it'd just be nice to have the ability to disable that entirely.

@traviscrist

This comment has been minimized.

Copy link
Contributor

commented Apr 10, 2019

The main purpose of the probing is to establish a connection where the path has a minimum precision before sending fulfillable packets. It also helps discover large asset scale differences without risking funds. I'd like to get rid of the probing but I don't have a better idea on establishment if you care about the exchange rate and precision.

  1. Supporting a receive only mode makes sense, I see the use case for allowing this and disabling the current connection establishment process and need for an exchange rate.

  2. Allowing a callback to be passed into STREAM which can have custom exchange rate logic seems like a good idea. I fail to see how this can do things on the fly very well though. This would not know the asset classes unless the optional ConnectionAssetDetails frame is sent. In general ILP does not care about the asset class by design.

  3. I don't think that is the case unless you get lucky and the first packet is fulfilled. I see 2-3 packets needed at minimum even if fullfillable. First gets rejected with an F08, the second is then likely a T04 and then the third finally makes it.

Also how is this first packet amount determined? It is a random number between some values so it cannot be guessed?

  1. This proposal would also not have support for large scale asset differences. If the first packet fails to fulfill since the receivers scale is magnitudes larger. How does the STREAM client know to increase their fulfill packet amount? There's no amount too small code...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.