New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add application-level ACKs for online messages #483

Open
JustinDrake opened this Issue Mar 31, 2017 · 12 comments

Comments

Projects
None yet
5 participants
@JustinDrake
Copy link
Contributor

JustinDrake commented Mar 31, 2017

When sending a message (e.g. a purchase order) over an open websockets connection, is there a way to know if it's been successfully received? As far as I can tell, websockets is used as a frame stream. So it seems possible for an application-level message to be partially sent and for openbazaar-go to not know whether or not it's been successfully received in full.

Should we add an application-level ACK for online messages sent over websockets?

@hoffmabc

This comment has been minimized.

Copy link
Member

hoffmabc commented Mar 31, 2017

Did we have much of a problem with this in the Python websockets code @cpacia? It certainly reasonable to think this could happen, but I don't recall if we ever did anything about it or saw it happen that often in the previous version.

@JustinDrake

This comment has been minimized.

Copy link
Contributor Author

JustinDrake commented Mar 31, 2017

Thinking slightly more abstractly, go-libp2p is transport agnostic and will support other transports in the future beyond TCP and WebSockets (e.g. WebRTC, UTP?). So we can't rely on transport-level ACKs. This then leads to two possible ways to do ACKs:

  1. At the secio level. I don't thing secio has ACKs, but I'm not 100% sure.
  2. At the application level
@cpacia

This comment has been minimized.

Copy link
Member

cpacia commented Mar 31, 2017

I believe UTP has an ack. Not sure about webrtc.

@JustinDrake

This comment has been minimized.

Copy link
Contributor Author

JustinDrake commented Apr 1, 2017

It turns out that secio does not have an ACK mechanism. Also, there are several new libp2p transports in the pipeline (including WebRTC, UTP, UDP, SHS, QUIC, I2P). Some of these transports are bound to not have an ACK mechanism (e.g. UDP).

It seems to me that therefore openbazaar-go needs an ACK mechanism at the application-level. What do you think @cpacia?

@JustinDrake JustinDrake changed the title Are application-level ACKs required for websockets? Add application-level ACKs for online messages Apr 1, 2017

@cpacia cpacia added the discussion label Apr 3, 2017

@cpacia

This comment has been minimized.

Copy link
Member

cpacia commented Apr 14, 2017

I'm not terribly convinced on this one. Do we ever see enabling a transport without reliability built in? The only reason I would see using UDP is for hole punching reasons (though our current setup seems to be working fairly well). And even there UTP seems to be a better option than UDP. I've never worked with webRTC but I've read there's a reliable mode. Adding an additional ACK on top of the transport ACK not only adds protocol complexity but more network overhead.

@JustinDrake

This comment has been minimized.

Copy link
Contributor Author

JustinDrake commented Apr 14, 2017

Do we ever see enabling a transport without reliability built in?

I was under the impression that openbazaar-go was embracing libp2p. (Is that the case?) For sure libp2p will support unreliable transports. So I think the answer is yes, we do see enabling a transport without reliability build in.

protocol complexity

Yeah, proper protocol design comes at a cost. Note that we need to notify the counterparty of errors. If we do not ACK, the counterparty would have to wait (e.g. with a timeout) for a possible error notification. If the success/failure is returned with the ACK the counterparty can confidently move on and update it's state machine.

more network overhead.

Negligible.

@JustinDrake

This comment has been minimized.

Copy link
Contributor Author

JustinDrake commented Apr 14, 2017

Do we ever see enabling a transport without reliability built in?

On second thought openbazaar-go has already enabled a transport without reliability built in.

@cpacia

This comment has been minimized.

Copy link
Member

cpacia commented Apr 14, 2017

Websockets are built on top of TCP. There are ACKs and re-transmissions. If the message isn't successfully delivered an error is raised.

@hoffmabc

This comment has been minimized.

Copy link
Member

hoffmabc commented Apr 14, 2017

Can't IPFS do uTP?

@JustinDrake

This comment has been minimized.

Copy link
Contributor Author

JustinDrake commented Apr 14, 2017

Websockets are built on top of TCP. There are ACKs and re-transmissions. If the message isn't successfully delivered an error is raised.

I don't think it's that easy, especially in a browser context. There are various Websockets modes, but IPFS uses websocket frame streams. That is, application messages are chunked into frames to form a continuous stream. The browser will give you some generic error eventually, but good luck knowing which frame errored and which message that corresponds to. If you know how to know whether or not an application message was fulled ACKed in JS I'm all ears!

@JustinDrake

This comment has been minimized.

Copy link
Contributor Author

JustinDrake commented Jul 20, 2017

This was discussed today with @cpacia. Doing a simple ACK scheme (no sequence numbers, no retries) is both easy to implement and valuable in the context of browser implementations of OpenBazaar communicating with peers over unreliable transports (WebRTC, and WebSockets) where (despite the underlying reliable TCP) the browser does not communicate which fragments have been received.

@jjeffryes

This comment has been minimized.

Copy link
Collaborator

jjeffryes commented Jun 29, 2018

I'm re-prioritizing this, as we'll need it for the browser-based buying work.

@placer14 placer14 added the discussion label Oct 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment