Implement uTP

[jlouis]

The goal is to implment BEP29. Note that this BEP is not sufficient to understand the protocol at all. One must peruse source code. Nameiy:

https://github.com/bittorrent/libutp

It is a fairly complex implementation in C++ and unwrapping it to Erlang takes some time. Currently I've opted for the idea where I translate C++ code into Erlang and then figure out the real data flow. Another thing is to kill the delimited continuations present due to a call-back event implementation and substitute this with processes.

Current status is that I believe the process design is sound, but the internal state machine of a single connection is far from done. It should come as no surprise: We are essentially reimplementing TCP in userland :) After I have the low-level calls down and untangled the data flow, I'll start on the queue buffer code for sends and receives.

[baryluk]

Keeping etorrent pure erlang would be really sweet, from safety, fault-tollerance,portability and code maintaince standpoint. I know it is probably lots of work, but should be better way than wraping things up.

[jlouis]

Yep I agree. Take a look at the uTP branch if you want to see where we are currently. There should be a fairly grand todo.org TODO list in the apps/utp directory if you are interested.

I am currently working on stabilizing the core a bit. Then I can go and implement the last couple of missing features (namely SACKs and Nagle's algorithm.

[baryluk]

SACK and Nagle? Sounds like reimplementing TCP to me :) Shouldn't kernel do this, or we are talking here about user-space connection-like streams based on UDP? Anyway, SACK is important, but can be omitted at the beginning, as normal ACK will probably suffice for start.

Anyway will start looking into it.

[jlouis]

Right, they are built on top of UDP. And yes, both SACK and Nagle are important, but I'd rather get the other parts working first and then pile yet another layer on top of the protocol. I have done some refactoring runs on the code, but it could definitely do with some more in the long run.

The current focus is on instrumentation. I need to have ways to picture and plot internal counters while I am having a connection. That way, I can figure out if there is something wrong with the internals easily.