[SSHD-1262] Eliminate buffering in TCP/IP port forwarding

[tomaswolf]

TCP/IP port forwarding started reading on the local port before the SSH
tunnel was established, i.e., before SSH_MSG_CHANNEL_OPEN_CONFIRMATION
was received. It buffered data until then, and then tried to flush the
buffer once the confirmation was received.

This didn't work if the channel window was exhausted while that buffered
data was to be written, and in particular it failed when the channel was
opened with an initial window size of zero.

Change the whole forwarding setup: first set up the SSH tunnel, only
then start reading from the local port. That way, no data needs to be
buffered at all.

Use a ChannelAsyncOutputStream to forward data read into the SSH tunnel;
this nicely adapts to the channel window. To avoid concurrent writes on
this channel, suspend reading from the local port before writing, and
resume it only after the data was written. This automatically throttles
reading to the speed of writing, which is what one wants in this case.

Implement IOSession.suspendRead() and IOSession.resumeRead() also for
the MINA and Netty back-ends. Fix the implementation for the NIO2
back-end: the former implementation of resumeRead() could cause
concurrent reads if the channel indeed did an asynchronous read, and if
the channel did a synchronous read it led to deep completion handler
chains on the stack, and could delay the next read on some other
session's completion handler. Resolve this for our use-case by
scheduling an asynchronous read explicitly if necessary, and letting
the normal read completion handler do the read if possible.

Do this also for the TcpipServerChannel: it started reading from the
connected port before it even sent SSH_MSG_CHANNEL_OPEN_CONFIRMATION.
Remove the BufferedIoOutputStream and use a ChannelAsyncOutputStream
always.

Include technical documentation on TCP/IP port forwarding.

[lgoldstein]

Seems OK to me - I like your approach - much cleaner. I do have one concern - I was not able to determine whether pending peer packets are flushed before EOF is sent and the channel is closed.

[tomaswolf]

I was not able to determine whether pending peer packets are flushed before EOF is sent and the channel is closed.

They should be. On a graceful close, the channel first closes the ChannelAsyncOutputStream gracefully. The stream's close future is fulfilled only once the last write has been completed. Sending the EOF happens in a future listener on the stream's close future, so it goes out after the stream is closed, and it's done not via the stream but directly via the SSH session.

See TcpipServerChannel, lines 208ff and TcpipClientChannel, lines 157ff. I've added comments at both places.

[tomaswolf]

BTW: apparently this also fixes SSHD-1256.

[lgoldstein]

I was not able to determine whether pending peer packets are flushed before EOF is sent and the channel is closed.

They should be. On a graceful close, the channel first closes the ChannelAsyncOutputStream gracefully. The stream's close future is fulfilled only once the last write has been completed. Sending the EOF happens in a future listener on the stream's close future, so it goes out after the stream is closed, and it's done not via the stream but directly via the SSH session.

Great