Use splice() for low-end routers

[ghost]

splice() moves data between two file descriptors without copying
between kernel address space and user address space. It transfers up
to len bytes of data from the file descriptor fd_in to the file
descriptor fd_out, where one of the file descriptors must refer to a
pipe.

http://man7.org/linux/man-pages/man2/splice.2.html

Also see

https://github.com/wongsyrone/transocks-wong/blob/6ee0f0a821a7a144af6bfdf683aed0b75ea82433/src/splicepump.c#L68

[ghost]

It seems that the reference only supports TCP, though.
wongsyrone/transocks-wong#5

[zfl9]

http://man7.org/linux/man-pages/man2/splice.2.html

看了下文档，两个fd中必须至少有一个是pipe管道，感觉没太大意义？

[ghost]

Pipe is nothing but a buffer in the kernel. And with SPLICE_F_MOVE flag it moves data from socket, instead of copying it, into that buffer.

https://blog.cloudflare.com/sockmap-tcp-splicing-of-the-future/

[zfl9]

看起来似乎有点意思，周末在研究下

[zfl9]

不过看cf的那个博客，感觉sockmap更加有趣哈。UPDATE：sockmap似乎很复杂。还是splice()方便。

[zfl9]

Pipe is nothing but a buffer in the kernel. And with SPLICE_F_MOVE flag it moves data from socket, instead of copying it, into that buffer.

https://blog.cloudflare.com/sockmap-tcp-splicing-of-the-future/

仔细想想其实是这个道理，只不过我现在是通过用户空间的一个buffer做中转，而利用splice()的话，其实就是用一个pipe管道充当这个buffer而已。

[zfl9]

   SPLICE_F_MOVE
          Attempt to move pages instead of copying.  This is only a hint
          to the kernel: pages may still be copied if the kernel cannot
          move the pages from the pipe, or if the pipe buffers don't
          refer to full pages.  The initial implementation of this flag
          was buggy: therefore starting in Linux 2.6.21 it is a no-op
          (but is still permitted in a splice() call); in the future, a
          correct implementation may be restored.

如果真如文档所说（或许现在已实现真正的Move?），我觉得 splice() 与我现在的实现方式并无本质区别（这两种方式都存在两次 copy 操作），当然性能可能是会有差距，可能是因为内核空间复制数据更快？另外也少了几个系统调用的开销。不过，还是决定实现一个看看，究竟性能如何。

[ghost]

The blog tests it with 4.14 kernel and it seems that the performance of splice() is pretty close to sockmap

[zfl9]

已替换为 splice() 接口。感觉还可以（在 rpi3b 上测试）。欢迎继续测试，感谢你的好建议。

[ghost]

Works great with glibc. Great job!

But it may still need some patch when compiling with uclibc, since splice may be still too new for it.

https://github.com/ThingMesh/openwrt-luci/blob/ac597e471793ff37b7ebc759f6bd8ffa744995dd/libs/nixio/src/splice.c#L45

[zfl9]

有编译报错的可以请请教下 @simon-on-gh （报错应该是 splice() 啥的未定义吧）。有这问题的应该都是嵌入式设备的Linux发行版