latency & bindwidth tcp http

[jiacai2050]

• XMLHttpRequest (XHR) Uses Multiple Packets for HTTP POST?
• Why is he using 1460 as the packet size?
• http://carlos.bueno.org/2009/10/dismal-guide-to-bandwidth.html

Guide

• https://developer.yahoo.com/performance/rules.html
• https://en.wikipedia.org/wiki/Traffic_shaping

[jiacai2050]

rwnd 控制 sender 与 receiver 之间的传输速率。但 TCP 还需要有措施控制整个网络中的流量，主要有以下几类措施：

slow start

Congestion window size (cwnd) Sender-side limit on the amount of data the sender can have in flight before re‐ ceiving an acknowledgment (ACK) from the client.
4 segments by RFC 2581 in April 1999
10 segments by RFC 6928 in April 2013

监控 client 与 server 之间的带宽。一个 TCP 链接刚建立时，最大传输速率是 min(rwnd, cwnd)，之后每次收到一个 ack， cwnd 翻倍，直到超过接收端的 flow-control window，a system-configured congestion threshold（ssthresh） 或者丢包发生。

Slow-Start Restart

主要针对空闲的长链接，原因也很简单，在空闲期间，网络情况可能发生了变化。可选择在 server 禁用：

sysctl -w net.ipv4.tcp_slow_start_after_idle=0

与之对应的优化手段：keepalive, pipelining, and multiplexing。

congestion avoidance

在 cwnd 增长期间，如果发生丢包，那么「拥塞避免」算法就开始工作了。
之处，TCP used the Multiplicative Decrease and Additive Increase (AIMD) algorithm: when packet loss occurs, halve the congestion window size, and then slowly increase the window by a fixed amount per roundtrip。但 AIMD 太保守了，新的算法 Proportional Rate Reduction（by RFC 6937）在 Linux 3.2+ 中内置。

• https://lwn.net/Articles/458610/
• https://en.wikipedia.org/wiki/TCP_congestion_control
• http://sgros.blogspot.hk/2012/12/controlling-which-congestion-control.html

cat /proc/sys/net/ipv4/tcp_congestion_control
# 查看系统中可用的算法
cat /proc/sys/net/ipv4/tcp_available_congestion_control

• https://wiki.aalto.fi/download/attachments/69901948/TCP-CongestionControlFinal.pdf

3. fast retransmit
4. fast recovery

[jiacai2050]

Optimizing for TCP

Additional mechanisms, such as selective acknowledgments (SACK), delayed acknowledgments, and fast retransmit, among many others, make each TCP session much more complicated (or interesting, depending on your perspective) to understand, analyze, and tune.

Having said that, while the specific details of each algorithm and feedback mechanism will continue to evolve, the core principles and their implications remain unchanged:
• TCP three-way handshake introduces a full roundtrip of latency.
• TCP slow-start is applied to every new connection.
• TCP flow and congestion control regulate throughput of all connections.
• TCP throughput is regulated by current congestion window size.

措施

1. 检查 initcwnd. ss -nli|fgrep cwnd
   • https://www.cdnplanet.com/blog/tune-tcp-initcwnd-for-optimum-performance/
2. 关闭 slow start restart sysctl -w net.ipv4.tcp_slow_start_after_idle=0
3. 开启 window scaling sysctl -w net.ipv4.tcp_window_scaling=1
4. 开启 TCP Fast Open，允许应用在 SYN 包中发送数据，需要 client 与 server 支持才行。

ss --options --extended --memory --processes --info

[jiacai2050]

UDP

UDP is a simple and a commonly used protocol for bootstrapping new transport protocols. In fact, the primary feature of UDP is all the features it omits: no connection state, handshakes, retransmissions, reassembly, reordering, congestion control, congestion avoidance, flow control, or even optional error checking. However, the flexibility that this minimal message-oriented transport layer affords is also a liability for the implementer.

NAT traversal

Session Traversal Utilities for NAT (STUN) is a protocol (RFC 5389) that allows the host application to discover the presence of a network address translator on the network, and when present to obtain the allocated public IP and port tuple for the current connection

However, in practice, STUN is not sufficient to deal with all NAT topologies and network configurations. Further, unfortunately, in some cases UDP may be blocked altogether by a firewall or some other network appliance—not an uncommon scenario for many enterprise networks. To address this issue, whenever STUN fails, we can use the Traversal Using Relays around NAT (TURN) protocol (RFC 5766) as a fallback, which can run over UDP and switch to TCP if all else fails.

Interactive Connectivity Establishment (ICE) protocol (RFC 5245) is a protocol, and a set of methods, that seek to establish the most efficient tunnel between the participants (Figure 3-7): direct connection where possible, leveraging STUN negotiation where needed, and finally fallback to TURN if all else fails.

[jiacai2050]

ss -i 的内容

ESTAB       0      0 ::ffff:127.0.0.1:2181 ::ffff:127.0.0.1:52150
cubic wscale:7,7 rto:204 rtt:7.5/3 ato:40 mss:21888 cwnd:10 send 233.5Mbps rcv_space:43690

前面两个 0 依次表示 Recv-Q Send-Q
cubic 使用的拥塞避免算法
wscale 表示 rwnd 的 scale 值，逗号前面的
rto 表示 retransmission timeout，单位 ms 一个 segment 在 204ms 内没收到 ack 时会重传

[jiacai2050]

initial_ssthread 为 0 的问题

cat /sys/module/tcp_cubic/parameters/initial_ssthresh

• https://lists.gt.net/linux/kernel/830993#830993
• https://superuser.com/questions/307213/tcp-slow-start-and-congestion-control-and-how-to-modify-them
• http://lists.openwall.net/netdev/2007/05/16/10
• https://tools.ietf.org/html/rfc5681
• https://books.google.com/books?id=Z3ciBgAAQBAJ&pg=PT114&lpg=PT114&dq=how+to+change+/sys/module/tcp_cubic/parameters/&source=bl&ots=6EGUP0DgA7&sig=mQMu6oKswaycXvA34IA6KYP-K8I&hl=en&sa=X&ved=2ahUKEwinhP79mITaAhVB5IMKHc43CSkQ6AEwA3oECAAQPg#v=onepage&q=how%20to%20change%20%2Fsys%2Fmodule%2Ftcp_cubic%2Fparameters%2F&f=false

[jiacai2050]

http2

• https://bagder.gitbooks.io/http2-explained/content/en/part2.html

[jiacai2050]

CS 457 – Lecture 24 Congestion PDF