Networking Stack

Networking Stack

A complete in-kernel TCP/IP stack written from scratch, from the NIC register writes up to an HTTPS client:

RTL8139 → Ethernet → ARP → IPv4 → ICMP / UDP / TCP → DHCP / DNS / HTTP / TLS
                                        ↓
                              BSD sockets (ring 3)

See also: Drivers, Cryptography and TLS, Selene Browser, Syscalls, Shell

Link layer

RTL8139 (rtl8139.c)

PCI NIC with both I/O and MMIO BARs, 4 TX descriptors, a circular RX ring, 10/100 Mbps, and link detection. Under QEMU: -nic user,model=rtl8139.

The RX ring is the part worth understanding. The driver tracks its own read offset rather than re-deriving it from CAPR + 16, checks the BUFE (buffer-empty) bit before consuming, and initialises CAPR to 0xFFF0. The earlier version wrote the offset back without subtracting 16, which was asymmetric and left the ring misaligned after the very first packet.

Ethernet (ethernet.c)

Frame construction and demultiplexing by EtherType.

ARP (arp.c)

A 16-entry cache with TTL. Sends "who has" broadcasts, processes replies, and answers requests for our own address.

Network layer

IPv4 (ip.c)

Send and receive with header checksums. Routes off-subnet traffic via the gateway rather than ARPing internet addresses directly, and delivers loopback internally without touching the NIC.

ICMP (icmp.c)

Answers echo requests (type 8) with replies (type 0). The ping command sends four requests and reports RTT and loss.

Transport

UDP (udp.c)

Datagram send plus port-based listener registration.

TCP (tcp.c)

A real state machine over 32 connections (TCP_MAX_CONNS):

CLOSED → SYN_SENT → ESTABLISHED → FIN_WAIT → TIME_WAIT → CLOSED
           ↑                          ↓
        LISTEN ← accept          CLOSE_WAIT → LAST_ACK

Feature	Detail
Active open	connect() blocks until the three-way handshake completes
Passive open	listen()/accept() — NyxOS serves HTTP to real external clients through QEMU hostfwd
Server half-close	Replies are sent from CLOSE_WAIT, the standard pattern when a client FINs immediately after its request
Retransmission	Exponential backoff, 300 ms → 2400 ms cap, 5 retries. Segments are buffered verbatim; a cumulative ACK clears the buffer
Loopback	127.0.0.x over a software ring — the whole handshake, data transfer and teardown are testable in-guest with no NIC

tcpdrop N deliberately drops the next N transmitted segments so you can watch retransmission work; tcploop runs the in-guest self-test.

A note on byte order

Every field that crosses the wire is stored in network byte order, funnelled through htons/htonl/ntohs/ntohl. This sounds obvious, and it was the single largest source of bugs in this subsystem's history. The IP header checksum stored little-endian made the host silently drop every frame, which in turn masked the same class of bug in the TCP header, DNS question counts, and ARP addresses — none of which could be observed until the checksum was fixed. If you touch a header field, check the endianness.

Application protocols

DHCP (dhcp.c)

DISCOVER → OFFER → REQUEST → ACK, setting the local IP, netmask and gateway. Auto-DHCP runs at boot when a NIC is present, so the network is ready without a manual dhcp. Under QEMU's user-mode networking the lease is typically 10.0.2.15.

DNS (dns.c)

A single A-record UDP query to the configured server. dns <hostname> from the shell.

HTTP (http.c)

HTTP/1.1 GET over TCP with a Host: header and a timeout-based receive. http_get() waits for the handshake to complete before sending. Redirects are followed by wget.

HTTPS

TLS 1.2 with certificate-chain verification. It has its own page: Cryptography and TLS.

BSD sockets (net.c)

32 socket slots, exposed to ring 3 as ordinary file descriptors. A socket fd works with read, write, close and poll exactly like a pipe or a file.

TCP client

int fd = socket(AF_INET, SOCK_STREAM, 0);
connect(fd, inet_ipv4(93, 184, 216, 34), 80);
write(fd, "GET / HTTP/1.0\r\n\r\n", 18);
int n = read(fd, buf, sizeof buf);
close(fd);

TCP server

int s = socket(AF_INET, SOCK_STREAM, 0);
bind(s, INADDR_ANY, 8080);
listen(s, 4);
int c = accept(s);          // blocks; returns a NEW fd, the listener stays open

UDP

int u = socket(AF_INET, SOCK_DGRAM, 0);
sendto(u, msg, len, 0, inet_ipv4(10,0,2,2), 7);      // auto-binds an ephemeral port
unsigned ip; int port;
recvfrom(u, buf, sizeof buf, 0, &ip, &port);

send()/recv() are aliases for write()/read(); their flags argument is accepted and ignored.

poll()

struct pollfd pf[2] = { {0, POLLIN, 0}, {sock, POLLIN, 0} };
int n = poll(pf, 2, 1000);      // ms; <0 blocks forever

Works over sockets, pipes and stdin — this is what makes nc full-duplex rather than strictly turn-taking.

Address convention

inet_ipv4(a, b, c, d) builds a network-order address with the first octet in the low byte, matching the kernel's internal convention. Ports are host order.

Userspace network tools

Program	Purpose
nc	Netcat — bridges stdin/stdout to a TCP or UDP socket, full-duplex via poll()
wget	HTTP client with its own DNS-over-UDP resolver; follows redirects
sockdemo	TCP client demo
srvdemo	TCP server demo
udpdemo	UDP datagram demo
polldemo	poll() over several fds
netstorm	Concurrent multi-process socket stress test

Built-in services for testing without an external peer: a TCP echo service and a UDP echo service, both started at boot.

Kernel shell commands

Command	Purpose
ifconfig	Interface list, addresses, link state
dhcp	Request a lease manually
setip <ip> <mask> <gw>	Static configuration
ping <ip|host>	4 ICMP echoes with RTT and loss
dns <hostname>	Resolve an A record
httpget <url>	Fetch a URL and print the response
tcptest <ip> <port>	Open a connection
tcpserve [port]	Serve one TCP/HTTP connection
tcploop [drop]	In-guest loopback self-test
tcpdrop <n>	Drop the next N TX segments, forcing retransmission
tls <host>	TLS handshake against host:443
tlsstrict [on|off]	Toggle strict certificate enforcement

Known limits

• IPv4 only. No IPv6.
• No fragmentation or reassembly.
• 32 concurrent TCP connections, 32 socket slots.
• Minimal hardening. The stack is not defended against malformed or hostile packets; see Security.