Skip to content

Networking Stack

Jump to bottom
kazah-png edited this page Aug 25, 2026 · 8 revisions

Networking Stack

A complete in-kernel TCP/IP stack written from scratch, from the NIC register writes up to an HTTPS client:

RTL8139 → Ethernet → ARP → IPv4 → ICMP / UDP / TCP → DHCP / DNS / HTTP / TLS
                                        ↓
                              BSD sockets (ring 3)

See also: Drivers, Cryptography-and-TLS, Selene-Browser, Syscalls, Shell

Link layer

Supported hardware: Ethernet only. rtl8139.c is the sole entry in OBJS_KERNEL for a network interface card — there is no Wi-Fi driver, no 802.11 stack, and no WPA supplicant. A load_wifi_firmware(void) {} stub exists in kernel.c alongside a handful of other empty remote-syscall stubs, but nothing calls it; it is dead code, most likely a leftover from the pre-v2.0.0 "hacking/offensive code" removal, not a partial feature (see Known limits below).

RTL8139 (rtl8139.c)

PCI NIC with both I/O and MMIO BARs, 4 TX descriptors, a circular RX ring, 10/100 Mbps, and link detection. Under QEMU: -nic user,model=rtl8139.

The RX ring is the part worth understanding. The driver tracks its own read offset rather than re-deriving it from CAPR + 16, checks the BUFE (buffer-empty) bit before consuming, and initialises CAPR to 0xFFF0. The earlier version wrote the offset back without subtracting 16, which was asymmetric and left the ring misaligned after the very first packet.

Ethernet (ethernet.c)

Frame construction and demultiplexing by EtherType.

ARP (arp.c)

A 16-entry cache with TTL. Sends "who has" broadcasts, processes replies, and answers requests for our own address.

Network layer

IPv4 (ip.c)

Send and receive with header checksums. Routes off-subnet traffic via the gateway rather than ARPing internet addresses directly, and delivers loopback internally without touching the NIC.

ICMP (icmp.c)

Answers echo requests (type 8) with replies (type 0). The ping command sends four requests and reports RTT and loss.

Transport

UDP (udp.c)

Datagram send plus port-based listener registration.

TCP (tcp.c)

A real state machine over 32 connections (TCP_MAX_CONNS):

CLOSED → SYN_SENT → ESTABLISHED → FIN_WAIT → TIME_WAIT → CLOSED
           ↑                          ↓
        LISTEN ← accept          CLOSE_WAIT → LAST_ACK
Feature Detail
Active open connect() blocks until the three-way handshake completes
Passive open listen()/accept() — NyxOS serves HTTP to real external clients through QEMU hostfwd
Server half-close Replies are sent from CLOSE_WAIT, the standard pattern when a client FINs immediately after its request
Retransmission Exponential backoff, 300 ms → 2400 ms cap, 5 retries. Segments are buffered verbatim; a cumulative ACK clears the buffer
Loopback 127.0.0.x over a software ring — the whole handshake, data transfer and teardown are testable in-guest with no NIC

tcpdrop N deliberately drops the next N transmitted segments so you can watch retransmission work; tcploop runs the in-guest self-test.

Receive-path robustness

The v5.9.100+ maintenance pass hardened tcp_handle_packet against several real defects:

Fix Release Detail
In-order gate v5.9.112 Payload is accepted only when seq == conn->ack. A duplicate (retransmit) or out-of-order segment is dropped and a duplicate-ACK of the true conn->ack is sent — previously such a segment was appended blindly, corrupting the byte stream
data + FIN v5.9.111 A segment carrying the peer's last data and its FIN (an HTTP/1.0 server piggybacks FIN on its final data) took only the data branch of an if/else if chain, so the FIN was ignored and graceful close stalled until the peer's RTO retransmitted a bare FIN. FIN handling is now its own check after the data path
OOM drop v5.9.104 A remote peer sending data while the kernel heap was exhausted hit an unchecked kmalloc and NULL-dereferenced — a remotely-triggerable kernel crash. The allocation is now checked; on failure the segment is dropped un-acked and the peer retransmits
Checksum validated v6.4.86 The TCP checksum is now verified on receive — a segment with a bad checksum is dropped rather than acted on, so corrupted headers/payload can't reach the connection state
Close handshake v6.4.88 tcp_close keeps the connection alive so FIN_WAIT/LAST_ACK actually reach CLOSED, instead of tearing the state down immediately
Retransmit queue v6.4.90 An unacked segment is queued, so a second send can no longer drop the first still-unacknowledged segment
Receive-window flow control v6.4.349 The advertised receive window bounds the receive buffer, so a fast sender can't grow it without limit (#80)

All the receive-path fixes are behaviour-preserving on every in-order, in-memory path; only the malformed/exhausted edge cases change.

Note

SMP-safety (v6.4.89). The TCP connection table is now serialised behind a coarse tcp_lock spinlock across every public entry point, so two cores in the stack at once can't corrupt shared connection state. /proc got the same treatment (v6.4.87) — a real spinlock in place of preempt_disable, which only stops the local core.

Note

Async-networking groundwork. rtl8139 now wires its RX/TX interrupt (v6.4.92) and waits for TX-descriptor completion before reusing a descriptor (v6.4.102) — the first steps toward interrupt-driven networking, replacing the polled path. Sockets now yield (sleep) between polls instead of busy-spinning (v6.4.331), so a blocked socket no longer burns a core.

Tip

Inspection tools (v6.4.291–318). netstat shows the TCP connection table (local/foreign/state), route shows the IPv4 routing table, arp shows the ARP cache, and httpd is a one-request HTTP server that serves files from the VFS. ipcalc is an IPv4 subnet calculator.

A note on byte order

Every field that crosses the wire is stored in network byte order, funnelled through htons/htonl/ntohs/ntohl. This sounds obvious, and it was the single largest source of bugs in this subsystem's history. The IP header checksum stored little-endian made the host silently drop every frame, which in turn masked the same class of bug in the TCP header, DNS question counts, and ARP addresses — none of which could be observed until the checksum was fixed. If you touch a header field, check the endianness.

Application protocols

DHCP (dhcp.c)

DISCOVER → OFFER → REQUEST → ACK, setting the local IP, netmask and gateway. Auto-DHCP runs at boot when a NIC is present, so the network is ready without a manual dhcp. Under QEMU's user-mode networking the lease is typically 10.0.2.15.

DNS (dns.c)

A single A-record UDP query to the configured server. dns <hostname> from the shell.

Spoofing resistance (v6.4.54, hardened v6.4.109). Each query now carries a random CSPRNG transaction ID, and a response whose ID does not match the outstanding query is rejected — so an off-path attacker can no longer forge an answer by racing the real server. The response parser was later hardened further and locked with an adversarial KAT. See Security.

Note

Parser hardening (v6.4.123–181). Every text/wire parser that touches untrusted input was tightened and, where practical, locked with a KAT: a strict ipv4_parse (v6.4.123), a strict RFC 4291 IPv6 address parser (v6.4.165), one hardened URL parser that bounds the port and rejects an over-long host (v6.4.161), and a single bounds-checked DHCP option walker (v6.4.181). Overflow-checked integer parsers (numparse, v6.4.126) sit underneath them.

HTTP (http.c)

HTTP/1.1 GET over TCP with a Host: header and a timeout-based receive. http_get() waits for the handshake to complete before sending. http_parse_response extracts the Location: header (case-insensitive http_get_header), so Selene-Browser follows redirects; wget follows them too.

Note

http_request builds into a 512-byte stack buffer. C99 snprintf returns the length it would have written, so a long host/path could once make req_len exceed the buffer and tcp_send read past it — leaking kernel stack. Since v5.9.108 req_len is clamped to [0, 511] before the send. A malformed request is rejected by the server, but there is no out-of-bounds read.

HTTPS

TLS 1.2 with certificate-chain verification. It has its own page: Cryptography-and-TLS.

BSD sockets (net.c)

32 socket slots, exposed to ring 3 as ordinary file descriptors. A socket fd works with read, write, close and poll exactly like a pipe or a file.

TCP client

int fd = socket(AF_INET, SOCK_STREAM, 0);
connect(fd, inet_ipv4(93, 184, 216, 34), 80);
write(fd, "GET / HTTP/1.0\r\n\r\n", 18);
int n = read(fd, buf, sizeof buf);
close(fd);

TCP server

int s = socket(AF_INET, SOCK_STREAM, 0);
bind(s, INADDR_ANY, 8080);
listen(s, 4);
int c = accept(s);          // blocks; returns a NEW fd, the listener stays open

UDP

int u = socket(AF_INET, SOCK_DGRAM, 0);
sendto(u, msg, len, 0, inet_ipv4(10,0,2,2), 7);      // auto-binds an ephemeral port
unsigned ip; int port;
recvfrom(u, buf, sizeof buf, 0, &ip, &port);

send()/recv() are aliases for write()/read(); their flags argument is accepted and ignored.

poll()

struct pollfd pf[2] = { {0, POLLIN, 0}, {sock, POLLIN, 0} };
int n = poll(pf, 2, 1000);      // ms; <0 blocks forever

Works over sockets, pipes and stdin — this is what makes nc full-duplex rather than strictly turn-taking.

Address convention

inet_ipv4(a, b, c, d) builds a network-order address with the first octet in the low byte, matching the kernel's internal convention. Ports are host order.

Userspace network tools

Program Purpose
nc Netcat — bridges stdin/stdout to a TCP or UDP socket, full-duplex via poll()
wget HTTP client with its own DNS-over-UDP resolver; follows redirects
sockdemo TCP client demo
srvdemo TCP server demo
udpdemo UDP datagram demo
polldemo poll() over several fds
netstorm Concurrent multi-process socket stress test

Built-in services for testing without an external peer: a TCP echo service and a UDP echo service, both started at boot.

Kernel shell commands

Command Purpose
ifconfig Interface list, addresses, link state
dhcp Request a lease manually
setip <ip> <mask> <gw> Static configuration
ping <ip|host> 4 ICMP echoes with RTT and loss
dns <hostname> Resolve an A record
httpget <url> Fetch a URL and print the response
tcptest <ip> <port> Open a connection
tcpserve [port] Serve one TCP/HTTP connection
tcploop [drop] In-guest loopback self-test
tcpdrop <n> Drop the next N TX segments, forcing retransmission
tls <host> TLS handshake against host:443
tlsstrict [on|off] Toggle strict certificate enforcement

Known limits

  • Ethernet only. No Wi-Fi, no 802.11, no other NIC family — see the link-layer section above.
  • IPv4 only. No IPv6.
  • No fragmentation or reassembly.
  • 32 concurrent TCP connections, 32 socket slots.
  • Minimal hardening. The stack is not defended against malformed or hostile packets; see Security.

See also

External resources

NyxOS v6.4.363 · GPL v2 · GitHub · uselessalter on Discord · nyxos@inbox.lv

NyxOS Wiki

Getting started

Kernel

Storage & network

Graphics & apps

Userspace

HOWTO

Reference

Project

Clone this wiki locally