Syscalls

System Calls

The ring-3 → ring-0 interface, entered with the x86_64 syscall instruction and returned from with iretq. 61 calls, numbered 0–60. The numbers and the C wrappers are defined in user/syscall.h; the dispatch lives in kernel/core/syscall.c.

See also: Userspace, Process-Management, Memory-Management, Networking-Stack, Security

Calling convention

Register	Purpose
RAX	Syscall number (and the return value)
RDI	arg1
RSI	arg2
RDX	arg3
R10	arg4 (not RCX — syscall overwrites it)
R8	arg5
R9	arg6
Clobbered	RCX (return RIP), R11 (saved RFLAGS)

The sixth argument is marshalled onto the stack by the entry stub and passed as the seventh C parameter to the dispatcher — which is what lets mmap honour a real offset.

Negative return values are errors. Where a specific code matters it is noted below.

#include "syscall.h"

long r = syscall3(SYS_WRITE, 1, (long)"hi\n", 3);

In practice you call the inline wrappers in user/syscall.h (write, open, fork, mmap, …) rather than syscallN directly. See Userspace.

The table

Process and program lifecycle

#	Call	Notes
0	exit(code)	Becomes a zombie, wakes the parent
6	getpid()
47	getppid()
9	exec(path)	Spawn + wait, foreground. Legacy — prefer fork + execve
10	fork()	Copy-on-write clone. Child gets 0, parent gets the child's pid
11	waitpid(pid, *status, options)	Reap a child. pid <= 0 waits for any. WNOHANG returns 0 if still running; WUNTRACED also reports a stopped child
13	execve(path, argv, envp)	Replace the image in place — same pid, same fds, new address space
50	clone(fn, stack_top, arg, flags)	With CLONE_VM, a real thread sharing this address space. Returns the new tid
51	futex(uaddr, op, val)	FUTEX_WAIT sleeps while *uaddr == val; FUTEX_WAKE wakes up to val waiters
33	setfg(pid)	Make pid the terminal foreground process, so Ctrl-C/Ctrl-Z target it. 0 clears

Signals

#	Call	Notes
16	kill(pid, sig)	sig 0 probes existence
17	signal(sig, handler)	SIG_DFL, SIG_IGN, or a handler. Returns the previous disposition, or SIG_ERR
18	sigreturn()	Restores the interrupted context after a handler; entered via the crt0 trampoline, not called directly
41	sigprocmask(how, set, oldset)	SIG_BLOCK / SIG_UNBLOCK / SIG_SETMASK over a 32-bit mask
42	alarm(seconds)	Schedule SIGALRM; 0 cancels. Returns the seconds left on any previous alarm

File I/O

#	Call	Notes
1	write(fd, buf, len)
2	print(msg)	Debug print. Legacy
3	open(path, flags, mode)	O_CREAT, O_TRUNC, O_APPEND
4	read(fd, buf, len)	Files, pipes, sockets, and stdin (canonical or raw)
5	close(fd)
8	fsize(fd)
44	stat(path, *st)	Fills st_size, st_mode, st_ino
45	fstat(fd, *st)
46	lseek(fd, offset, whence)	SEEK_SET/CUR/END. Returns the new absolute offset; pipes and sockets are not seekable
12	pipe(fds)	fds[0] read end, fds[1] write end. Blocking read, reference-counted ends, survives fork
14	dup2(oldfd, newfd)	The redirection primitive; closes newfd first
48	dup(oldfd)	Lowest available fd for the same stream
15	getdents(path, buf, max)	Directory enumeration into 68-byte nyx_dirent_t records
21	chdir(path)	Per-process cwd; relative paths resolve against it
22	getcwd(buf, size)
23	mkdir(path, mode)
24	unlink(path)	File or empty directory
49	rename(oldpath, newpath)

Buffer residency. getdents and getprocs copy into a user buffer, and the kernel cannot fault in a lazy-sbrk heap page on your behalf. Use a .bss array, or memset the buffer before the call.

Memory

#	Call	Notes
7	sbrk(increment)	Moves the break; pages fault in on first touch
19	mmap(addr, length, prot, flags, fd, offset)	Anonymous demand-zero or file-backed. Returns the base VA or MAP_FAILED
20	munmap(addr, length)	Partial ranges split the VMA
26	mprotect(addr, len, prot)	Rewrites present-page flags and the VMA prot; partial ranges split the VMA

Terminal and input

#	Call	Notes
25	ttymode(mode)	TTY_CANON (kernel line discipline) or TTY_RAW (byte-at-a-time, no echo, arrows as ANSI escapes). Returns the previous mode; execve resets it
28	readkey(timeout_ms)	One keypress, blocking up to timeout_ms. Returns the key, 0 on timeout, negative if interrupted. 0 blocks forever. No echo, independent of ttymode
56	getkeyevent()	Non-blocking raw key event — press and release, with a Set-1 scancode. The full-screen-game path

Framebuffer

#	Call	Notes
54	fbinfo(out[3])	Screen {width, height, bpp}
55	fbpresent(buf, w, h)	Blit a w × h 32bpp buffer to the screen, nearest-neighbour scaled to fullscreen. While a program keeps calling it, it owns the screen and the compositor yields

Windows (v6.4.354)

A ring-3 program can open a real desktop window — no fullscreen takeover — and the compositor keeps running around it. wintest is the worked example; the N-language nwin draws one from N (see N-Language).

#	Call	Notes
57	win_create(w, h, title)	Open a window with a w × h client area. Returns a window id, or −1
58	win_destroy(id)	Close the window. Idempotent (returns 0 even if already gone)
59	win_present(id, buf, w, h)	Blit a w × h XRGB (0x00RRGGBB) buffer as the whole client area
60	win_poll_event(id, ev)	Pop one input event into ev (non-blocking): 1 got one, 0 empty, −1 bad id

Sockets

Full detail in Networking-Stack.

#	Call	Notes
34	socket(domain, type, protocol)	AF_INET with SOCK_STREAM or SOCK_DGRAM. Returns an fd usable with read/write/close
35	connect(fd, ip, port)	Blocks until the TCP handshake completes
36	bind(fd, ip, port)	INADDR_ANY binds all interfaces
37	listen(fd, backlog)
38	accept(fd)	Blocks; returns a new fd for the connection
39	sendto(fd, buf, len, flags, ip, port)	UDP; auto-binds an ephemeral source port on first send
40	recvfrom(fd, buf, len, flags, *ip, *port)	UDP; fills the sender's address
43	poll(fds, nfds, timeout)	POLLIN/POLLOUT over sockets, pipes and stdin. Timeout in ms, <0 blocks forever

send()/recv() are wrappers around write()/read(); the flags argument is accepted and ignored.

Time

#	Call	Notes
31	time(*tm)	Broken-down local time from the RTC
32	sleep(ms)	Blocks; other processes run. Negative if a signal interrupted it
52	gettimeofday(*tv, tz)	Seconds + microseconds since the Unix epoch. tv_sec is 64-bit, so y2038-safe. tz is ignored
53	nanosleep(*req, *rem)	Rounded up to the 1 ms tick. rem is zeroed — no precise remainder is tracked

Introspection and dynamic loading

#	Call	Notes
27	getprocs(buf, max)	Snapshot the process table into 48-byte nyx_procinfo_t records — the ps primitive
29	dlopen(path)	Load a prelinked .so on demand. Returns a handle ≥ 1
30	dlsym(handle, name)	Resolve a symbol to its address

Shared record layouts

These structs are copied verbatim between kernel and userspace, so their field order must match kernel/core/syscall.c:

typedef struct { char name[64]; unsigned int type; } nyx_dirent_t;          // 68 bytes
typedef struct { unsigned pid, ppid, state, cpu_time; char comm[32]; }      // 48 bytes
        nyx_procinfo_t;
typedef struct { int sec, min, hour, mday, mon, year; } nyx_tm;             // mon 1..12
struct stat { unsigned st_size, st_mode, st_ino; };
struct pollfd { int fd; short events; short revents; };
struct timeval  { long tv_sec; long tv_usec; };
struct timespec { long tv_sec; long tv_nsec; };

Boundary hardening

The syscall boundary is the only path from ring 3 into the kernel, so it is where the hardening lives. See Security for the full picture.

• Pointer validation. user_ptr_ok() rejects anything outside [USER_SPACE_MIN, USER_SPACE_END) — that is, [0x1000, 0x800000000000). Page 0 is deliberately excluded so a NULL dereference traps.
• Opaque fds. Userspace receives small integers indexed into a per-process table. Kernel VFS handles are never exposed, so an fd cannot be forged into a kernel pointer.
• copy_from_user / copy_to_user. These walk the user page tables to a physical address and bounce through the higher-half alias, rather than dereferencing a user pointer directly.
• Interrupts masked. Syscalls run with IF=0, which makes current_idx stable for the duration of the call.
• Per-CPU entry state. user_rsp, user_cr3, kernel_rsp and syscall_frame_ptr live in a per-CPU block reached through GS, not in globals — otherwise two cores entering the kernel at once would overwrite each other's saved state.
• Flags sanitised. The entry paths issue cld — ring 3 could otherwise leave RFLAGS.DF set and change the direction of every kernel string operation.

See also

• Syscall-Reference - all 61 calls, one entry each
• HOWTO-Add-a-system-call - the procedure, step by step
• Userspace - the libc wrappers around these
• Kernel-Data-Structures - the shared record layouts
• Security - what the boundary defends against

External resources

• System V AMD64 ABI - the calling convention
• Intel SDM Vol. 2 - SYSCALL/SYSRET - Intel
• POSIX.1-2017 - System Interfaces - The Open Group