Building

Building and Running

NyxOS builds with a stock GCC targeting -m64 freestanding — no cross-compiler is strictly required, though x86_64-elf-gcc works too. The primary development environment is Windows with WSL: build.ps1 shells into WSL to run make, and run.ps1 launches QEMU on the Windows side.

See also: Architecture, Boot-Process, Networking-Stack, Version-History

Prerequisites

Tool	Notes
GCC	Host gcc with -m64, or x86_64-elf-gcc
NASM	Assembles boot.asm, crt0.asm, the AP trampoline and the ISR stubs
GNU ld	Linked with -T linker.ld
grub-mkrescue + xorriso	Builds the bootable ISO (grub-pc-bin on Debian/Ubuntu)
QEMU	7.0+ recommended (qemu-system-x86_64)
Python 3	Regenerates the initramfs

On Debian/Ubuntu or WSL:

sudo apt install build-essential nasm grub-pc-bin grub-common xorriso qemu-system-x86 python3

Quick start

Windows (PowerShell)

.\build.ps1

Compiles the kernel through WSL, reports the size of kernel/nyx-kernel.bin, then generates NyxOS.iso with grub-mkrescue. Add -Clean to force a full rebuild.

.\run.ps1

Boots the ISO in QEMU. Modes and options:

Invocation	Effect
.\run.ps1	SDL display, serial log to qemu_serial.txt (default)
.\run.ps1 -Mode serial	-nographic, serial on stdio — the kernel shell
.\run.ps1 -Mode net	SDL display plus an RTL8139 NIC (-nic user,model=rtl8139)
.\run.ps1 -Mode debug	SDL display plus -d cpu_reset,int
-Cpus <n>	CPU count, default 4 (see SMP)
-Sound	Attaches an SB16 device (implied by -Mode net)

The guest gets 512 MB and -cpu qemu64. If ext2-test.img is present it is attached as -hda; NyxOS auto-mounts it at /mnt, which is also where doom1.wad must live for DOOM to start.

Linux / WSL

make -C kernel

This builds the kernel and every user/*.elf — all: depends on $(USER_ELFS) as well as nyx-kernel.bin. Then:

qemu-system-x86_64 -cdrom NyxOS.iso -m 512M -smp 4 -no-reboot -cpu qemu64 \
  -hda ext2-test.img -nic user,model=rtl8139 -serial stdio

Build system

Kernel

make -C kernel          # build kernel + user ELFs
make -C kernel clean    # remove objects, .d files and binaries

Compiler flags:

-std=gnu99 -ffreestanding -Os -Wall -Wextra -Wshadow
-nostdlib -nostartfiles -nodefaultlibs
-fno-stack-protector -fno-pie -m64
-mno-red-zone -mno-sse -mno-mmx -mno-sse2 -mno-3dnow
-mcmodel=large -ffunction-sections -fdata-sections -MMD -MP -I.

-mno-red-zone is mandatory: interrupt handlers would otherwise clobber the 128-byte red zone below RSP. -mcmodel=large is what lets kernel code at 0xFFFFFF80_00000000 reference its own symbols. -Wshadow became a permanent flag in v5.9.103 after the two accumulated shadow warnings were fixed.

The build is expected to be 100 % warning-clean — C and NASM, and has been since v5.9.101. If make emits any warning, that is a regression. The kernel sources live in subject subfolders (core/, mm/, net/, gui/apps/, …) resolved by VPATH; objects still build flat into kernel/. See Source-Tree-Reference.

Note

-Wsign-conversion is deliberately not in the global build — the tree has ~1900 mostly-benign hits. The maintenance pass instead ran it file-by-file over the security-sensitive parsers (crypto/der.c, the image decoders, net/http.c), leaving each one individually clean under the flag.

Stale-build protection

Three separate bugs in this Makefile once shipped stale kernels silently, and the fixes are worth knowing about:

• .DEFAULT_GOAL := all is set before -include. Each auto-generated .d file starts with a rule like kernel.o: kernel.c …; because -include sits above the all: target, the first included rule would otherwise hijack the default goal, so a bare make built only kernel.o and never relinked.
• -MMD -MP header dependencies. Every object now depends on every header it includes, so editing a macro in tcp.h rebuilds tcp.o. Previously only kernel.h was tracked, and a header-only change left stale objects — which masked a real capacity fix as a phantom "Heisenbug".
• linker.ld is a prerequisite of nyx-kernel.bin. Editing the section layout now forces a relink.

If you suspect a stale build, make -C kernel clean is the reliable answer.

Userspace programs

User ELFs are built by the same Makefile from user/. Each is linked as:

ld -nostdlib -m elf_x86_64 -e _start -Ttext 0x10000 \
   -o prog.elf crt0.o --just-symbols=libc.so prog.o

--just-symbols=libc.so resolves libc symbols against the prelinked shared library without copying its code in — libc exists once in memory and is mapped into every process. A userspace program compiles with:

-std=gnu99 -Os -ffreestanding -nostdlib -m64 -mno-red-zone -I../user

See Userspace for the ABI and the available library surface.

Initramfs

The root filesystem is a CPIO (newc) archive embedded in the kernel as a C byte array. Regenerate it after adding or changing files:

python3 tools/mkinitramfs.py kernel/fs/initramfs_data.h c

ISO

Both build.ps1 and run.ps1 generate iso/boot/grub/grub.cfg and call:

grub-mkrescue -o NyxOS.iso iso/

QEMU reference

run.ps1 assembles the invocation below. Use it directly when you need options the script does not expose.

CODE — The full equivalent invocation

host $ qemu-system-x86_64 \
    -cdrom NyxOS.iso \
    -m 512M \
    -smp 4 \
    -no-reboot \
    -cpu qemu64 \
    -hda ext2-test.img \
    -display sdl \
    -serial file:qemu_serial.txt \
    -nic user,model=rtl8139

Option	Why NyxOS needs it
-cdrom NyxOS.iso	The kernel is booted by GRUB via Multiboot 2; there is no direct-kernel path
-m 512M	The DOOM zone plus WAD buffers need it; 256 MB is not enough
-smp N	Number of cores. Without it no application processor ever runs — see SMP
-no-reboot	A triple fault halts instead of looping, so you can read the last state
-cpu qemu64	The tested model. It does not expose SMEP/SMAP
-hda ext2-test.img	Auto-mounted at /mnt; holds /etc/passwd and doom1.wad
-display sdl	Chosen over GTK for Windows compatibility
-serial file:… / -serial stdio	The kernel log. See Debugging
-nic user,model=rtl8139	The only supported NIC
-d cpu_reset,int	Interrupt and reset trace, added by -Mode debug
-audiodev … -device sb16,…	SB16 audio, added by -Sound

Useful additions not wired into run.ps1:

Option	Purpose
-object filter-dump,id=f,netdev=…,file=net.pcap	Capture all traffic to a pcap. This is how the byte-order bugs were found
-monitor stdio	QEMU monitor — sendkey, screendump, info registers
-hostfwd=tcp::8080-:8080 (inside -nic user,…)	Reach a NyxOS TCP server from the host
-S -gdb tcp::1234	Halt at reset and wait for a debugger

Tip

-object filter-dump writes a pcap you can open in Wireshark. When a packet leaves NyxOS and the peer ignores it, this tells you within seconds whether the frame is malformed — usually a byte-order mistake. See Networking-Stack.

Troubleshooting

Symptom	Cause / fix
gcc: command not found	Install GCC, or set CC in kernel/Makefile
nasm: command not found	sudo apt install nasm
grub-mkrescue: not found	sudo apt install grub-pc-bin grub-common xorriso
ISO builds but QEMU shows the GRUB rescue prompt	grub-mkrescue needs xorriso; check its stderr, which build.ps1 suppresses
Changes don't take effect	Stale build — make -C kernel clean and rebuild
Triple fault on boot	Run -Mode debug and read the exception chain — see Debugging
Desktop never appears	VBE mode set failed; the kernel falls back to the text shell
DOOM exits immediately	/mnt/doom1.wad is missing; attach ext2-test.img
No network	Boot with -Mode net; check ifconfig for a DHCP lease
qemu-system-x86_64 not found	run.ps1 probes the usual install paths; add QEMU to PATH

A fuller symptom index is in Troubleshooting.

Next steps

• Debugging — the serial log, interrupt traces, and the self-test suite
• Shell — what to type once it boots
• HOWTO-Write-a-userspace-program — add your own program
• Contributing — coding standards and verification expectations

See also

• Boot-Process — what the kernel does with what GRUB hands it
• Architecture — the source layout the Makefile builds
• Troubleshooting — symptom-first index

External resources

• QEMU invocation reference — QEMU
• GRUB manual — grub-mkrescue — GNU
• Multiboot 2 specification — GNU
• GCC x86 options — GNU