Troubleshooting

Troubleshooting

A symptom-first index. Find the symptom, follow the cause, apply the fix. For the underlying techniques see Debugging.

See also: Debugging, Building, Boot-Process, Networking-Stack

Tip

Before anything else, rule out a stale build. This Makefile has shipped stale kernels through three separate mechanisms, and the symptom is always "my change had no effect".

host $ make -C kernel clean && make -C kernel

Build

Symptom	Cause	Fix
gcc: command not found	No compiler	Install GCC, or set CC in kernel/Makefile
nasm: command not found	No assembler	sudo apt install nasm
grub-mkrescue: command not found	Missing GRUB tooling	sudo apt install grub-pc-bin grub-common xorriso
make finishes but nyx-kernel.bin is unchanged	The default goal was hijacked by an included .d rule	Already fixed by .DEFAULT_GOAL := all; if you edited the Makefile, keep that line above the -include
A header change had no effect	Stale objects	-MMD -MP tracks this now; otherwise make clean
A linker.ld change had no effect	Layout change did not force a relink	linker.ld is a prerequisite of nyx-kernel.bin; keep it there
Build emits warnings	Regression	The build is expected warning-free under -Wall -Wextra; fix rather than suppress
undefined reference to a *_win function	New .o not in OBJS_KERNEL	Add it — see HOWTO-Add-a-GUI-application
Error mentioning SSE/MMX registers	Floating point in kernel code	The kernel is built -mno-sse; use integer maths
ISO builds but GRUB drops to a rescue prompt	grub-mkrescue failed quietly	Its stderr is suppressed by build.ps1; run it by hand to see the error

Boot

Symptom	Cause	Fix
Immediate reboot loop	Triple fault	Boot -Mode debug and read the exception chain — see Debugging
Log stops at [INIT] Paging...	New page tables do not map the running code	Check the identity map covers the kernel image
Log stops at [INIT] Enabling interrupts (sti)	A handler faults on the first interrupt	Check the IDT entries and the IST stacks
Log stops at [INITRAMFS] Loading...	Corrupt or stale initramfs_data.h	Regenerate with mkinitramfs.py
Boots to a text shell, no desktop	VBE mode set failed	Look for [VBE] mode set failed; the text-shell fallback is intentional
Desktop appears but nothing responds	Timer never started	Check for [TIMER] 1000 Hz; the PIT arrives on I/O APIC pin 2, not pin 0
[CPU] SMEP=unavailable SMAP=unavailable	The qemu64 model does not expose them	Not an error. Use a CPU model that does if you need them
[SB16] DSP reset failed	No SB16 device attached	Not an error. Add -Sound to run.ps1
[MODULES] 0 module(s)	No GRUB modules configured	Normal; the initramfs is embedded in the kernel
Double fault reboots instead of showing the panic screen	IST stack pointers wrong	Check the tss_entry_t offsets — see Kernel-Data-Structures

Login

Symptom	Cause	Fix
Login rejects the default credentials	/etc/passwd exists with different accounts	Detach the EXT2 image to fall back to the in-memory account, or add one with useradd
"EXT2 not available"	No disk attached	Attach ext2-test.img; run.ps1 does this automatically when it exists
Locked out after failed attempts	LOGIN_MAX_ATTEMPTS is 3, then a cooldown	Wait for the cooldown, or restart the VM
Accounts vanish after reboot	Written to the ramdisk, not the disk	The account store must be on the mounted EXT2 filesystem

Filesystem

Symptom	Cause	Fix
Writes to /mnt do not survive a reboot	Fd never closed, so the node was not flushed	vfs_close is what flushes; close the file
cannot create in a directory that exists	VFS node pool exhausted (MAX_INODES = 512); check with vfsstat	Remove files, or raise the cap
e2fsck reports errors after NyxOS wrote to the image	A real driver bug	Reproduce, then verify against e2fsck — never against the driver itself
A file read after unlink returns garbage	Node recycled while an fd still held it	open_refs guards this; check it is incremented on every path that hands out an fd
/proc/<pid> missing for a live process	proc_sync() not called on that path	It runs from vfs_open and vfs_isdir; add it to any new entry point
DOOM exits immediately	/mnt/doom1.wad missing	Attach ext2-test.img

Processes and userspace

Symptom	Cause	Fix
command not found for a program you just built	Not packed into the initramfs	Rerun mkinitramfs.py, rebuild
Program faults on entry	Missing -mno-red-zone, or linked without crt0.o	Check the link rule — see HOWTO-Write-a-userspace-program
Process exits with status 139	SIGSEGV (128 + 11)	A genuine ring-3 fault; the kernel surviving it is correct
Process exits with 143	SIGTERM (128 + 15)	It was killed
getdents/getprocs return 0 or partial data	User buffer pages not resident	Use a .bss array, or memset the buffer first
Shell has no echo after a program crashed	Raw tty mode not restored	ttymode(TTY_CANON); execve also resets it
Output disappears inside a pipeline	Diagnostics written to fd 1	Write them to fd 2
malloc succeeds but the kernel cannot write there	Lazy sbrk — pages appear on first write	Touch the buffer before passing it to the kernel
Background job never reaped	Parent never called waitpid	The shell reaps with WNOHANG at each prompt; orphans go to the background reaper
Foreground job freezes the desktop	Blocking on the compositor thread	Use sleep(ms), or run it as a scheduled process

SMP

Symptom	Cause	Fix
cpus shows only CPU 0	QEMU started without -smp	run.ps1 -Cpus 4
AP tick counters stay at zero	AP LAPIC timer not programmed	Check smp_start_ap_threads() ran
Correct on one core, corrupts on four	preempt_disable() where a spinlock is required	Take a real spinlock — see SMP
Hang once threads are spread across cores	Per-CPU state answered from the BSP	get_current_process() must resolve per-CPU
Stale data read on another core after a remap	Missing TLB shootdown	Call tlb_shootdown() on every path that removes or restricts a mapping
Deadlock during a shootdown	Two cores waiting on each other	The shootdown lock must be taken without disabling interrupts
Instability under smpbalance on with heavy fork/exec	Known limit	Leave balancing off; thread groups stay pinned to one core

Networking

Symptom	Cause	Fix
No interface in ifconfig	No NIC attached	run.ps1 -Mode net, or add -nic user,model=rtl8139
DHCP never completes	NIC not receiving	Check [RTL8139] Initialized successfully in the log
Every packet silently dropped by the peer	A header field stored in the wrong byte order	Route it through htons/htonl — this is the single most common bug in this subsystem
ping gets no reply	Off-subnet without a gateway	Check ifconfig; ip_send routes off-subnet via the gateway
DNS times out	Wrong server, or no lease	Verify with ifconfig, then dns <host>
TCP connects then resets	Wrong ACK number	Ethernet padding counted as payload; payload_len must come from the IP header, not the frame
Retransmission never fires	RTO path not exercised	tcpdrop <n> forces it
Connections beyond 32 fail	TCP_MAX_CONNS reached	Raise the cap and rebuild; a header-only change now rebuilds correctly
Concurrent multi-process sockets garble	Historically a capacity limit masked by a stale build	Rebuild clean, then check MAX_SOCKETS

TLS and HTTPS

Symptom	Cause	Fix
TLS: cannot resolve <host>	DNS failed	Check the lease and the DNS server
Handshake stops after ServerHello	Cipher suite not offered	See the offered list in Cryptography-and-TLS
Handshake fails on a site that works elsewhere	Server requires a group NyxOS does not offer	Only x25519, secp256r1 and secp384r1 are supported
Chain reports X509_INCOMPLETE	Root not in the bundled trust store	Expected — five anchors are bundled. Not a forgery
Chain reports X509_FORGED	A link failed cryptographic verification	Treat as an attack or a genuinely broken chain
Certificate rejected as out of date	RTC wrong	Check date; validity is checked against the RTC
Strict mode refuses everything	Working as designed	tlsstrict off for development
Large page truncated	Historically a non-streaming receive	Streaming receive is implemented; check the output buffer capacity

GUI

Symptom	Cause	Fix
Window opens blank	No draw callback	Pass it to window_create
Clicks ignored	Callbacks assigned when the context allocation failed	Assign them inside if (win->reserved)
Elements offset, or scaled twice	Layout derived from fb_get_width()	Use the 1024×768 design grid
Desktop sluggish	on_tick always returns 1	Return 0 when idle
Two windows of one app share state	State in a static	Move it into the per-window context
Desktop flickers	Double buffering off	fb_enable_backbuffer(); login deliberately draws direct
Icons garbled or a fault at startup	Icon arrays out of sync with NUM_DESKTOP_ICONS	Update all three together
Windows stranded off-screen after a resolution change	Geometry not re-flowed	init_desktop_icons() and the snap re-derivation handle this
Arrow keys do nothing anywhere	The keyboard E0-prefix check ran after the press-bit mask	Fixed in v5.8.16; check that ordering if you touch keyboard.c

Getting help

If none of the above matches, gather this before asking:

1. qemu_serial.txt from the failing boot
2. The exact run.ps1 invocation, including -Cpus
3. Output of version and uname
4. Whether a clean rebuild changes the behaviour
5. Whether -Cpus 1 changes the behaviour

Report at https://github.com/kazah-png/nyx-os/issues. Security issues go to the address in SECURITY.md.

See also

• Debugging — the techniques behind these fixes
• Building — build requirements and flags
• Boot-Process — what each stage does
• Contributing — how to submit a fix

External resources

• QEMU networking documentation — QEMU
• OSDev Wiki — Troubleshooting — generic OS-development failure modes