Boot Process

Boot Process

From GRUB handing off control to a logged-in desktop. Every stage below is real code you can follow: kernel/core/boot.asm for stages 1–2, kernel_main() in kernel/core/kernel.c for stages 3–6.

See also: Architecture, Memory-Management, SMP, GUI-Subsystem, Security

Stage 1 — GRUB

boot.asm publishes two Multiboot headers: a Multiboot 1 header (0x1BADB002) kept for compatibility, and a Multiboot 2 header (0xE85250D6), which is what the generated grub.cfg actually uses:

menuentry 'NyxOS' {
    multiboot2 /boot/nyx-kernel.bin
    boot
}

GRUB loads the kernel, leaves the CPU in 32-bit protected mode, and jumps to _start with the boot magic in EAX and the info-structure pointer in EBX. Both are stashed immediately, because the long-mode climb clobbers general registers.

Note

UEFI and real hardware (v6.4.223+). A GRUB-EFI ISO builder boots NyxOS under UEFI (OVMF and real firmware), and the display comes up on the GOP framebuffer GRUB provides (see the Display stage). Real-hardware bring-up also fixed a boot #PF (the Multiboot 2 info had been read as Multiboot 1), bounded the PS/2 and RTC polling loops with timeouts to stop boot hangs, added PS/2 mouse polling and MouseKeys for machines that don't deliver IRQ12, and a rotate= cmdline for portrait panels. An installed disk (see Installation) boots the same way, BIOS or UEFI.

Stage 2 — Protected mode → long mode (boot.asm)

1. cli; save EAX/EBX (Multiboot magic and info pointer)
2. Set up a minimal 32-bit GDT
3. Confirm long-mode support via CPUID leaf 0x80000001
4. Enable PAE in CR4
5. Build temporary page tables — PML4 → PDPT → two page directories covering the low 128 MB with 2 MB pages
6. Load CR3
7. Set EFER.LME
8. Enable paging (CR0.PG) — the CPU is now in long mode
9. Load the 64-bit GDT (ring-0 code/data + ring-3 code/data)
10. Far jump to reload CS, then load the data segment registers
11. Point RSP at the 128 KB .bss boot stack
12. Zero BSS and call kernel_main(magic, mboot_ptr)

The boot stack is 128 KB and is placed before the page tables in .bss. It was 16 KB and sat directly below them, so a deep compositor redraw chain could overflow into the live page tables and corrupt them — a bug that presented as crashes at unrelated addresses.

Stage 3 — kernel_main()

Initialisation runs in a fixed order, because each step depends on the ones above it. The 23-step progress bar on the splash screen is driven by bootsplash_update() calls threaded through this sequence.

Early — no memory manager yet

init_screen() → clear_screen() → init_gdt() → init_idt() → init_isr()
→ init_irq() → init_serial() → enable_sse_fpu()

Memory

The Multiboot info block is parsed here. For Multiboot 2 the memory-map tag is copied out entry by entry (base, length, type) and handed to the allocator, so the physical allocator honours real firmware holes instead of assuming a flat region.

init_memory(mem_total, mmap, mmap_count) → init_paging() → enable_smep_smap()

Core services

init_apic() → init_heap() → slab_init_all() → smp_init()

smp_init() runs here — before the timer — because the IST allocation below needs to know how many cores are online.

Display

vbe_init() → fb_init(w, h, bpp, lfb) → bootsplash_init()

Since v6.4.222 the display comes up on the framebuffer the bootloader provides — the Multiboot 2 framebuffer tag (GRUB fills it via GOP on UEFI, or VBE on BIOS) is parsed in stage 3 and its lfb/w/h/bpp handed straight to fb_init. This is what lets NyxOS boot graphically on real UEFI/GOP hardware rather than only on QEMU's Bochs VBE adapter, which is now used only for runtime mode changes. If no framebuffer is available at all, the kernel stays in text mode and later drops to the serial shell instead of the desktop.

Per-CPU exception stacks

One double-fault and one NMI IST stack per online core (8 KB each), so a fault on one core cannot scribble on another core's exception stack.

Scheduling, syscalls, filesystems

init_timer(1000) → init_keyboard() → init_process() → ensure_idle_process()
→ smp_start_ap_threads() → init_syscalls() → cpu_install_gs_base(0)
→ setup_syscall_msrs() → [allocate syscall kernel stack]
→ init_vfs() → init_load_modules() → init_ext2()

cpu_install_gs_base() must run before anything can issue a syscall: the entry stub reaches its per-CPU scratch slots through GS and has no spare register to find them any other way.

Network

init_net() → tcp_init() → tcp_echo_init() → [auto-DHCP if a NIC is present]

Auto-DHCP leases an address at boot so ping, httpget and Selene-Browser work without a manual dhcp. It is skipped entirely when no NIC is present, so a diskless/netless boot never stalls.

Devices and interrupts

init_background_tasks() → mouse_init() → speaker_init() → sb16_init()
→ rtc read → irq_install_handler(1, 5, 12)
→ ioapic_redirect_irq(2, 32, apic_id) → ioapic_unmask_irq(2)
→ irq_unmask(1|5|12) → sti

The PIT arrives on I/O APIC pin 2, not pin 0 — QEMU's ACPI interrupt-source override remaps ISA IRQ 0. Unmasking pin 0 was a no-op, which is why tick_count stayed at zero for several releases.

Userspace

initramfs_load() → initramfs_boot() → shared_libc_load() → [register /init.elf]
→ ext2 probe/mount at /mnt

shared_libc_load() maps the single prelinked libc.so that every user program links against (see Userspace).

Stage 4 — Boot splash

Roughly five seconds: the NyxOS crescent-moon logo, a starfield, a spinner, and a 23-step progress bar whose captions come from the bootsplash_update() calls above. Fades to black before the login screen.

Stage 5 — Login

auth_setup() then login_screen(), drawn straight to the linear framebuffer (no back buffer). The screen offers a username/password card and a create-account panel with profile-picture selection.

• Credentials live in /etc/passwd on the EXT2 disk, one line per user: user:salt_hex:iterations:hex_hash:avatar
• Hashing is PBKDF2-HMAC-SHA256, 10 000 iterations, with a random 8-byte per-user salt
• LOGIN_MAX_ATTEMPTS is 3, after which the screen enters a lockout cooldown
• With no EXT2 disk attached, an in-memory fallback account is used so the machine still boots

See Security for the threat model.

Stage 6 — Desktop

create_process("compositor", compositor_run, 0)   // registered with the scheduler
loop:
    fb_use_lfb_direct(); login_screen()
    compositor_init(); compositor_run()
    if (!compositor_logout_requested) break

The compositor comes up with 10 desktop icons, a taskbar, a Start menu and 4 workspaces. Choosing Log out from the taskbar user menu returns to the login screen without rebooting — compositor_init() frees the previous session's windows first, so the next user starts on a clean desktop.

If no framebuffer was available, launch_shell() runs the kernel shell over serial instead. See Shell.

Boot protocol contract

For anyone loading this kernel with their own bootloader instead of GRUB — what _start requires, precisely.

It is not loaded from a fixed disk sector. GRUB reads /boot/nyx-kernel.bin as an ISO9660 file and maps it according to its own ELF program headers — there is no sector-based loading contract to replicate. What is fixed is the load address and the program-header layout, from kernel/linker.ld:

Property	Value
File format	ELF64, ET_EXEC, x86-64, LSB
Load base	0x100000 (1 MiB)
Entry point	0x100040
PT_LOAD segments	2

TYPE   OFFSET     VADDR      FILESZ     MEMSZ      FLAGS
LOAD   0x1000     0x100000   0x4983d    0x4983d    R-X   align 0x1000
LOAD   0x4a840    0x149840   0x12f7c0   0x129a4c0  RW-   align 0x1000

A minimal loader copies each segment's FILESZ bytes from OFFSET to VADDR. The Multiboot header lives inside .text's KEEP(*(.multiboot)), so it is within the first 32 KiB of the file.

Warning

MEMSZ (≈18.9 MB) is far larger than FILESZ (≈1.47 MB) — the difference is .bss, captured deliberately (see the comment in linker.ld about page_refcount[]/page_pinned[]). Do not place a memory map, module, or any loader-owned structure inside [0x100000, 0x13E3D00); boot.asm zeroes this range itself before calling kernel_main.

CPU state _start assumes on entry:

Requirement	Value
Mode	32-bit protected, paging off
EIP	0x100040
EAX	Multiboot magic (0x36D76289 for Multiboot 2)
EBX	Physical address of the Multiboot info structure
A20	Enabled
GDT	Flat 32-bit (the kernel loads its own within microseconds)
Interrupts	Disabled

Not EFI. The published ISO carries an El Torito boot record and a 55AA boot signature, but no BOOTX64.EFI / efi.img — it is BIOS-only. _start publishes both a Multiboot 1 header (0x1BADB002, compatibility) and a Multiboot 2 header (0xE85250D6, what grub.cfg actually invokes). The linker script's filename (linker.ld, not kernel_entry.ld) has no bearing on the boot protocol — that is determined entirely by the .multiboot header content and the CPU state above.

Note

kernel_main(magic, mboot_ptr) does not gate on magic — it is used only to choose the Multiboot v1 vs v2 info-parsing path. Passing garbage in EAX will not be rejected; it will boot incorrectly instead. Passing a genuine memory map matters more than the magic does: without one, the physical allocator falls back to a flat guess and stops respecting firmware-reserved regions, which is the exact class of bug documented in Memory-Management.

Targeting UEFI instead is a separate rewrite, not a drop-in: EFI hands off with boot services live, no Multiboot structure, and the CPU typically already in long mode. The cheapest bridge is an EFI loader that calls ExitBootServices and then synthesizes the Multiboot 2 handoff above by hand.

See also

• Building - producing the ISO GRUB boots
• Debugging - reading a boot log, and what a stalled stage means
• SMP - application-processor bringup
• Security - the login and credential model

External resources

• Multiboot 2 specification - GNU
• Intel SDM Vol. 3A, Ch. 9 - Processor Management and Initialization - Intel
• OSDev Wiki - Setting Up Long Mode - OSDev