Memory Management

Memory Management

Four layers: a bitmap physical page allocator, 4-level paging with demand faulting, a kernel heap, and slab caches for small objects. On top of them sits the userspace VM — lazy sbrk, mmap VMAs, copy-on-write fork, and demand-grown stacks.

See also: Architecture, Boot Process, Process Management, SMP, Security

Physical allocator (memory.c)

A bitmap with one bit per 4 KB frame. alloc_page() returns a physical address, free_page() releases one.

Firmware-aware. The allocator is given the Multiboot 2 memory map — every {base, length, type} entry the bootloader reported — rather than assuming a flat region of RAM. Regions the firmware did not mark as available are never handed out. This matters: handing out a reserved page produced a memory-corruption bug that took many releases to pin down, because the symptom (a corrupted shell pipeline) was nowhere near the cause.

Reference counting. Each frame carries a share count. page_incref() bumps it when a frame becomes shared — by fork's copy-on-write clone, or by the shared libc mapped into every process — and free_page() only actually frees at the last reference. Without this, one process exiting would unmap pages still live in its sibling.

Allocation is protected by a spinlock taken with interrupts disabled, not by preempt_disable(). The distinction is the whole point on a multi-core machine: disabling preemption stops a context switch on the local core and means nothing to another one. The smpstress command hammers the allocator from every core at once to prove it holds.

Paging (paging.c)

Standard x86_64 4-level translation: PML4 → PDPT → PD → PT → 4 KB page. The kernel lives in the top PML4 entry and is mirrored into every address space, so an interrupt or syscall taken in any process always finds its code and stacks mapped. All physical RAM is aliased at KERNEL_BASE (0xFFFFFF8000000000) — the identity map covers all detected RAM, not a fixed 64 MB window as it once did.

Software PTE bits

The x86 page-table format leaves bits 9–11 available to the OS. NyxOS uses them to record why a page is absent or restricted:

Bit	Name	Meaning
9	PTE_DEMAND	Not yet allocated; supply a zeroed frame on first touch
10	PTE_COW	Shared read-only; make a private copy on write

vm_handle_fault

The page-fault handler is the single place where lazy memory becomes real. It resolves:

• Demand-zero pages — a PTE_DEMAND entry becomes a fresh zeroed frame
• Copy-on-write — a write to a PTE_COW page allocates a private copy and drops the shared reference
• Lazy sbrk — a fault inside [heap_start, program_break) materialises a heap page
• File-backed mmap — the faulting page's slice is copied out of the VMA's snapshot buffer
• Stack growth — a fault just below the current stack extends it (see below)

Anything else in the user half is a genuine fault, and becomes a SIGSEGV delivered to the process rather than a kernel panic (see Process Management).

Control-register state

• CR0.WP — supervisor writes to read-only pages fault. Required for copy-on-write to work when the kernel writes into a user page.
• NX — user stack, heap and data pages are mapped non-executable.
• SMEP / SMAP — enabled at boot; see Security.

User address space

Demand-grown stacks

User stacks start at USER_STACK_TOP (0x00007FFFFFFFE000) and grow down.

Committed at load	4 pages (16 KB)
Ceiling	128 pages (512 KB)
Guard page	The page below USER_STACK_LOW is never mapped

A fault in the growth window extends the stack by a page; a fault below the guard page is a real stack overflow and kills the process. Before this, every process pre-committed a fixed 64 KB whether it needed it or not.

Lazy sbrk

SYS_SBRK moves program_break without allocating anything. Pages appear on first write. A malloc(8000) therefore costs only the pages the program actually touches, and the break is inherited across fork.

mmap (mmap.c)

VMAs are recorded in a per-thread-group table; mappings live in [4 GiB, 112 TiB), clear of both the heap and the stack.

Call	Behaviour
mmap anonymous	MAP_PRIVATE demand-zero pages; prot honoured (writable only with PROT_WRITE, NX unless PROT_EXEC)
mmap file-backed	The file is snapshotted into a per-VMA kernel buffer at the given offset; faulting pages copy their slice out of it
mprotect	Rewrites the flags of present pages and the VMA's recorded prot. A partial-range call splits the VMA so the change is precise
munmap	Frees present pages (refcount-aware) and drops the VMAs; a partial-range call splits the VMA rather than silently doing nothing

Mappings are inherited copy-on-write across fork (file buffers are deep-copied), and dropped by execve.

Every path that removes or restricts a mapping — munmap, mprotect, the COW remap — issues a TLB shootdown IPI so no other core keeps a stale translation. See SMP.

Copy-on-write fork

clone_page_directory_cow() walks the user half of the parent's PML4:

• Writable leaf pages are downgraded to read-only + PTE_COW in both parent and child, and the frame's refcount is bumped
• Read-only pages (program text) are shared as-is
• The first write by either side faults, vm_handle_fault allocates a private copy, and the shared reference is dropped

The child returns 0 from fork(), the parent returns the child's pid, and both continue in the round-robin.

Threads are different. clone(CLONE_VM) deliberately does not copy the address space — the new task shares it. See Process Management.

Kernel heap (heap.c)

A 16 MB heap at KERNEL_HEAP_START with a free-list, header/footer metadata and coalescing on free.

kmalloc routes small requests to the slab allocator and falls back to the heap when no slab class fits. Allocations carry one of two header magics (_SLAB / _HEAP) so kfree routes correctly regardless of size. This is not a detail: for several releases kmalloc sent everything ≤ 1024 B to a slab that only had classes up to 512 B, so every allocation between 505 and 1016 bytes silently returned NULL — which showed up as files that wrote successfully but came back empty.

Slab allocator (slab.c)

Fixed-size object caches at 8, 16, 32, 64, 128, 256, 512 and 1024 bytes (SLAB_MAX_OBJ). Anything larger goes to the heap. Like the physical allocator, the slab caches are spinlock-protected for SMP safety.

Inspecting memory at runtime

Where	What
mem (kernel shell)	Physical page usage and heap statistics
free (ring 3)	Reads /proc/meminfo
pmap [pid] (ring 3)	Reads /proc/<pid>/maps — the process's mapped regions
cowtest (kernel shell)	Self-test for demand paging and copy-on-write
tlbtest (kernel shell)	Proves cross-CPU TLB shootdown works, by first showing a stale read without it