Security

Security

NyxOS is a hobby operating system and offers no security guarantees. It has never been audited by anyone outside the project, and it should not be trusted with anything that matters. What follows is an honest description of the boundaries that do exist, and of the ones that do not.

See also: Syscalls, Memory Management, Process Management, Cryptography and TLS, Architecture

Privilege boundaries

Ring	Runs	Can reach
0	Scheduler, drivers, VFS, network stack, compositor	All physical memory; user memory only through copy_from_user/copy_to_user
3	ELF programs	Its own mapped pages, and the kernel only through syscall

Memory protection

Mechanism	Effect
Page-table isolation	The kernel occupies the top PML4 entry; user page tables carry no identity mapping of physical memory, so a process cannot name kernel memory even by guessing
NX	User stack, heap and data pages are mapped non-executable
SMEP	Ring 0 cannot execute a ring-3 page — enabled at boot by enable_smep_smap()
SMAP	Ring 0 cannot access a ring-3 page outside an explicit window — also enabled at boot
CR0.WP	Supervisor writes to read-only pages fault; required for copy-on-write correctness
Guard page	Below the user stack's growth ceiling, so an overflow faults instead of running into other mappings
Page 0 unmapped	USER_SPACE_MIN is 0x1000, so a NULL dereference always traps

Syscall boundary

This is the only path from ring 3 into the kernel and is where most hardening lives.

• Pointer validation. user_ptr_ok() rejects anything outside [0x1000, 0x800000000000).
• Opaque file descriptors. Userspace gets small integers indexed into a per-process table; kernel VFS handles are never exposed. Before this, ring 3 received raw kernel pointers — a combined information leak and arbitrary-kernel-read/write reachable from any program.
• copy_from_user / copy_to_user. These walk the user page tables to a physical address and bounce through the higher-half alias, rather than dereferencing a user pointer directly.
• Interrupts masked. Syscalls run with IF=0.
• Per-CPU entry state. Saved ring-3 registers live in a per-CPU block, so two cores entering the kernel simultaneously cannot clobber each other.
• Flags sanitised. Every interrupt and syscall entry path issues cld. Ring 3 could otherwise leave RFLAGS.DF set and reverse the direction of every string operation the kernel performs.
• ELF validation. Headers and program headers are checked before anything is mapped; libseg_load (the shared-library path) once validated nothing at all.

Fault isolation

A ring-3 fault does not panic the kernel. Page faults, divide errors and illegal instructions are converted into SIGSEGV, SIGFPE and SIGILL and delivered to the faulting process. With a handler installed the program can recover — with sigsetjmp/siglongjmp it can survive repeated faults; without one it dies with status 128 + signo and the system carries on.

Authentication

Accounts live in /etc/passwd on the EXT2 disk, one line per user:

user:salt_hex:iterations:hex_hash:avatar

Property	Value
KDF	PBKDF2-HMAC-SHA256
Iterations	10 000, stored per entry so the work factor can be raised without breaking existing accounts
Salt	Random 8 bytes per user, generated from RDTSC jitter, the tick counter and the RTC
Minimum password length	4 (AUTH_MIN_PASS)
Failed-login lockout	3 attempts (LOGIN_MAX_ATTEMPTS), then a cooldown

Accounts persist across reboots and can be created from the login screen. With no EXT2 disk attached, an in-memory fallback account is used so the machine still boots.

This is not strong authentication. The salt generator is not a CSPRNG, 10 000 PBKDF2 iterations is low by modern standards, and the disk is unencrypted — anyone with the image has the hashes.

TLS

The TLS 1.2 stack verifies certificate chains against pinned trust anchors, checks hostnames against subjectAltName, checks validity dates, and verifies the ServerKeyExchange signature. In strict mode (tlsstrict on) a handshake that fails any of those checks is refused rather than warned about.

It is off by default, it has five bundled trust anchors rather than a real root store, and none of the cryptographic implementations have been reviewed by anyone else. Constant-time behaviour is not claimed. See Cryptography and TLS.

What is not defended

Being explicit about this is more useful than a list of features:

• No IOMMU. A malicious PCI device has full DMA access.
• No Spectre, Meltdown or MDS mitigations. No retpolines, no KPTI, no speculation barriers.
• No ASLR. Programs load at fixed addresses; shared libraries are at fixed addresses by design, which is why the dynamic loader performs no relocation.
• No stack canaries. -fno-stack-protector is a build flag.
• No filesystem permissions. Mode bits are stored but not enforced; any process can read or write any file.
• No process privilege model. There is no root/non-root distinction after login, no capabilities, no setuid.
• Minimal network hardening. The TCP/IP stack is not defended against malformed or adversarial packets.
• No audit or side-channel review of the cryptographic code.
• No secure boot, no disk encryption.

Ongoing work

The v5.9.0-rc* release series was a systematic security and correctness audit; each release in it closes a specific confirmed finding, including a ring-3 → ring-0 arbitrary write, missing ELF validation, a double free in dup(), and ring-3 flags surviving into the kernel. v5.9.0-LTS is the snapshot at the end of that pass. See Version History.

Security reports go to the address in SECURITY.md at the repository root.