Skip to content

Cryptography and TLS

Jump to bottom
kazah-png edited this page Aug 25, 2026 · 7 revisions

Cryptography and TLS

NyxOS speaks TLS 1.2 to real HTTPS servers, using cryptographic primitives written from scratch in the kernel. It performs an ECDHE key agreement, encrypts with AES-128-GCM, verifies the server's ServerKeyExchange signature, and validates the certificate chain to a pinned trust anchor — with hostname and validity-date checks.

Since the v6.4 crypto run the kernel also carries a general-purpose cryptographic library that goes well past what TLS needs — modern AEADs and stream ciphers, several hash and MAC families, an HKDF, and Ed25519 — every one landed with a known-answer test. See The general-purpose crypto library below.

Every primitive here has a known-answer self-test runnable from the shell, checked against the published vectors for that algorithm.

Do not trust this with anything real. These implementations have never been reviewed by anyone outside the project, no constant-time behaviour is claimed, and the trust store holds five anchors rather than a real root set. See Security.

See also: Networking-Stack, Selene-Browser, Security, Shell

The primitives

Area File Provides Self-test
Hashing sha256.c SHA-256, HMAC-SHA256, PBKDF2 —
Hashing sha512.c SHA-512, SHA-384 (FIPS 180-4) sha512test
AEAD aes_gcm.c AES-128-GCM seal/open gcmtest
ECDH curve25519.c X25519 (RFC 7748) x25519test
ECC p256.c secp256r1 arithmetic, ECDSA-P256 verify p256test
ECC p384.c secp384r1 arithmetic, ECDSA-P384/SHA-384 verify p384test
RSA rsa.c PKCS#1 v1.5 and PSS signature verification rsatest
RNG csprng.c HMAC_DRBG-SHA256 (NIST SP 800-90A) csprngtest
ASN.1 der.c DER reader dertest
PKI x509.c Certificate-chain verification, SAN and date checks chaintest
KDF tls_prf.c TLS 1.2 PRF, P_SHA256 prftest
Protocol tls.c Handshake, key schedule, record layer tlskeytest, tlsrectest, skp384test
OTP totp.c TOTP / HOTP one-time passwords (RFC 6238 / 4226) KAT
Encoding base64.c Base64 codec (RFC 4648) KAT
Encoding base32.c Base32 codec (RFC 4648) KAT
Encoding utf8.c Strict UTF-8 validator / decoder KAT
Checksum (crc32) CRC-32 (used by PNG chunk validation) CI self-test

Note

The v6.4 primitive-and-KAT run. A late v6.4 sweep added HMAC-based TOTP/HOTP (v6.4.120), strict Base64/Base32 codecs (v6.4.103, v6.4.117), a UTF-8 validator (v6.4.113), CRC-32 (v6.4.96, wired into PNG chunk validation at v6.4.97) and a deterministic 64-bit primality test (v6.4.93) — and retrofitted known-answer tests onto existing code, including locking the password KDF PBKDF2-HMAC-SHA256 with a KAT (v6.4.106). The theme is regression-proofing: every primitive now fails loudly if it drifts.

The general-purpose crypto library

v6.4.121–v6.4.182 grew a broad, RFC-vectored crypto toolbox — most of it not on any TLS path. Every entry ships with a known-answer test, and the verify paths use a constant-time compare (ct_memcmp, v6.4.156, in ct.c).

Category Primitive File Reference
Hash SHA-1 (+ HMAC-SHA1) sha1.c FIPS 180 / RFC 2202 (v6.4.167)
Hash SHA3-256 (Keccak sponge) sha3.c FIPS 202 (v6.4.154)
Hash BLAKE2s-256 (+ keyed MAC mode) blake2s.c RFC 7693 (v6.4.140, v6.4.142)
Hash MD5 (legacy interop) md5.c RFC 1321 (v6.4.173)
Stream ChaCha20 chacha20.c RFC 8439 (v6.4.132)
AEAD ChaCha20-Poly1305 chacha20poly1305.c RFC 8439 (v6.4.138)
MAC Poly1305 one-time authenticator poly1305.c RFC 8439 (v6.4.136)
MAC AES-CMAC (over AES-128) aes_cmac.c RFC 4493 (v6.4.146)
MAC SipHash-2-4 (hash-flooding defence) siphash.c (v6.4.134)
Cipher AES-128-CTR aes_ctr.c NIST SP 800-38A (v6.4.150)
Cipher AES-128-CBC + inverse cipher aes_cbc.c NIST SP 800-38A (v6.4.159)
Key wrap AES Key Wrap aes_kw.c RFC 3394 (v6.4.180)
KDF HKDF-Extract/Expand (HMAC-SHA256) hkdf.c RFC 5869 (v6.4.130)
Public key Ed25519 key derivation and signature verify (TweetNaCl port) ed25519.c RFC 8032 (v6.4.182, v6.4.183)
OTP TOTP/HOTP-SHA1 (authenticator default) totp.c RFC 6238 (v6.4.168)
Encoding Base16 / hex codec base16.c RFC 4648 (v6.4.144)
Encoding Ascii85 / base85 codec — (v6.4.195)
Encoding Base58 (Bitcoin) codec — (v6.4.211)
Encoding bech32 checksummed base-32 codec — BIP-173 (v6.4.214)
Encoding URL percent-encoding codec — RFC 3986 (v6.4.202)
Coding LEB128 varint (signed + unsigned) — (v6.4.170)
Checksum CRC-16/CCITT-FALSE, CRC-32C (Castagnoli) — (v6.4.177, v6.4.205)
Checksum Fletcher-16/32, MurmurHash3-32, FNV-1a — (v6.4.218, v6.4.221, v6.4.199)

Three userland digest coreutils sit on top: sha256sum (v6.4.141), sha512sum (v6.4.145) and md5sum (v6.4.175), matching the GNU output format.

Applied crypto commands landed too: hmac (HMAC-SHA256, RFC 4231 KAT, v6.4.289), totp authenticator codes from a Base32 secret (v6.4.299), uuid — RFC 4122 v4 from the CSPRNG (v6.4.323), and encrypt/decrypt — password-based file encryption with AES-128-GCM + PBKDF2 (v6.4.334). /dev/urandom is now served from the same CSPRNG (v6.4.341, see Filesystem).

Warning

A constant-time compare and a KAT are not a security review. None of this has been audited, side-channel behaviour beyond ct_memcmp is not claimed, and MD5/SHA-1 are provided for interop only — not because they are safe. The do-not-trust warning at the top still stands.

Randomness

csprng_bytes(out, n) is an HMAC_DRBG-SHA256, instantiated on first use from the best available hardware entropy — RDSEED/RDRAND, plus RDTSC jitter and the tick counter — and periodically reseeded. The TLS ClientHello random and every ephemeral key come from it.

This is a different thing from /dev/random, which is a plain xorshift64 PRNG suitable for scripts and nothing else. See Filesystem.

AEAD

aes128_gcm_encrypt(key, iv12, aad, aad_len, pt, pt_len, ct, tag16);
int ok = aes128_gcm_decrypt(key, iv12, aad, aad_len, ct, ct_len, tag, pt);

decrypt verifies the tag first and only writes the plaintext if it is valid — on failure the output buffer is left untouched.

Signature verification

int p256_ecdsa_verify(Qx, Qy, hash, r, s);              // 0 = valid
int p384_ecdsa_verify(Qx, Qy, hash48, r, s);
int rsa_pkcs1_sha256_verify(N, Nlen, e, sig, siglen, hash);
int rsa_pkcs1_sha512_verify(N, Nlen, e, sig, siglen, hash);   // v5.9.92
int rsa_pss_sha256_verify (N, Nlen, e, sig, siglen, mHash);

The two RSA PKCS#1 verifiers share their work through rsa_recover_em + rsa_pkcs1_check — recover EM = sig^e mod N, then check 00 01 FF… 00 ‖ DigestInfo ‖ hash — differing only in the DigestInfo prefix and hash length. rsatest covers all of it (7/7): PKCS1-SHA256 ×3, PSS ×2, and PKCS1-SHA512 accept + tampered-reject.

The TLS 1.2 handshake

ClientHello        →   TLS 1.2, CSPRNG random, SNI, cipher list, supported_groups
                   ←   ServerHello          (chosen cipher suite)
                   ←   Certificate          (the chain)
                   ←   ServerKeyExchange    (ECDHE params + signature)
                   ←   ServerHelloDone
                        │
                        ├─ verify the chain to a pinned root      (x509.c)
                        ├─ check the hostname against the SAN     (x509.c)
                        ├─ check the validity dates against RTC   (x509.c)
                        └─ verify the SKE signature under the leaf key
                        │
ClientKeyExchange  →   our ECDHE public value
ChangeCipherSpec   →
Finished           →   verify_data over the handshake transcript
                   ←   ChangeCipherSpec
                   ←   Finished
        ═══════ AES-128-GCM application records ═══════

Offered cipher suites

Value Suite
0xC02F ECDHE_RSA_WITH_AES_128_GCM_SHA256
0xC02B ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
0xC030 ECDHE_RSA_WITH_AES_256_GCM_SHA384
0xC02C ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
0x009C / 0x009D RSA_WITH_AES_*_GCM_*
0x002F / 0x0035 RSA_WITH_AES_*_CBC_SHA

Supported groups

x25519, secp256r1 (P-256), secp384r1 (P-384). P-256 matters in practice — a large fraction of servers prefer it and will not negotiate x25519.

ServerKeyExchange signature schemes

For an ECDHE handshake the server signs SHA-x(client_random ‖ server_random ‖ ServerECDHParams) with its certificate key. Since v5.9.92, NyxOS verifies every common scheme, so strict mode can enforce key-exchange authenticity whatever the server chooses:

Value Scheme
0x0401 RSA PKCS#1 v1.5 / SHA-256
0x0403 ECDSA-P256 / SHA-256
0x0503 ECDSA-P384 / SHA-384
0x0804 RSA-PSS / SHA-256
0x0601 RSA PKCS#1 v1.5 / SHA-512

Note

0x0503 (ECDSA-P384) and 0x0601 (RSA-SHA512) are rare on today's web. Their tls.c dispatch branches are KAT-proven and structurally identical to the live-verified RSA/ECDSA branches, but no reachable server was found that selects them, so they are not yet exercised end-to-end against a live peer.

Key schedule and records

The TLS 1.2 PRF (P_SHA256) derives the master secret from the ECDHE pre-master secret and the two randoms, then expands it into the key block. Records are framed on the wire as:

explicit_nonce(8) || GCM_ciphertext || GCM_tag(16)

tlskeytest and tlsrectest check the schedule and the record layer, including the Finished verify_data, against known answers.

The trust model (x509.c)

Chain verification takes the certificates the server sent, verifies each under the next one's public key, and requires the top to carry a pinned trusted-root public key from x509_roots.h.

Result Meaning
X509_OK (0) Every link verified and the top is a pinned trusted root
X509_INCOMPLETE (1) Links checked, but not anchored — unknown root or an unsupported signature algorithm. Not a detected forgery
X509_FORGED (−1) A link using a supported algorithm failed cryptographic verification

That three-way distinction is deliberate. "I could not check this" and "I checked this and it is forged" are different statements, and collapsing them would either cry wolf or hide an attack.

Warning

basicConstraints bypass (fixed v6.4.69). x509_verify_chain did not require an intermediate to be a CA certificate, so a valid end-entity leaf could be used to sign for another host — a classic basicConstraints confusion. A non-CA certificate is now rejected as a chain issuer.

Two further checks complete "verify the peer":

  • x509_check_host — matches subjectAltName dNSName entries case-insensitively, honouring a single leading *. wildcard label
  • x509_check_validity — checks the certificate is inside its validity window, per the RTC

Note

DER date-parser hardening (v5.9.106). der.c's parse_time (the UTCTime/GeneralizedTime parser feeding the validity check) accepted a 'Z' in any of the first 14 bytes, not just as the trailing timezone marker — so a malformed Time with 'Z' in a real date slot parsed to a bogus-but-bounded date instead of being rejected. It now requires the exact digit count (12 for UTCTime, 14 for GeneralizedTime) with every date position an actual 0-9 digit, reading each field in unsigned arithmetic. Well-formed certificates are unaffected; the whole file is now -Wsign-conversion-clean. This parser reads attacker-controlled certificate bytes, so it is exactly the kind of surface the maintenance pass targeted — see Security.

Bundled trust anchors

Five, in x509_roots.h:

Anchor Key
SSL.com TLS ECC Root CA 2022 EC
Amazon Root CA 1 RSA-2048
DigiCert Global G2 TLS RSA SHA256 2020 CA1 RSA-2048
GTS Root R4 EC
Root YR RSA-4096

chaintest runs a real chain (accept), a one-byte-tampered copy (reject as forged), and a root-less prefix (reject as not-anchored).

Strict mode

tlsstrict on

Off by default. When on, a handshake is refused unless:

  1. The chain anchors to a pinned root
  2. The hostname matches the certificate
  3. The certificate is in date
  4. The ServerKeyExchange signature verifies

With it off, failures are reported and the connection proceeds — useful for development, useless as a defence. Strict mode is what turns the trust model from "reported" into "blocked".

API

int tls_hello(const char* host, int iface_idx);

int tls_https_fetch(const char* host, const char* path, int iface_idx,
                    uint8_t* out, uint32_t cap, int verbose);

int tls_https_request(const char* host, const char* path, const char* method,
                      const uint8_t* body, uint32_t body_len, int iface_idx,
                      uint8_t* out, uint32_t cap, int verbose);

void tls_set_strict(int on);
int  tls_get_strict(void);

tls_https_request sends a non-NULL body as application/x-www-form-urlencoded with a Content-Length — this is what backs POST forms in Selene-Browser.

Receiving is streaming, so pages larger than the record buffer load correctly.

Trying it

tls example.com          # handshake against host:443, with detail
tlsstrict on             # then repeat — a failing chain now refuses
posttest                 # live HTTP POST over TLS to httpbin
httpget https://…        # fetch and print
selene                   # the browser, over HTTPS

History

The whole stack was built in sequence between v5.9.48 and v5.9.80, one primitive per release — ClientHello, X25519, the PRF, ECDHE, AES-GCM, the record layer, a live handshake, then the trust model (DER, ECDSA-P256, RSA, SHA-512, ECDSA-P384, chain verification, hostname and dates), then enforcement and the wider trust store. Version-History has the release-by-release account.

See also

External resources

NyxOS v6.4.363 · GPL v2 · GitHub · uselessalter on Discord · nyxos@inbox.lv

NyxOS Wiki

Getting started

Kernel

Storage & network

Graphics & apps

Userspace

HOWTO

Reference

Project

Clone this wiki locally