IEEE-754 ⟷ bytes codec in pure Standard ML: decode and encode f16 (half),
bf16 (bfloat16), f32 (single) and f64 (double) to/from a
Word8Vector.vector, in little- or big-endian byte order. For reading and
writing model weights and binary formats — safetensors, GGUF/npy, the CBOR /
MessagePack float types, WAVE float samples, and so on.
Decoding operates on the sign/exponent/mantissa fields with integer arithmetic
plus Math.pow, so it is exact and byte-identical under MLton and Poly/ML.
Subnormals, signed zero, ±∞ and NaN are all handled. No FFI, no Pack*, no
dependencies.
smlpkg add github.com/sjqtentacles/sml-float
smlpkg sync
No dependencies; Basis library only.
open Float
(* decode: (bytes, offset) -> real *)
val one = decodeF32Le (bytes, 0) (* little-endian single at byte 0 *)
val x = decodeF16Be (bytes, 6) (* big-endian half at byte 6 *)
val w = decodeBf16Le (bytes, 0) (* bfloat16 (common in model weights) *)
(* encode: real -> fresh Word8Vector *)
val b = encodeF32Le 3.14159 (* 4 bytes, little-endian *)Round-trips hold: decodeF32Le (encodeF32Le x, 0) reproduces x to f32
precision, and encodeF32Be 1.0 = [0x3F, 0x80, 0x00, 0x00].
(* decode: (bytes, offset) -> real -- little-endian (*Le) and big-endian (*Be) *)
val decodeF16Le : Word8Vector.vector * int -> real
val decodeBf16Le : Word8Vector.vector * int -> real
val decodeF32Le : Word8Vector.vector * int -> real
val decodeF64Le : Word8Vector.vector * int -> real
val decodeF16Be : Word8Vector.vector * int -> real (* ... and Bf16/F32/F64 Be *)
(* encode: real -> fresh Word8Vector *)
val encodeF32Le : real -> Word8Vector.vector
val encodeF64Le : real -> Word8Vector.vector
val encodeF16Le : real -> Word8Vector.vector
val encodeBf16Le : real -> Word8Vector.vector (* ... and *Be variants *)decode* raises Subscript if the bytes are not present at the offset.
f16/bf16 encoders saturate to ±∞ on overflow.
Pure + - * / mod div plus Math.pow (which both MLton and Poly/ML defer to
the same libm); the bit pattern is assembled as an IntInf split into bytes
with div/mod (never IntInf.~>>, whose large-negative sign handling differs
between the compilers). Reals are compared in the test suite with an explicit
tolerance — never Real.toString. So make test and make test-poly produce
byte-identical output.
make example builds and runs examples/demo.sml, which
round-trips pi through every width/byte-order combination and encodes/decodes
+inf, -inf, and NaN (output is byte-identical under MLton and Poly/ML):
Float codec demo
encodeF32Le pi = db 0f 49 40
decodeF32Le round-trip = 3.141593
encodeF64Le/decode = 11 2d 44 54 fb 21 09 40 -> 3.141592653590
f16 round-trip of 1.5 = 00 3e -> 1.5000
bf16 round-trip of pi = 49 40 -> 3.1406
big-endian f32 of pi = 40 49 0f db -> 3.141593
+inf round-trip via f32 = +inf
-inf round-trip via f32 = -inf
NaN round-trip via f32 = NaN
make test # MLton
make test-poly # Poly/ML
make all-tests # both
45 assertions, green on MLton and Poly/ML with byte-identical output, against
textbook IEEE-754 bit patterns (1.0 = 0x3F800000, π = 0x40490FDB, subnormals,
±∞, NaN) plus round-trips.
MIT