Skip to content

Z# 0.6.2

Latest

Choose a tag to compare

@ZigoteProject ZigoteProject released this 29 Oct 20:24
· 76 commits to master since this release

The first public release, for microcontrollers: firmware for a real board, the ST Nucleo-F401RE,
written as tasks on the runtime's loop, with interrupt handlers, registers as [<Bits>] records,
code checked not to touch the heap, and its log over RTT (examples/nucleo). Beside it: package
versions and the orphan rule for impls, Concurrent ML's events for tasks, lldb, doc tests,
zs doctor, allocation hints in the editor, and the guide's page for F# programmers.

Archives for Linux, macOS and Windows on x86-64 and arm64, each cross-built from Linux x86-64.
Linux x86-64's suite passes natively, Linux arm64's under QEMU (its compiler builds itself to
the same bytes), Windows x64's under Wine; the macOS builds and arm64 Windows are untested on
a real machine. The flagship runs under QEMU's STM32F405, not yet on the board itself.

Language

  • What an F# programmer writes in the first hour parses: let y = x + 1 in y * 2; a condition
    continued on a line starting with || or && (and |||, &&&, ^^^); a record on the line
    after its constructor (Some then { x = 1 } indented); match a, b with without
    parentheses. handle is a keyword only where a handler starts, so let handle x = … is a
    function. [<CLayout>] and the other attributes apply to a private type.
  • [<Unsafe>] on a function: only unsafe code may call it (Bytes.writer, which writes
    wherever its pointer points, is one).
  • A top-level Buffer of one type (let private seen: Buffer<int> = Buffer.empty, or a record
    holding one) is one Buffer, computed once at startup, so a push to it is seen at every use; it
    was a new Buffer at each use, and the push was lost. A generic one, Buffer.empty itself, is
    still a new Buffer at each use.
  • The orphan rule (K9): an impl written lives in its trait's package or its type's, so two
    packages can't both write one that a program using both would have twice. To rewrite: an
    impl of the library's trait for the library's type, or for a dependency's: wrap the type in a
    one-case union of your own. A hand-written impl used before it's checked (further on, or in a
    later module) is an error saying to write it before its first use; it was a wrong
    "does not implement" or "already implemented".
  • Patterns at the top level: let (width, height) = (800, 600) declares both names. A tuple of
    constants stays constants; anything else is computed once at startup. A local let's pattern
    takes an annotation (let (a, b): int * string = …), and a let whose pattern can fail says
    so: a let's pattern must match every value, and this one misses None.
  • Layout facts: sizeof<T>, alignof<T> and offsetof<T> "field" are int constants for the
    target, a [<CLayout>] type's as C lays it out. A top-level assert on constants is checked
    when compiling (assert (sizeof<Packet> = 16)), naming what it read when it fails; any other
    top-level assert runs at startup. The header zs build --lib writes checks each struct's size
    with _Static_assert.
  • C unions: a [<CLayout>] union has no tag, each case a view of the same bytes. It's built by
    case (the bytes after the case's field zero) and read by case name inside unsafe
    (unsafe d.U32); it can't be matched, compared or printed. It crosses to C by value as C
    passes a union on every target, sits in [<CLayout>] records, and is declared in --lib
    headers; debuggers show it as C's. Hovering a [<CLayout>] type shows its size and alignment.
  • The cycle warning (0021): a type whose objects can form a reference cycle through a ref,
    Buffer, Mutex or Channel (two ways back, or two types reaching each other), which
    counting never frees, is warned where it's declared, with the path and notes at the other
    types' fields, and the fixes: a weak back-pointer, [<Cyclic>] on a type whose cycles the
    program breaks, or [<NoWarn "cycle">].
  • A public constant that uses a private one of its module works from other modules (it was
    'x' is private to module M at the use).
  • A type private record's fields are private too: outside its module, code can't read, set,
    build or match them, even on a value it reached through a public record's field.
  • A task cancelled at a wait no longer leaks a value it took apart with a tuple pattern
    (let! (x, reply, nack) = …) before that wait.

Platforms

  • The flagship, examples/nucleo: firmware for the Nucleo-F401RE as a project (zs build there
    builds it for the board, zs test tests its shell on the computer), three tasks on the
    runtime's loop: the LED's, the button's (Irq.wait) and a serial shell fed by USART2's
    handler through an Spsc and a Signal, its registers [<Bits>] records, its log over RTT.
    ./test.sh runs it on QEMU's STM32F405, typing at its USART2.
  • Tasks on a microcontroller (0016): the same task { }, Task.run, sleep, both, all,
    race, timeout and scope as on a computer, on the embedded runtime's event loop: timers on
    SysTick, the core asleep (wfi) when nothing runs. Irq.wait n is a task that goes on once
    line n fires; Signal.set from a program's own handler wakes the task in Signal.wait.
    Runtime.setClock hz after changing the core's clock. Task moved from Std to Core (the same
    namespace, so nothing changes on a computer); on a chip, reaching Task.spawn,
    Task.blocking or Task.waitFor is a compile error naming the chain. examples/mcu/uart.zs
    receives a line in a task waiting for the UART's interrupt. Task.fromCallback makes a task
    of a callback, for a driver's own waits.
  • A real board, the ST Nucleo-F401RE (--target nucleo-f401, 0027): linked for its STM32F401RE
    (512 KB of flash at 0x08000000, 96 KB of RAM), with its interrupt lines by name
    ([<Interrupt "USART2">]), and output over RTT, a ring in RAM the debug probe reads while the
    chip runs. zs run --target nucleo-f401 flashes it with probe-rs, which shows the output and
    returns the exit status. With no probe attached, nothing waits: output that doesn't fit is
    dropped, and an exit leaves the core asleep (semihosting's breakpoint would fault there).
    Targets are found by name, so a board shares its core's triple: [target.nucleo-f401] and
    [c.nucleo-f401] are its own.
  • A chip's image keeps its symbol table (zig cc dropped it at -Os) and its entry point (it
    was 0, where a debugger's load jumps).
  • 32-bit Arm Linux (--target arm-linux-musleabihf) builds with 32-bit words, as its runtime
    has: it built with 64-bit ones, linking against a runtime that read them as 32.
  • Interrupt handlers (0012): [<Interrupt 38>], [<Interrupt "UARTRX0">] (the board's names,
    runtime/boards/<board>.irq) or [<Interrupt "SysTick">]/"HardFault"/… on a
    unit -> unit function make it the line's handler over the runtime's weak defaults. A handler
    may reach only what's Share (interrupt handler 'tick' uses the top-level value 'count' : ref<int>, …); Atomic is in Core now (masking interrupts on a Cortex-M0), and Spsc<'a> is a
    queue of one producer and one consumer that neither waits nor locks. Irq.enable, disable
    and pend.
  • Cpu (0012): wfi, wfe, sev, dmb, dsb, isb, nop, enableInterrupts,
    disableInterrupts, one instruction each, and Cpu.critical (fun () -> …), its body run with
    interrupts masked (it nests). On a computer the barriers are fences and the rest nothing; masking
    is an error there.
  • [<Bits>] records (0013): a record that is one unsigned integer, its fields ranges of its
    bits (bool, enums, u8 to u64, [<Width n>] and [<Bit n>], the first field
    attributes): { ue = true; te = true } is one constant, a read a shift and a mask,
    { r with f = v } a masked write keeping the gaps; compared as the integer, printed as the
    record. Bits.toU32, ofU32 and the other widths, raw and ofRaw.
  • [<NoAlloc>] and --no-heap (0012): a function that, or anything it calls, may allocate or
    free is an error naming the chain (reply is [<NoAlloc>], but reply -> describe builds a string); interrupt handlers and Cpu.critical's bodies are checked the same way.
    --no-heap checks the whole program, startup values included, and builds a chip's image with
    no heap.
  • A string built with + (an interpolation is one) is one allocation: its pieces are joined at
    once, where each + made a string of its own ($"{name}: {qty}\n" made five, and makes one
    beside qty's text). An empty literal piece is left out, and a piece alone is the string
    itself, so $"{s}" allocates nothing and passes [<NoAlloc>].
  • [<NoAlloc>] passes code that frees nothing: dropping a value matched to a case without
    fields (a static cell), an option's Some s keeping its string, and string of a string (its
    own text). Its errors name an operator as written (Ledger.<+, which showed as Ledger.) and
    a dropped local without the compiler's numbering ('o', not 'o.8').
  • A function that makes a span over a parameter (Array.fold f 0 xs over an array) borrows it.
  • Runtime.ticks on a chip counts a SysTick wrap its handler hasn't run for yet, and never goes
    back (it could, by a wrap, right at one).
  • An [<Export>] function returning unit compiles (the compiler panicked in its C wrapper).
  • C interop on a microcontroller, both ways: structs by value follow AAPCS (they were passed
    with x86-64's rules), hard and soft float; [<Link>] C sources are compiled for the chip; C
    callbacks work there. zs build --lib --target cortex-m4 makes an archive and its header for a
    C firmware, which takes the library's output, stop and ticks by defining zs_write,
    zs_stop and zs_ticks.
  • WebAssembly: [<Link>] C sources are linked in, and --lib -o lib.a makes an archive for a C
    program built for WebAssembly.
  • import c brings C unions in, a member a case named through its union (epoll_data.fd 3,
    unsafe ev.data.fd), and a struct or union without a name in a field as a type named after
    both (Msg_body). A struct with bit fields is left out, saying so (it was imported with every
    bit field a whole field, the wrong layout).
  • import c in a build for a chip or WebAssembly reads the header as the target's compiler does
    (its triple and flags, freestanding), so a vendor's header sees the chip it's written for.
    size_t and uintptr_t are usize, and ptrdiff_t, intptr_t and ssize_t are int, on
    every target (they followed the target's C type: u32 on a chip).
  • A chip's runtime has C's malloc, free, calloc, realloc and abort (weak) over the Z#
    heap, for C, Rust or Zig code linked in; reuses freed blocks over 256 bytes (they were never
    reused); and has what Core needed that it lacked: String.trim, toUpper, toLower,
    replace and float parsing, Ptr.alloc and Ptr.free, handles for C, Runtime.mallocs.
  • zs.toml: target under [package] (what zs build and zs run build for without
    --target), [c.<target>] sections added to [c] for one target (host, cortex-m4, a
    triple), and [target.<name>] with a chip's heap.
  • zs run for a chip runs the image under QEMU on its board, and a WebAssembly module under Node.
  • examples/rust and examples/zig: Rust and Zig code and packages with Z# through the C ABI,
    both ways, on an OS and on a Cortex-M4 under QEMU.
  • macOS and Windows: a module's file is found by its exact name. Their file systems ignore case,
    so a file could be taken for a module of another name: fixed.zs for the library's Fixed,
    or simd/x86.zs for Simd.X86.
  • An exported function that returns a struct through memory declares the struct's own alignment
    for the pointer it's given (it said 8, which lets a Cortex-M fault on a 4-aligned struct).

Security

  • An allocation whose size overflows the word panics out of memory; wrapped, it was a small
    object that writes then ran past: Array.create and Buffer.create of more than the address
    space (2^61 ints made a 16-byte array its fill wrote past), Channel.make's and
    Spsc.create's slots, and on a 32-bit target a string's length (String.replace of 65,536
    as by 65,537 bytes made a 64 KB string and wrote 4 GB into it). String.sub's range check on
    a 32-bit chip no longer overflows either.
  • Loop versioning evaluates a for loop's bounds once. Its range test and the loop each
    evaluated them, so a bound with an effect ran twice, and the copy without bounds checks could
    run over values the test never saw, reading outside the array.
  • A float converted to an integer saturates, NaN to 0, as F32x8.convert's lanes do
    (u8 300.0 is 255): out of range it was LLVM's poison, x86's answer on one machine and Arm's
    on another.
  • A NUL byte in a path, an environment variable's name, a host or a socket's path fails with
    Invalid argument: the OS read the name only up to it, so "secret\0.txt" opened secret, and
    a leading NUL made a Linux abstract socket, which any user can reach. Process.run and
    Process.start refuse an argument, program, directory or environment holding one ("a\0b"
    reached the program as two arguments), and pass an empty argument, alone or last (start
    dropped a last one, both a lone one).
  • UTF-8: String.ofBytes and for c in s (String.decodeAt) take only RFC 3629's sequences, as
    Utf8.validate does: an overlong form, a surrogate or a value past U+10FFFF is U+FFFD.
    C0 AE C0 AF decoded as ./, past a check of the bytes for ../.
  • A string's hash is SipHash-1-3 under a key drawn per process, as Rust's HashMap hashes:
    FNV-1a's was the same in every run, so whoever picked a HashMap's keys (a JSON object's, from
    a socket) could make them all collide and every lookup walk them. A string hashes differently
    in each run; ints still hash to themselves.
  • Numbers as text keep their . whatever locale a C library sets (setlocale: GTK's, an
    ncurses program's), on Linux and macOS: in de_DE's, 1.5 printed 1,5 and Float.parse "1.5"
    failed, so Json.parse read no fractions. String.isFloat and String.toFloat read as
    Float.parse does, without leading spaces or hex.
  • Tools: the build server's directory is private (0700; under a umask of 0, any user could reach
    its socket and run zs commands as you), and two servers no longer share their request files.
    The MCP docs tool and zs test --doc on a library file write their scratch program in your
    cache, not under a fixed name in the shared temporary directory, where another user's link
    there was followed, and a .zs-doctest.zs they put there first could change between its
    writing and its build, which runs it. A build no longer leaves an empty file there for each
    object it compiles.

Library

  • Threading.Alt: Concurrent ML's events for tasks, on one event loop, as F#'s Hopac has them.
    Alt.choose commits exactly one branch and takes the others back, withNack tells the
    branches that lost, guard builds an event at each sync, and Ch and Mailbox are its
    channels. A cancelled sync leaves no waiter behind; deep chains of syncs and maps don't grow the
    stack. The guide's tasks chapter says when to choose events over plain tasks.
  • The thread pool wakes a sleeping worker when work arrives: a worker woke every millisecond to
    look for work it could have missed (tests/parallel/wake.zs).
  • Bytes: readU16BE, readI32LE, writeU64BE and the rest, integers at an offset in either
    byte order. Text.Hex and Text.Base64 (with the URL-safe, unpadded encodeUrl/decodeUrl
    JWTs use), Sha256.hash and Hmac.sha256, in Core, so on chips and WebAssembly too.
    Span.indexOf, startsWith, and on bytes equalsIgnoreCase, trim and parseInt, so an
    HTTP or JSON scanner is ordinary code over spans (track S, items 12 and 13).
  • Byte strings: b"Content-Length: " is a span<u8> of the literal's constant, nothing copied
    at startup. Ptr.readUnaligned and writeUnaligned (at any address: a u64 or a SIMD
    vector in one load) and Ptr.diff (C's end - start).
  • A byte writer in Bytes: append, appendUtf8, appendInt and appendJsonString (scanned
    16 bytes at a time) add to a Bytes.Writer and return it, so a response is a pipeline:
    w |> Bytes.append b"Content-Length: " |> Bytes.appendInt n. A Writer pushes onto a Buffer
    (Bytes.writerOf) or fills fixed C memory such as a send buffer (Bytes.writer p n, unsafe;
    writing past its end panics); Bytes.written is what it wrote, as a span.
  • Bytes.readF64BE, writeF32LE and the rest of the floats, and Float.toBits32/ofBits32
    beside toBits/ofBits.
  • List.collect and List.concat are linear: they copied what they'd gathered for every list,
    quadratic in how many there were. Set.union adds the smaller set to the larger one,
    intersect walks the smaller one, both build their result from its sorted elements in linear
    time, and difference gives back a itself when b takes nothing from it.
  • List.maxBy and minBy; Array.max, min, maxBy, minBy and sumBy.
  • unit has Show, printing () as F# does, so result<unit, int> and the like print.
  • Math: floor, ceil, round (half to even, as F#'s), trunc, exp, log, log2,
    log10, sin, cos, tan, pow and atan2 for f64 and f32, as LLVM's intrinsics (an
    instruction where the target has one, else libm's function); clamp, pi and e.
  • Random (Numerics, a new namespace open everywhere): PCG32, so a seed gives the same numbers
    on every target, a chip's included: create, nextU32, nextInt (unbiased), nextFloat,
    nextBytes and shuffle; Random.shared (), seeded from the operating system (Std).
  • A chip's and WebAssembly's runtime has pow, atan2 and roundeven (fdlibm's, weak), which
    compiler-rt lacks; tests/libm.sh checks them against the host's libm.
  • List.init calls its function in index order, as F#'s (it went from the last index down).
  • F#'s other collection functions (S2): groupBy, countBy, chunkBySize, windowed,
    pairwise, scan and foldBack on lists and arrays, Array.tryFindIndex and
    Array.binarySearch, Map.change, Set.isSubset and isSuperset; and Queue, Stack and
    PriorityQueue (lowest priority first, a leftist heap), values like lists.
  • Text.Utf8 (S4): validate (the offset of the first malformed byte), decode (malformed
    bytes as U+FFFD), tryDecode and encodeChar, Z# over spans, so on a chip too.
  • A function that takes an array from a record parameter into a let mutable and writes it in
    place owns the record, so a unique array is changed in place (the record was borrowed, the
    caller's reference stayed, and every call copied the array: quadratic once inside a while).
  • The runtime's threads (the pool's workers, the event loop, helpers) have 8 MB stacks, as Linux
    gives and as the main thread has: on macOS they had 512 KB, which deep recursion ran out of.
  • Map.add and Map.remove (so Set.add and Set.remove) build the new node in the cell of the
    one they took apart when the map is unique and the node is still balanced, as it almost always
    is: adding to a map nothing else holds allocates one node, not one for each level. A million
    adds and lookups take 136 ms where they took 289.
  • The runtime's string search (String.indexOf, and through it contains, split, replace and
    lastIndexOf) goes from one place the first byte is to the next with memchr, comparing the
    rest only there: it called memcmp at every byte. logstat reads 2 million lines on one core in
    1.21 s where it took 1.78.
  • An integer's text is made without snprintf, and println writes a line under 256 bytes with
    its newline in one write, so another thread's output can't land between them: printing 5
    million lines of two numbers takes 0.46 s where it took 0.79.

Packages

  • Versions (0018 item 2): json = { git = "…", version = "1.2" } takes the repository's releases
    (tags v1.2.3, 1.2.0 or later below 2.0.0) by minimal version selection, as Go's: each
    package gets the highest version asked for it, so a diamond builds the same on every machine
    with no solver. zs.lock records the release; zs update raises the project's own versions
    to the newest. A major of 2 or more is another package with its own key (json2), so both build
    in one program.

Tools and docs

  • Allocation hints in the editor (0011, 0026): with allocationHints: true in the language
    server's initialization options (zsharp.allocationHints in VS Code,
    lsp.zs.initialization_options in Zed), each line shows what it does to the heap, from the
    compiler's passes after checking: allocates ×2, reuses t, copies on write, the objects
    and their sizes in the tooltip (builds a cell of Tree (Node, 32 B)). A line counts the one
    path through it that allocates most, so a match's arms (and a decision tree's copies of one)
    aren't added up. Hover on a function adds what its own code allocates and whether
    [<NoAlloc>] would pass, or the chain to what doesn't. A module of a project (a zs.toml
    above it naming another main file) has the hints the project's program gives its code.
  • The compiler is faster, found with its own allocation hints and a profiler: building itself from
    an empty cache takes 16.8 s where it took 23.8 s, its own passes 3.7 s where they took 8.1 s,
    with 217 million allocations instead of 626 million and a peak of 744 MB instead of 859. A node's children are listed without copying the node, the names
    an expression binds and uses are gathered in one walk, a type a substitution leaves as it is
    isn't copied, code generation keeps each type's LLVM type and each union's cases, borrowed
    parameters are worked out again only for the functions whose callees changed, and reference
    counting no longer makes a closure of its whole state at every node. A change to the inliner
    or closure conversion rebuilds in 0.65 s where it took 4.7 s: their functions that take
    functions say those are pure, so a caller with effects no longer compiles its own copy of them.
  • A program with threads has less code: a type's share and move walks are one function (the mode
    an argument), and a closure's walk and drop are one function for each list of captured types,
    not one for each closure. The compiler's binary is 9% smaller, its IR 8%.
  • A program with threads allocates faster on macOS: what the allocator keeps for a thread (its
    counts, the pool's free lists and chunk) is one thread-local, found once per allocation; each
    was its own, and macOS finds a thread-local by a call (the compiler spent 4% of its time there).
  • Loop versioning proves more of a for loop once, before it runs: its writes (a.[i] <- v) as
    well as its reads, indexes without the loop's variable (x.[i] in a loop over j) or any whose
    range follows from the loop's (r * w + c, (j + 3) % 5), and its + - * over the variable and
    what the loop doesn't change, which then run without overflow checks. Each such expression's
    range is computed before the loop; if computing it overflows, the checked copy runs
    (tests/versioned_loops.zs). nbody takes 0.80 s where it took 0.91 (C's 0.76), spectralnorm
    0.42 s where it took 0.48 (C's 0.68).
  • A build whose checked modules came from the cache, made by a build that named a file another
    way (tests/x.zs, then its full path), gives a pass's error its file and line: it printed
    : error: with neither.
  • The guide's "Z# for F# programmers" page (docs/guide/17-from-fsharp.md), and posts on the
    strongest results (docs/posts: Perceus on a Cortex-M, atomic counts only where threads meet,
    SIMD next to C).
  • zs doctor: what builds need, found here or not, with how to get what's missing (clang,
    zig, libclang, git, QEMU, probe-rs, Node, lldb or gdb). zs alone lists the commands.
  • Doc tests: a fenced zsharp block in a /// comment is an example that zs test --doc builds
    and runs, its // => text lines checked against what it prints, and zs test --doc --update
    rewrites them. The library's first examples (List.groupBy, chunkBySize, maxBy,
    Array.maxBy, Map.change, Utf8.validate, Math.clamp) run in ./test.sh.
  • lldb, the debugger on macOS, works on zs build -g programs as gdb does: breakpoints on Z#
    functions and lines, stepping, backtraces with Z# files and lines, and strings, records and
    unions as values (tests/debug/lldb.sh). It names the types as C's: a 64-bit int is a long.
  • A VS Code extension (editors/vscode) and a Rider plugin (editors/rider, any JetBrains IDE
    2025.2 or later): highlighting, the language server and, in VS Code, the MCP server for the
    agent, as the Zed extension gives. Both share one TextMate grammar.