Skip to content

v0.2.0

Choose a tag to compare

@tamnd tamnd released this 02 Sep 18:44
· 79 commits to main since this release
v0.2.0
337feb7

M1 is complete. core.errors is the first package in this library with code in it, and the first at full parity. It is the mechanism every fallible function written here from now on is written against, which is why it comes before anything that could use it.

The problem it solves is that Mojo's Error carries a string and nothing else, while half of Go's standard library returns a struct with a path, a syscall number or a byte count in it. The answer is a record in thread local storage, matched to the error by the message it was raised with, and capture for when the error has to outlive the raise. All three of the ways that can fail silently now have a test that was watched failing.

Two language probes and the design facts they pin, ahead of core.errors in M1.

Mojo has no global mutable state. A module level var is refused outright and the message tells you to move it into a function or make it a comptime constant, so there is nowhere in the language to put a package level counter, cache, registry or default. Go's standard library has one in a dozen places. docs/design.md now records that, and that every one of those becomes a value the caller owns and passes.

The one exception is the thread local error record in section 4, which has to outlive the call that wrote it and cannot be passed. It gets its slot from a small C object that core.errors links, described below. The cost is stated in the design rather than discovered later, and the alternative it was weighed against was threading an explicit context through every fallible call in the library, which puts the error mechanism in the signature of every function in it.

tools/probe/probes/thread_local.mojo pins that pthread's per thread storage really is per thread. Four threads each claim a slot, hand its address to pthread_setspecific, wait at a barrier until all four have written, then read the pointer back and write through it, while the main thread holds a different value in the same key across all of it. Without the barrier a shared slot would still look correct, because each thread would set and read before the next arrived. The failure this rules out is one thread reading another thread's error fields, which is a wrong answer rather than a crash.

Both probes were checked against the two ways they can fail: compiling when they should not, and still being refused for a different reason than the one recorded.

The first library code in the tree: the thread local error record, which is the mechanism every fallible function in this library will be written against.

Report(message).with_field("path", name).error() writes a record into this thread's slot and hands back the Error to raise. field(e, "path"), code(e) and partial(e) read it back at the catch site, so the fields Go would have put in a struct survive a raise that carries only a string, and so does the n from Go's (n, err) that a raise would otherwise drop.

A record is matched to an error by the message it was raised with and by nothing else. That is what makes the two silent failures safe. An error raised by std, or by anything that has never heard of this mechanism, finds a record whose message is not its own and is correctly reported as carrying nothing. An error held past the next raise on the same thread finds the newer record, sees a different message, and reports nothing rather than the newer error's fields. Both of those would otherwise run, print something plausible and be wrong, so both have a test, and both tests were watched failing with the identity check removed. Nothing is appended to the message to make this work: a message with a token in it is a message that cannot be printed, and matching on text that somebody also reads would make every wording change a breaking change to a lookup.

The slot is C, in core/errors/shim/slot.c, and that directory's README says why at length. It is a pthread key rather than a _Thread_local pointer because a key has a destructor, so a thread that exits still holding a record frees it. The destructor is a Mojo function handed to the shim rather than exported for it to find, because a function only C calls is a function a dead code pass removes, and the exported version linked on Linux and not on macOS. core.errors therefore declares unsafe = true, taking the linter's count from fifteen packages to sixteen, and running the tests now needs a C compiler on the host.

core.errors is tier zero, so every binary built on this library links that object. The cost is stated in docs/design.md section 4 rather than left to be found in a link line, and the alternative it was weighed against, threading an explicit context through every fallible call, is recorded there too.

Two more language facts, each with a probe. A struct valued field cannot be moved out of an owned self, which is why errors.Report is both the builder and the record rather than the two structs that would read better. And the pthread key destructor really does call back into Mojo on a worker thread's exit, which was proved before the record depended on it.

tools/lib/native.py is now the one place that goes looking for a C compiler, shared by pixi run baseline and the test runner, and it explains why neither takes one from the lockfile.

The rest of Go's errors package: wrap, matches, join, unwrap, causes and new. core.errors is the first package in this library at full parity, five symbols present and two waived.

The record is a tree now, because wrapping and joining both make one. It is an arena of links with integer indices, which is the technique the design already committed to for every recursive type here, used for the first time. A chain survives any number of levels with each level keeping its own fields, and wrap copies the cause out of the thread's slot before the new record replaces it, which is the only order that works when there is one slot per thread. field and code answer about the error you hand them and not about what it wraps, because two links in a chain can each carry a path and the wrong one is worse than nothing. matches is the one that walks, and it walks every cause of a joined error rather than the first, which is the part of Go's contract that is easy to get wrong and which now fails a test when it is.

A sentinel is a code, and nobody picks the number. Go's errors.Is(err, io.EOF) works because io.EOF is a value you can hold, and there is none here, so a sentinel is an integer on the record. Integers chosen by hand collide, and a collision makes matches(e, io.EOF) quietly true for an os error, so core/errors/codes.toml lists every sentinel in the library and tools/gen/codes.py numbers them. That makes a collision impossible rather than unlikely, at the cost that the numbers move when a line is inserted, so a code is meaningless outside the process that produced it and the type says so. Code is a struct rather than an Int, which is what stops with_code(300) compiling where with_count(300) was meant.

errors.As and errors.AsType are waived. Both are reflection over a type hierarchy and this library has neither, so there is no honest partial version; the replacement is the field lookup, the sentinel comparison, and a per package helper such as os.PathError.of(e). errors.Is is renamed to matches, because is is a Mojo keyword.

join is weaker than Go's and the deviations page says so rather than leaving it to be found. Go holds error values and every field with them. At most one of the errors passed here still owns the thread's record, so the others contribute their message and their place in the tree and nothing more. capture is what will close it.

Sixteen new tests, and the suite was checked against three ways of getting this wrong: a matches that does not walk the chain, a join that reports only its first cause, and a wrap that refers to the record instead of copying it. Each one fails the tests that name it and nothing else.

errors.capture(e) and ErrorValue, which is what makes an error a value again. It copies the error's whole subtree out of the thread and owns every message, field and cause it can reach, so a failure can go in a list, sit in a struct field, or be read by a thread that never saw it raised. ErrorValue.error() is the way back: it installs the record on whatever thread calls it, so a task's error can be re-raised by the thread that collected it and every function in this package then works on it as though it had just happened.

The cross thread case is the one that decides whether any of this is real, and it now has a test. The reading thread first raises an error of its own, so its slot holds something else entirely, and then reads the captured value. Routing ErrorValue.field through the thread's slot instead of the value makes that test fail with left: /the wrong one, which is the exact wrong answer it exists to rule out.

join over captured errors keeps every field of every cause. Over live errors it cannot, because a record is written at raise time and the next raise replaces it, so all but one arrive as a message. Both halves have a test, so the code and the deviations page cannot drift apart.

With that, the three ways design.md section 4 can fail silently are all covered: a foreign error mistaken for ours, an error held past the next raise, and a capture read on a thread whose own storage holds something else.

What's Changed

  • probe: pin no global mutable state and per thread storage by @tamnd in #102
  • core.errors: the thread local error record by @tamnd in #103
  • core.errors: wrap, matches, join and the code registry by @tamnd in #104
  • core.errors: capture and ErrorValue, including across a thread by @tamnd in #105

Full Changelog: v0.1.0...v0.2.0