Skip to content

v0.2.0 — Core position & span types

Pre-release
Pre-release

Choose a tag to compare

@jamesgober jamesgober released this 19 Jun 04:03

span-lang v0.2.0 — Core position & span types

The correctness foundation. v0.2.0 is the hard part of the roadmap, built
first rather than deferred: the byte position and span value types, and the
UTF-8-correct line/column resolver with its O(log lines) line index that every
diagnostic in the ecosystem will lean on. Four public types land — BytePos,
Span, LineCol, and LineIndex — each with rustdoc and a runnable example,
and the project invariants come under property tests cross-checked against a
naive reference resolver. No prior public API to break; this is the first release
with a surface.

What is span-lang?

The source-position substrate for language tooling. It defines the small,
copyable coordinate types that a lexer, parser, and diagnostic renderer all
share: a byte position, a byte-offset span, a resolved line/column, and the index
that maps between them — correctly over UTF-8, across \n and \r\n. It owns
positions and nothing else: it does not load source and does not render
diagnostics, which is what lets every layer above depend on it without inheriting
I/O or formatting.

What's new in 0.2.0

BytePos — the byte offset

A Copy newtype over a u32, so it is cheap to move and packs two-to-a-span. The
32-bit width bounds a single source to 4 GiB, the addressing envelope language
front-ends use. It exposes new, to_u32, to_usize, u32 conversions both
ways, and a Display that prints the bare offset.

use span_lang::BytePos;

let at = BytePos::new(42);
let src = b"...";
let _ = at.to_usize(); // ready to index a byte slice

Span — a packed half-open byte range

Two packed offsets, eight bytes, Copy. The range is half-open (start
included, end excluded), so the length is exactly end - start and adjacent
spans do not overlap. The start <= end invariant is enforced at construction:
Span::new orders its two arguments, so a span can never be built inverted and is
never constructed by a panicking path.

use span_lang::Span;

let a = Span::new(4, 10);
let b = Span::new(8, 14);
assert_eq!(a.merge(b), Span::new(4, 14)); // smallest covering range

// Folding a node's children into the node's own span.
let children = [Span::new(10, 12), Span::new(4, 6), Span::new(20, 25)];
let parent = children.iter().copied().reduce(Span::merge).unwrap();
assert_eq!(parent, Span::new(4, 25));

merge is commutative and associative, so the order spans are combined in never
changes the result — both properties are property-tested.

LineCol — a resolved coordinate

A 1-based line and column. The column counts Unicode scalar values (chars),
not bytes, not UTF-16 units, and not grapheme clusters, so it never lands inside a
multi-byte sequence. It formats as line:col, the convention editors and
compilers display.

LineIndex — byte ↔ line/column in O(log lines)

Built once per source with a single linear scan that records where each line
begins. After that, line_col is a binary search over those line starts followed
by a character count within the one located line; offset is the checked inverse.
Neither lookup allocates.

use span_lang::{BytePos, LineCol, LineIndex};

let index = LineIndex::new("let x = 1;\nlet y = 2;\n");
assert_eq!(index.line_col(BytePos::new(11)), LineCol::new(2, 1));
assert_eq!(index.offset(LineCol::new(2, 1)), Some(BytePos::new(11)));

line_col is total: an offset past the end clamps to the end, and an offset
inside a multi-byte character rounds down to the character start, so resolution
never panics and never returns a coordinate that is not a real position. offset
returns None for a coordinate that does not exist (line or column 0, a line
past the last, a column past the end of its line). A \r\n is one line break, the
empty string is one line, and a source with no trailing newline keeps its final
unterminated line.

Correctness — property tests against a naive reference

tests/properties.rs holds the section-4 invariants over a wide input space, with
sources drawn from an alphabet of ASCII, tabs, bare \r, \r\n, \n, and two-,
three-, and four-byte characters:

  • Span::new always yields start <= end, ordering its arguments.
  • merge is commutative, associative, and returns exactly min(starts)..max(ends).
  • Forward resolution agrees with a full naive character scan on every byte offset.
  • Byte → (line, col) → byte round-trips for every valid position.

Performance — benchmarks recorded

benches/bench.rs covers the hot paths with criterion. Latest local means
(Windows x86_64, Rust stable):

  • Span::merge — ~0.6 ns/op
  • LineIndex::offset — ~2.5 ns/op
  • LineIndex::line_col — ~8.7 ns/op
  • LineIndex::new — ~8.4 µs to index 1 000 lines

The forward lookup uses an ASCII fast path (a length read instead of a decode
loop) that the property tests prove agrees with the general path on every input.

Breaking changes

None. v0.1.0 had no public API; this is the first surface. It is additive
across the remaining 0.x series and frozen at 1.0.

Verification

Run on Windows x86_64, Rust stable 1.95.x and MSRV 1.85; identical commands pass
on Linux (WSL2 Ubuntu) and via the configured CI matrix:

cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo clippy --no-default-features --all-targets -- -D warnings
cargo test
cargo test --all-features
cargo build --no-default-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo +1.85 build --all-features
cargo audit
cargo deny check

All green. Counts at this tag:

  • Default features: 21 unit + 21 doctests.
  • --all-features: 21 unit + 6 property tests + 21 doctests.

What's next

  • 0.3.0 — Source coordinate mapping & line index. Prove the O(log lines)
    lookup by benchmark scaling rather than by claim, and harden the empty-source,
    no-trailing-newline, and CRLF edges with their own coverage.

Installation

[dependencies]
span-lang = "0.2"

MSRV: Rust 1.85.

Documentation


Full diff: v0.1.0...v0.2.0.
Changelog: CHANGELOG.md.