Skip to content

Releases

v1.0.0

Choose a tag to compare

@github-actions github-actions released this 26 Oct 03:03

The first stable release. Every package's public API and the platforms in the README's
Platform support table are now a commitment under Semantic
Versioning: breaking or dropping either takes a new major version. Every binding reads Apple
Numbers beside XLSX and ODS; a live stream hands over each header and batch as it arrives,
in Rust, Python and Swift; a plan can no longer ask for unbounded memory, with one ordinal
cap, 2^20, in every binding; the Rust API is fixed at what a 1.0 can promise
(#[non_exhaustive], a kernel surface of only what a binding calls); what was never used
goes (Java's GraalWasm backend, Python's BACKEND); and every shipped file verifies from
the release tag alone.

Added

  • Rust — Batch::values::<T> is public. It lends any typed column whole — decimals,
    UUIDs, timestamps, dates, date-times and durations as well as the primitives — where only
    the primitives had accessors (batch.i64(col) and its siblings, which it now underlies).
    The slice borrows the reader as theirs do, and a cell that did not cast holds the door's
    zero beside its verdict, as theirs does. (Rust)
  • C#, Java and Swift lend more columns whole. C#'s Batch.Values<T> lends a decimal
    column as HyperCast's RawDecimal, a timestamp, Unix or Excel-serial column as
    RawTimestamp, a date column as RawDate, a date-time column as RawCivil and a
    duration column as RawDuration, zero-copy, and a time column as ulong nanoseconds; a
    UUID column is still read with Get<Guid>, since the core's RFC 9562 bytes are not a
    Guid's layout. Java's values(column) returns the column segment for every door but
    text, read with HyperCast's NativeValues readers at their *_BYTES stride. Swift's
    values(_:as: UInt64.self) lends a time column's nanoseconds; HyperCast's Swift package
    has no value types with the core's layout, so the other doors stay a cell at a time.
    (C#, Java, Swift)
  • Apple Numbers. Every binding reads a .numbers document — Apple's IWA packaging:
    Snappy-compressed chunks of Protocol Buffers objects — the way it reads XLSX and ODS,
    told by what the container holds, never by its name. Each table is a sheet, listed
    under its sheet's name (<sheet> - <table> when a sheet holds several, as Numbers'
    own Excel export names them) and hidden when its sheet is; cells are read from the
    v5 storage Numbers has written since 2018 — decimals and currencies as numbers, dates
    as wall clocks or, under a format that shows no date (a custom format's pattern looked
    up in the document's list), times of day, durations as spans, booleans, text and rich
    text with the runs joined, error cells as #VALUE!
    (the file does not record which error), the cells a merge covers as empty — and a
    row a tile leaves out is an empty row. The core's new kernel/workbook/{iwa,snappy, proto,numbers}.rs stay inside the no-panic, no-allocation proofs; the window holds
    only the chunks of the object in hand, and a deflated part is inflated into its far
    end. Format gains Numbers (3), Opened.strings_bytes is what the
    data-list parts decode to, and a new failure kind iwa (26, part 10) names
    a Numbers part that is not what the packaging says — found 1 a broken chunk, 2 a
    malformed message, 3 an object a reference names that is not there (expected its
    identifier), 4 cell storage of a version the core does not read (expected the
    version). The not_a_workbook message now says "not an XLSX, ODS or Numbers
    workbook". The corpus gains numbers-data.numbers, written by Numbers 26.3.1 on
    macOS, its Excel export numbers-data.xlsx, and two generated Numbers packages in the
    layouts Numbers never writes small (chunks of a few bytes, seven-row tiles, wide
    offsets, a segmented string list, rows left out of their tile); workbook.json grows
    from 198 to 246 cases, every one of them new. (Rust, C#, Java, Go, Python, Ruby, PHP, Swift)
  • A core freshness check for the suites that link committed binaries. Go's, Swift's
    and PHP's suites refuse to run against go/staticlib, the Swift artifact bundle or
    php/src/native when rust/src, rust/Cargo.toml or rust/Cargo.lock differ from the
    commit those binaries were staged from, and name the local-core command to run instead
    (Go and PHP stop before any test; Swift fails CoreFreshnessTests). CI, the local core,
    shallow clones and a checkout with fresh binaries written into the tree skip it.
    .github/scripts/local-core.sh finds a Forge checkout at ../HyperForge as well as
    ../.github. (repository only)

Changed

  • The release flow, before its first run here. The wheels are uploaded by a step of
    release.yml's own pypi-publish job, so their PEP 740 publish attestations name
    release.yml, as 0.9.0's do (the Forge's composite action nested the upload, where it
    cannot find its own container image; HyperUuid's and HyperCast's 1.0.0 wheels went up
    from another workflow for it); staging starts by itself once the release-mode CI on the
    preparation commit is green, and a second request for a commit already staged is skipped
    with a notice; the Forge commit's step output is revision, which keeps CodeQL's
    actions/cache-poisoning/poisonable-step quiet. (repository only)
  • Built on HyperCast 1.0.0. Every binding depends on HyperCast's first stable release,
    within its major: the crate 1.0.0, NuGet 1.0.0, Maven 1.0.0, Go v1.0.0, Python
    >=1.0.0,<2, Ruby ~> 1.0, PHP ^1.0.0 and Swift from: "1.0.0", where it was 0.9.0
    (Swift: a commit pin, which no tagged package may carry). (all packages)
  • Swift — on HyperCast 1.0.0, whose doors do not throw. HyperCast's Swift doors and
    Cast.nativeVersion() dropped throws (bad data was always the verdict's Fault), so a
    call needs no try; one left in place compiles with a warning. (Swift)
  • Breaking — Rust: the kernel's public surface is what a binding calls, and no more.
    kernel::abi, kernel::door, kernel::exports, kernel::delimited::fill, and
    kernel::workbook's state, memory, format constants and book and rows modules stay
    public; the delimited engine, scanner and unescape, the inflate, and the workbook's zip,
    XML, styles, number, part, sort, cell, Snappy, protobuf, IWA and Numbers modules are
    private to the crate, so 1.0.0 does not freeze them. The tests that drove
    them directly are compiled into the crate from tests/ and every integration target is
    listed in the manifest. (Rust)
  • Breaking — Rust: Column, Dialect, SheetOptions, Door, Error, Format and
    FailureKind are #[non_exhaustive].
    A setting, a door, a format the core learns to
    read or a failure it can report is an addition without a new major version once 1.0.0 is
    out. The three configuration structs are built with their constructors, constants and
    builder methods rather than struct literals; a match on an enum outside the crate names a
    wildcard arm, and a failure code this binding does not know reads as Container, as it
    already did. (Rust)
  • C# — HyperTabular.targets no longer lists an EmccExportedFunction per export. The
    NativeFileReference is the whole Blazor WebAssembly wiring: the SDK's generated pinvoke
    table calls each native function by symbol, so the link resolves them against the archive
    and LibraryImport("*") reaches them through that table. The smoke test reaches all
    fourteen exports with the list gone (on SDK 11.0.100-rc.2) and fails at the link with the
    archive unlinked, so the list was dead weight. (C#)
  • The release is two acts at one tag, and every shipped file verifies from the tag.
    Pushing vX.Y.Z packs every registry input, attests each package and the release as a
    whole, preserves the bytes and plans every upload without publishing; a dispatch of
    release.yml with prepared_run_id publishes from the preserved bytes through what was
    the recovery path. The release attestation is one statement signed at the tag whose
    subjects are every registry package and every native library committed for Go, Swift and
    PHP, and whose predicate is the release inventory, so
    gh attestation verify <file> --repo SkunkWerkx/HyperTabular --source-ref refs/tags/vX.Y.Z --predicate-type https://github.com/SkunkWerkx/.github/predicates/release/v1 answers for
    any of them, including the wheels and the committed natives that build provenance alone
    tied to the preparation commit on master (SkunkWerkx/HyperUuid#35). The GitHub Release
    notes come from this file's section for the version plus the verification commands, and a
    tag whose CHANGELOG lacks the section is refused before anything publishes. The job bodies
    of release.yml, prepare-release.yml, stage-native-binaries.yml and every static check
    in ci.yml moved to the Forge (SkunkWerkx/.github, called at master, composite actions
    where the repository's own identity signs or publishes), with this repository described to
    it by .github/hyper.json and .github/scripts/abi_probe.c; CI gained check-semver
    against the last crates.io release, the no_std consumer build and the shared library's
    export check on every RID. Dependabot bumps every ecosystem in one monthly pull request,
    every third-party action is pinned to a commit, and CodeQL runs weekly. CI also runs the
    Rust suite under wasmtime on wasm32-wasip1 (test-wasi) and SwiftLint's missing_docs
    over the Swift package's public API (swift/.swiftlint.yml), on every pull request.
    (repository only)

Removed

  • Java — the GraalWasm backend. The jar no longer carries the core a second time as a
    wasm32-wasip1 module, and Tabular.backend(), the hypertabular.backend property and the
    fall-back to the module on a platform without a native build are gone: the binding is FFM
    downcalls into the library it bundles. The module served a pla...
Read more

v0.9.0 - HyperCast 0.9.0's culture sweep in every column, free-threaded Python, and Blazor fixes

Choose a tag to compare

@buvinghausen buvinghausen released this 21 Oct 05:44
v0.9.0

HyperTabular 0.9.0 is built on HyperCast 0.9.0, its culture-fidelity sweep: everything a culture declares about a number now reaches every plan column, because the core casts every cell through HyperCast's doors. Also free-threaded Python, and the Blazor WebAssembly fixes from the same consumer feedback HyperCast answered, in the same shape. Nothing a binding's caller writes changes; one ABI layout does. The full list, with a bullet per binding, is in CHANGELOG.md.

Install

Language Package
Rust cargo add hypertabular
C# (.NET 11) dotnet add package HyperTabular
Java (JDK 25) implementation("io.github.skunkwerkx:hypertabular:0.9.0")
Go go get github.com/SkunkWerkx/HyperTabular/go@v0.9.0
Swift .package(url: "https://github.com/SkunkWerkx/HyperTabular", from: "0.9.0")
Python (3.11+) pip install hypertabular
Ruby (3.3+) gem install hypertabular
PHP (8.2+) composer require skunkwerkx/hypertabular

Highlights

  • A plan column's format carries everything HyperCast 0.9.0's NumFormat declares. The numeric doors of a plan column read the currency amount's own separators, further spellings of the minus and plus signs and the percent symbol, native digits beside ASCII, separators of one or two characters, and the TRAILING_SIGN lenience. Each binding's Column takes HyperCast's own NumFormat, which now takes the whole declaration: NumFormat.From(CultureInfo) in C#, from(Locale) in Java, and the rest. One rule throughout, HyperCast's: the ASCII or English default is always read and a declared symbol is read beside it, so no cell that parsed in 0.8.0 reads differently.
  • Python: free-threaded CPython, with the GIL off. Each platform gets three wheels where it had one: cp311-abi3 for CPython 3.11+ with the GIL as before, cp314-cp314t for free-threaded 3.14, and cp315-abi3.abi3t, PEP 803's stable ABI for 3.15+ with or without the GIL. The module declares it needs no GIL, so a free-threaded interpreter keeps it off after import hypertabular, and readers on different threads run in parallel. A Batch and a Workbook can be shared by any number of threads. A reader or a sheet is read by one thread at a time; a second thread calling into one meanwhile gets RuntimeError, never a race on its buffers. PyPI gets 25 wheels per release where it got 9.
  • C#: Blazor WebAssembly. A publish without the wasm-tools workload used to skip linking the native core and ship a runtime with no hypertabular_* symbols, with nothing louder than the SDK's own warning. The package's targets now fail the build with HYPERTABULAR002; an app that keeps a managed fallback sets HyperTabularRequireWasmWorkload to false. The browser-wasm archive is at runtimes/browser-wasm/nativeassets/net11.0/, the path the forge stages, and the targets live in build/net11.0/ with buildTransitive/ importing them, so a project that reaches the package through a class library gets the same wiring.
  • The format crosses the C ABI as HyperCast 0.9.0 lays it out. ColumnSpec is 104 bytes where it was 44: ordinal, door and parameter, then RawNumFormat, a size word followed by the 92-byte NumFormat in place. All zeros is still the invariant notation. Every binding restates the layout where it builds a spec and writes the format through HyperCast's own bridge (ToRaw, Raw, writeFormat, packed, rawFormat), so no bridge lays a byte of it out itself. The corpus replays are byte-identical in all eight languages.
  • Every README example was compiled and run against this release's build, the root README's C# and Rust ones included.

Upgrading from 0.8

  • HyperCast 0.9.0 is the floor in every package: hypercast = "0.9.0" for the crate, hypercast>=0.9.0,<0.10 on PyPI, HyperCast 0.9.0 on NuGet, hypercast 0.9.0 on Maven Central, ~> 0.9.0 for the gem, ^0.9.0 on Packagist, from: "0.9.0" for the Swift package and v0.9.0 for the Go module. The types this crate re-exports from HyperCast (Decimal, NumFormat, Fault, the temporal types) are HyperCast 0.9's.
  • Rust: NumFormat's separators are Symbol<8> where they were char, so build a format with NumFormat::new and its with_* methods rather than a struct literal. Symbol and NativeDigits are re-exported beside CurrencySymbol. The crate's python-values dependency is built on PyO3 0.30.
  • Direct C ABI callers: ColumnSpec is 104 bytes. Rebuild against the 0.9.0 header; a 44-byte spec is a contract violation, not a cell fault.
  • Python from source: a build from a checkout needs maturin 1.15 or newer. Wheels need nothing.
  • C#: nothing to change. A Blazor WebAssembly publish without wasm-tools now stops with HYPERTABULAR002 instead of shipping a runtime that cannot read.
  • Java, Go, Swift, Ruby, PHP: nothing to change beyond the HyperCast floor.

Verifying

Every artifact is built in CI and attested, the browser-wasm archive inside the NuGet package included: the one gap 0.8.0's notes reported is closed, and the NuGet pack now refuses any native without provenance.

gh attestation verify <file> --repo SkunkWerkx/HyperTabular

v0.8.0 - header first, streams, rows, UTF-16 text, async, and Android

Choose a tag to compare

@buvinghausen buvinghausen released this 09 Oct 07:03

HyperTabular 0.8.0 answers the first downstream feedback (#8–#12, filed against the C# binding) in every binding the language allows, and adds Android for C#, Go and Swift. It is built on HyperCast 0.8.0, the first HyperCast with Android. The native ABI is unchanged: the core's calls were always two-phase. The full list is in CHANGELOG.md.

Install

Language Package
Rust cargo add hypertabular
C# (.NET 11) dotnet add package HyperTabular
Java (JDK 25) implementation("io.github.skunkwerkx:hypertabular:0.8.0")
Go go get github.com/SkunkWerkx/HyperTabular/go@v0.8.0
Swift .package(url: "https://github.com/SkunkWerkx/HyperTabular", from: "0.8.0")
Python (3.11+) pip install hypertabular
Ruby (3.3+) gem install hypertabular
PHP (8.2+) composer require skunkwerkx/hypertabular

Highlights

  • Header first, plan after (#8). A delimited reader or a sheet opens without a plan and reads its header. You resolve column names to ordinals, then bind the plan once.
    • Name lookup is exact and case-sensitive, and returns the first match. A missing name is an error that names the column.
    • This works over a forward-only stream, with no second pass.
    • A plan known by position can still be given up front, as before.
  • Workbooks from a stream (#9). Embedded resources, HTTP responses, any stream: it is read to the end into memory the workbook owns. The format is detected from the bytes.
  • Rows (#10). A batch can also be read across, a row at a time, and every reader and sheet iterates rows across batches. The storage stays column-major. In C#, Row is a ref struct and allocates nothing.
  • Text without strings (#11).
    • C# GetChars and Java chars decode a text cell to UTF-16 in a batch-owned arena, ready for span-based parsers, without making a string.
    • Go TextString views the cell as a string without copying it.
    • Rust text_str borrows the cell as a str.
  • Async with cancellation (#12).
    • C#: ReadAsync, OpenAsync and IAsyncDisposable. These work on Blazor WebAssembly, whose browser streams cannot be read synchronously.
    • Swift: readAsync and async initializers.
    • Python: asyncio.
    • Rust: the async feature.
    • A cancelled read loses nothing, and the next read continues from where it was.
  • Android: arm64 and x86_64, 16 KB pages.
    • C#: CoreCLR and Native AOT, for .NET MAUI.
    • Go: GOOS=android.
    • Swift: the Swift SDK for Android.
    • CI builds the APKs and checks them with zipalign -P 16, then runs the x86_64 builds and the Go and Swift suites in an emulator.

By binding

Header lookup Plan-less open Rows Native text Async
Rust require / ordinal *_unbound for row in batch text_str async feature
C# Ordinal / TryOrdinal overloads + Bind foreach, Rows() GetChars ReadAsync, OpenAsync
Java ordinal / findOrdinal overloads + bind Iterable<Row>, forEachRow chars —
Go Ordinal / Find *Unbound + Bind All() iterators TextString —
Swift ordinal(of:) / firstIndex(of:) overloads + bind Collection of Row — readAsync
Python ordinal / get plan=None + bind iter(batch), rows() — asyncio
Ruby ordinal / find_ordinal plan = nil + bind Enumerable — —
PHP ordinal / find ?array $plan = null + bind foreach, rows() — —

A dash means the language has no use for the feature, not that it is missing. For example, Python, Ruby and PHP strings are already native, Go cancels through its io.Reader, and Java's virtual threads make a blocking read the norm. docs/design.md gives the reason for each.

Upgrading from 0.7

  • HyperCast 0.8.0 is the floor in every package.
  • C#, Java, Swift: the header property is now a Header, which is still a list of strings.
    • Code compiled against 0.7 needs recompiling.
    • In Swift, wrap it in Array(_:) where a [String] is expected.
  • Go: Header() returns Header, a named []string. Only reflection-based comparisons, such as reflect.DeepEqual or testify's Equal, need []string(header).
  • Rust: Error gains NoColumn, Unbound and AlreadyBound. A match that names every variant needs updating.
  • Ruby: a nil plan now means "bind later". Pass [] for an empty plan.

Verifying

Every artifact is built in CI, and all but one are attested, as in 0.7.0:

gh attestation verify <file> --repo SkunkWerkx/HyperTabular

The one exception, found after this release went out: the browser-wasm archive inside the NuGet package, runtimes/browser-wasm/nativeassets/net10.0/libhypertabular.a, the one a Blazor WebAssembly app links. It was built by the same CI workflow as the rest but never attested, so gh attestation verify finds no provenance for it. Everything else in the package, and every other artifact of this release, verifies. 0.9.0 attests the archive where it is built and checks every native before packing.

v0.7.0 - tabular data into typed columns, every cell a HyperCast verdict

Choose a tag to compare

@buvinghausen buvinghausen released this 07 Oct 23:32
594c2b0

HyperTabular's first release. It reads delimited text (CSV, TSV, any single-byte ASCII separator) and workbooks (XLSX and ODS) a batch at a time into typed columns, with a HyperCast verdict for every cell. It starts at 0.7.0 so its version matches HyperCast 0.7.0, the release it is built on. That is the first HyperCast with the interop surface and the typed doors every binding here reads cells through. The full list is in CHANGELOG.md.

Install

Language Package
Rust cargo add hypertabular
C# (.NET 11) dotnet add package HyperTabular
Java (JDK 25) implementation("io.github.skunkwerkx:hypertabular:0.7.0")
Go go get github.com/SkunkWerkx/HyperTabular/go@v0.7.0
Swift .package(url: "https://github.com/SkunkWerkx/HyperTabular", from: "0.7.0")
Python (3.11+) pip install hypertabular
Ruby (3.3+) gem install hypertabular
PHP (8.2+) composer require skunkwerkx/hypertabular

Highlights

  • One native core with no standard library, no allocation and no panic. The binding allocates the input, one value array and one verdict array per column, and the table that locates each cell. The core fills them in one call per batch, so the native boundary is crossed once per few thousand rows, not once per cell. rust/check-core.sh checks all three properties on every PR:

    • the library's imports must match an allow-list, so malloc fails it;
    • every export is under dtolnay's no-panic;
    • a counting allocator sees zero allocations across whole inputs.

    The C ABI has 14 exports, hypertabular_version among them. The library carries none of HyperCast's symbols and links beside libhypercast in one program.

  • Delimited text. A forward-only reader driven by a declared Dialect and a declared column plan; nothing is sniffed. A SIMD scanner (AVX2 and PCLMULQDQ on x86-64, NEON and PMULL on arm64) finds the cells, picked at run time. Text cells are spans into the input, not copies. The reader does about 575 MB/s on linux-x64.

  • Workbooks: XLSX (transitional, strict, .xlsm) and ODS. The core opens the container itself, ZIP64 included, with its own zip reader and inflater, because neither miniz_oxide nor zlib-rs passes the no-panic proof. Each numeric cell is converted according to its number format, through HyperCast's typed doors:

    • an exact decimal;
    • an Excel serial snapped to the time the writer stored;
    • a time of day;
    • a duration.

    The 1900 and 1904 date systems are both handled. Hidden sheets are listed and read. Encrypted packages and OLE files are refused by name. A 300 000-row Excel workbook reads in about 0.6 s on linux-x64.

  • A bad value is a verdict, a broken file is an error. A cell that doesn't cast is a HyperCast Fault (Empty, Malformed or OutOfRange, with the offending span), and the read goes on. A structural error is raised only after every intact row before it has been delivered. Examples are a record of the wrong width, or input that ends inside a quoted cell.

  • Eight packages, one shape. Every binding mirrors the Rust API and takes HyperCast's verdict, format and option types from its own package rather than copying them:

    • a batch per read();
    • values a whole column at a time or a cell at a time;
    • line(row) and each cell's raw text;
    • Workbook / Sheet.

    Per language:

    • C#: LibraryImport, plus Native AOT.
    • Java: FFM, plus GraalVM Native Image.
    • Go: cgo, with the core linked in.
    • Swift: a static library.
    • Python: PyO3, as abi3 wheels.
    • Ruby: a Magnus extension in precompiled platform gems, falling back to Fiddle.
    • PHP: ext-ffi.
  • iOS and Mac Catalyst, in C#, Swift and Go, matching HyperCast and HyperUuid 0.7.0. The core is linked in from static archives built and attested beside everything else:

    • C#: ios-arm64, iossimulator-arm64, and maccatalyst on arm64 and x64.
    • Swift: HyperTabularCoreApple.xcframework.
    • Go: ios, iossimulator and maccatalyst builds, selected by build tag.

    CI runs the suites in an iOS simulator and as a Mac Catalyst process on every PR, and links an iOS device build of each.

  • WebAssembly, wherever a binding's toolchain can link the core. The core imports nothing, so each wasm build links the same archive.

    Language How
    Rust The suite runs under wasmtime on wasm32-wasip1, and in the browser on wasm32-unknown-unknown.
    C# Blazor WebAssembly on .NET 11; the package links the core itself.
    Go TinyGo, -target=wasm or wasip1.
    Swift swift.org's WebAssembly SDK.
    Python A Pyodide 314 wheel; the whole pytest suite runs inside Pyodide under Node and in Chrome.
    Ruby The hypertabular-wasm gem, linked into a ruby.wasm interpreter with rbwasm build.
    Java An in-process GraalWasm backend (-Dhypertabular.backend=wasm). It is also the automatic fallback where the jar has no native build.

    PHP is not built for WebAssembly, for the same reasons as HyperCast.

  • Platforms.

    • Tested on real hardware: linux-x64, linux-arm64, linux-musl-x64, linux-musl-arm64, osx-arm64, win-x64 and win-arm64.
    • osx-x64 is built and tested at the core.
    • Plus iOS, Mac Catalyst and WebAssembly as above.
  • A shared conformance corpus. 33 delimited cases and 198 workbook cases, replayed byte for byte by the Rust suite and by every binding. The workbooks include files written by Excel for Windows and the web, LibreOffice and Google Sheets.

  • Fuzzed. Four libFuzzer targets, about four million executions before release, with nothing found in the core:

    • delimited text;
    • raw XLSX/ODS containers;
    • workbook XML;
    • the inflater, checked against zlib-rs.

Verifying

Every native library and static archive is built by CI and carries a build-provenance attestation:

gh attestation verify go/staticlib/linux_amd64/libhypertabular.a \
  --repo SkunkWerkx/HyperTabular --signer-repo SkunkWerkx/.github

The Python wheels and the crate are attested by the forge the same way, so they also need --signer-repo SkunkWerkx/.github. The gems and the NuGet package are attested by this repo's own release.yml, so --repo SkunkWerkx/HyperTabular alone verifies them.