Skip to content

v1.0.0

Latest

Choose a tag to compare

@github-actions github-actions released this 26 Oct 03:03
· 9 commits to master since this release

The first stable release. Every package's public API and the platforms in the README's
Platform support table are now a commitment under Semantic
Versioning: breaking or dropping either takes a new major version. Every binding reads Apple
Numbers beside XLSX and ODS; a live stream hands over each header and batch as it arrives,
in Rust, Python and Swift; a plan can no longer ask for unbounded memory, with one ordinal
cap, 2^20, in every binding; the Rust API is fixed at what a 1.0 can promise
(#[non_exhaustive], a kernel surface of only what a binding calls); what was never used
goes (Java's GraalWasm backend, Python's BACKEND); and every shipped file verifies from
the release tag alone.

Added

  • Rust — Batch::values::<T> is public. It lends any typed column whole — decimals,
    UUIDs, timestamps, dates, date-times and durations as well as the primitives — where only
    the primitives had accessors (batch.i64(col) and its siblings, which it now underlies).
    The slice borrows the reader as theirs do, and a cell that did not cast holds the door's
    zero beside its verdict, as theirs does. (Rust)
  • C#, Java and Swift lend more columns whole. C#'s Batch.Values<T> lends a decimal
    column as HyperCast's RawDecimal, a timestamp, Unix or Excel-serial column as
    RawTimestamp, a date column as RawDate, a date-time column as RawCivil and a
    duration column as RawDuration, zero-copy, and a time column as ulong nanoseconds; a
    UUID column is still read with Get<Guid>, since the core's RFC 9562 bytes are not a
    Guid's layout. Java's values(column) returns the column segment for every door but
    text, read with HyperCast's NativeValues readers at their *_BYTES stride. Swift's
    values(_:as: UInt64.self) lends a time column's nanoseconds; HyperCast's Swift package
    has no value types with the core's layout, so the other doors stay a cell at a time.
    (C#, Java, Swift)
  • Apple Numbers. Every binding reads a .numbers document — Apple's IWA packaging:
    Snappy-compressed chunks of Protocol Buffers objects — the way it reads XLSX and ODS,
    told by what the container holds, never by its name. Each table is a sheet, listed
    under its sheet's name (<sheet> - <table> when a sheet holds several, as Numbers'
    own Excel export names them) and hidden when its sheet is; cells are read from the
    v5 storage Numbers has written since 2018 — decimals and currencies as numbers, dates
    as wall clocks or, under a format that shows no date (a custom format's pattern looked
    up in the document's list), times of day, durations as spans, booleans, text and rich
    text with the runs joined, error cells as #VALUE!
    (the file does not record which error), the cells a merge covers as empty — and a
    row a tile leaves out is an empty row. The core's new kernel/workbook/{iwa,snappy, proto,numbers}.rs stay inside the no-panic, no-allocation proofs; the window holds
    only the chunks of the object in hand, and a deflated part is inflated into its far
    end. Format gains Numbers (3), Opened.strings_bytes is what the
    data-list parts decode to, and a new failure kind iwa (26, part 10) names
    a Numbers part that is not what the packaging says — found 1 a broken chunk, 2 a
    malformed message, 3 an object a reference names that is not there (expected its
    identifier), 4 cell storage of a version the core does not read (expected the
    version). The not_a_workbook message now says "not an XLSX, ODS or Numbers
    workbook". The corpus gains numbers-data.numbers, written by Numbers 26.3.1 on
    macOS, its Excel export numbers-data.xlsx, and two generated Numbers packages in the
    layouts Numbers never writes small (chunks of a few bytes, seven-row tiles, wide
    offsets, a segmented string list, rows left out of their tile); workbook.json grows
    from 198 to 246 cases, every one of them new. (Rust, C#, Java, Go, Python, Ruby, PHP, Swift)
  • A core freshness check for the suites that link committed binaries. Go's, Swift's
    and PHP's suites refuse to run against go/staticlib, the Swift artifact bundle or
    php/src/native when rust/src, rust/Cargo.toml or rust/Cargo.lock differ from the
    commit those binaries were staged from, and name the local-core command to run instead
    (Go and PHP stop before any test; Swift fails CoreFreshnessTests). CI, the local core,
    shallow clones and a checkout with fresh binaries written into the tree skip it.
    .github/scripts/local-core.sh finds a Forge checkout at ../HyperForge as well as
    ../.github. (repository only)

Changed

  • The release flow, before its first run here. The wheels are uploaded by a step of
    release.yml's own pypi-publish job, so their PEP 740 publish attestations name
    release.yml, as 0.9.0's do (the Forge's composite action nested the upload, where it
    cannot find its own container image; HyperUuid's and HyperCast's 1.0.0 wheels went up
    from another workflow for it); staging starts by itself once the release-mode CI on the
    preparation commit is green, and a second request for a commit already staged is skipped
    with a notice; the Forge commit's step output is revision, which keeps CodeQL's
    actions/cache-poisoning/poisonable-step quiet. (repository only)
  • Built on HyperCast 1.0.0. Every binding depends on HyperCast's first stable release,
    within its major: the crate 1.0.0, NuGet 1.0.0, Maven 1.0.0, Go v1.0.0, Python
    >=1.0.0,<2, Ruby ~> 1.0, PHP ^1.0.0 and Swift from: "1.0.0", where it was 0.9.0
    (Swift: a commit pin, which no tagged package may carry). (all packages)
  • Swift — on HyperCast 1.0.0, whose doors do not throw. HyperCast's Swift doors and
    Cast.nativeVersion() dropped throws (bad data was always the verdict's Fault), so a
    call needs no try; one left in place compiles with a warning. (Swift)
  • Breaking — Rust: the kernel's public surface is what a binding calls, and no more.
    kernel::abi, kernel::door, kernel::exports, kernel::delimited::fill, and
    kernel::workbook's state, memory, format constants and book and rows modules stay
    public; the delimited engine, scanner and unescape, the inflate, and the workbook's zip,
    XML, styles, number, part, sort, cell, Snappy, protobuf, IWA and Numbers modules are
    private to the crate, so 1.0.0 does not freeze them. The tests that drove
    them directly are compiled into the crate from tests/ and every integration target is
    listed in the manifest. (Rust)
  • Breaking — Rust: Column, Dialect, SheetOptions, Door, Error, Format and
    FailureKind are #[non_exhaustive].
    A setting, a door, a format the core learns to
    read or a failure it can report is an addition without a new major version once 1.0.0 is
    out. The three configuration structs are built with their constructors, constants and
    builder methods rather than struct literals; a match on an enum outside the crate names a
    wildcard arm, and a failure code this binding does not know reads as Container, as it
    already did. (Rust)
  • C# — HyperTabular.targets no longer lists an EmccExportedFunction per export. The
    NativeFileReference is the whole Blazor WebAssembly wiring: the SDK's generated pinvoke
    table calls each native function by symbol, so the link resolves them against the archive
    and LibraryImport("*") reaches them through that table. The smoke test reaches all
    fourteen exports with the list gone (on SDK 11.0.100-rc.2) and fails at the link with the
    archive unlinked, so the list was dead weight. (C#)
  • The release is two acts at one tag, and every shipped file verifies from the tag.
    Pushing vX.Y.Z packs every registry input, attests each package and the release as a
    whole, preserves the bytes and plans every upload without publishing; a dispatch of
    release.yml with prepared_run_id publishes from the preserved bytes through what was
    the recovery path. The release attestation is one statement signed at the tag whose
    subjects are every registry package and every native library committed for Go, Swift and
    PHP, and whose predicate is the release inventory, so
    gh attestation verify <file> --repo SkunkWerkx/HyperTabular --source-ref refs/tags/vX.Y.Z --predicate-type https://github.com/SkunkWerkx/.github/predicates/release/v1 answers for
    any of them, including the wheels and the committed natives that build provenance alone
    tied to the preparation commit on master (SkunkWerkx/HyperUuid#35). The GitHub Release
    notes come from this file's section for the version plus the verification commands, and a
    tag whose CHANGELOG lacks the section is refused before anything publishes. The job bodies
    of release.yml, prepare-release.yml, stage-native-binaries.yml and every static check
    in ci.yml moved to the Forge (SkunkWerkx/.github, called at master, composite actions
    where the repository's own identity signs or publishes), with this repository described to
    it by .github/hyper.json and .github/scripts/abi_probe.c; CI gained check-semver
    against the last crates.io release, the no_std consumer build and the shared library's
    export check on every RID. Dependabot bumps every ecosystem in one monthly pull request,
    every third-party action is pinned to a commit, and CodeQL runs weekly. CI also runs the
    Rust suite under wasmtime on wasm32-wasip1 (test-wasi) and SwiftLint's missing_docs
    over the Swift package's public API (swift/.swiftlint.yml), on every pull request.
    (repository only)

Removed

  • Java — the GraalWasm backend. The jar no longer carries the core a second time as a
    wasm32-wasip1 module, and Tabular.backend(), the hypertabular.backend property and the
    fall-back to the module on a platform without a native build are gone: the binding is FFM
    downcalls into the library it bundles. The module served a platform nobody had asked for,
    and replaying the suite through it in interpreter mode on five CI legs cost more than every
    other step of a leg together once the Numbers reader was in it (23 minutes against 15 on
    linux-x64). cargo wasm-module and the wasm32-wasip1 export flags leave
    rust/.cargo/config.toml with it. (Java)
  • Python — hypertabular.BACKEND. It was always "native" once the PyO3 extension
    became the only backend, so it named no choice. hypertabular.native_version() still
    reports the core the extension was built from. (PyPI)

Fixed

  • A worksheet cut off between two rows is a broken part, not a shorter sheet. An XLSX
    worksheet whose XML ended after a whole </row> without closing sheetData — cut off by
    the file, or by a zip directory that declared the part that long — read as a shorter
    sheet: the rows past the cut were gone with no failure. The part reader now counts the
    elements a part has opened and not closed, and the end of a part inside one is
    FailureKind::Xml (found 6, an unterminated element), raised after every intact row
    before it, final for the sheet, in every part the core reads — a string table cut
    between two strings is refused at load. (all packages)
  • A string table that claims a gigabyte is not given one. The size a workbook's zip
    directory declared for sharedStrings.xml was what every binding sized its string arena
    by, before the part was read, capped only at 256 MiB; a kilobyte of XLSX could ask for
    all of it. The core now holds that figure to what the part's compressed bytes can inflate
    to (1032 to one, deflate's own ceiling), so the arena is never larger than the part
    could fill, and an honest file is sized as before. (all packages)
  • Rust — the delimited row ceiling no longer depends on the buffer the reader began
    with.
    A stream's buffer started at buffer_bytes and doubled unclamped, so a record
    longer than DelimitedReader::MAX_ROW_BYTES was read when the buffer happened to hold it
    and refused when it did not. The buffer now starts at most at the ceiling and grows to it
    and no further, through Read and the async feature's AsyncRead alike, so a record
    that fills the ceiling without ending is FailureKind::RowTooLong whatever the buffer
    began at — as the other seven bindings already read. (Rust)
  • Rust — a live delimited stream no longer holds back a header or batch that has arrived.
    A refill, through Read or the async feature's AsyncRead, used to read until the
    buffer was full or the source ended, so a pipe or a socket that paused after a few
    records held them back; it now goes back to parsing after any read that returns bytes.
    Interrupted reads, records split across reads, cancellation and terminal errors behave
    as before. (Rust)
  • Python — the same, for a file and a file object. A file DelimitedReader.open opened
    (a FIFO, say) was read until the buffer was full, and a buffered file object's
    read(size) over a pipe or a socket waits for all of size. The reader now parses after
    any read that returns bytes, calls a file object's read1(size) where it has one, and
    opens the path with the interpreter released, so a FIFO whose writer is another Python
    thread no longer deadlocks the open. (PyPI)
  • Rust — a slice of 2 GiB or more is read. from_slice handed the core the whole slice
    in one call, and the core takes under 2 GiB a call (its offsets are 31 bits), so a larger
    slice failed at once as Error::Separator. It is handed over in windows under 2 GiB now,
    as Python already hands over bytes, and only a single record that long is
    FailureKind::RowTooLong. (Rust)
  • Swift — a live asynchronous sequence is read as it arrives, in whole batches. A reader
    opened from an AsyncSequence awaited its bytes until the read buffer filled or the
    sequence ended, so a header or row that had arrived from a live source (URLSession.bytes)
    was held back. A task owned by the reader now awaits the sequence, up to bufferBytes
    ahead, and a read takes everything that has arrived, waiting only when nothing has: a
    batch is batchRows rows when that many are in, and a row from a paused source is read
    within 200 µs. A cancelled read still loses nothing, a sequence's error still follows
    every byte before it, and the task stops when the reader is released. The sequence is
    passed sending. (Swift)
  • Swift — a FIFO or pipe opened by path is read as it is written.
    DelimitedReader(contentsOfFile:) read through FileHandle.read(upToCount:), which on a
    FIFO waits for the whole count; off Windows it makes one read(2) per refill. (Swift)
  • Every binding refuses a plan ordinal of 2^20 (1,048,576) or more, as its own plan
    error.
    No workbook format has that many columns (XLSX and ODS stop at 16,384, Numbers
    at 1,000), and at the cap one row of cell table is at most 8 MiB and a sheet's row slots
    at most 16 MiB, so no plan can ask for more. Before, the bounds differed and some broke:
    Rust took any ordinal that fit a u32, so on wasm32 Column::text(usize::MAX)
    overflowed and panicked and on a 64-bit host an ordinal near u32::MAX sized a cell table
    of tens of terabytes and aborted on the allocation; Swift crashed on a precondition for an
    ordinal of 2³¹ − 1 or more; C# overflowed Ordinal + 1 at int.MaxValue and bound a
    corrupt width; Java took every int; Python, Ruby, Go and PHP stopped at 2³¹ − 1. Now an
    ordinal of 2^20 or more is refused for a delimited plan and a sheet's alike, before
    anything is allocated: Rust Error::Plan, C# ArgumentException, Java
    IllegalArgumentException, Swift the new PlanError.outOfReach(column:), Python
    ValueError, Ruby ArgumentError, PHP InvalidArgumentException, Go's plan error.
    (Rust, C#, Java, Go, Python, Ruby, PHP, Swift)
  • A plan column's ordinal no longer sizes a batch of cell table. A delimited reader's
    cell table took the plan's widest ordinal plus two entries for every row of a batch, and
    blanked them all on every row, so on a two-row file Column::i64(100_000) took 3.2 GB,
    2 000 000 took 8 GB, and 1 000 000 000 was killed past 10 GB with nothing read. Now,
    once a header has fixed the width every record has, bind — or a constructor given the
    plan, as soon as it has read the header — refuses a column at or past that width as the
    binding's own plan error naming it (Rust Error::Plan, C# ArgumentException, Java
    IllegalArgumentException, Swift PlanError.outOfReach(column:), Python ValueError,
    Ruby ArgumentError, PHP InvalidArgumentException, Go's plan error): such a column
    could only ever be empty. Without a header, where the width is not known at bind, the
    table holds as many rows as 4096 × 257 entries (8 MiB, a default batch of a plan
    reaching 256 source columns) allow, and never less than one: a plan reaching further
    reads shorter batches, and its far columns read as empty in the memory one row's reach
    takes, at most 8 MiB under the 2^20 cap. An ordinary plan's batches are full as before, and a sheet is unchanged — its cell
    table is a batch of plan columns, not of reach, and its header does not fix the width of
    the rows below. The corpus case "a plan is a projection, in its own order", which read
    ordinal 9 behind a four-name header as empty, reads the same record without a header.
    (Rust, C#, Java, Go, Python, Ruby, PHP, Swift)

The full list is in CHANGELOG.md.

Verifying

Every file in this release verifies from the tag alone. One attestation, signed by this
repository's release.yml at v1.0.0, names all 102 of them: every registry
package (NuGet, RubyGems, PyPI, crates.io and Maven Central), the 34 native
libraries committed for Go, Swift and PHP, and the 8 metadata files staged beside
them. Its predicate is the release inventory attached below.

gh attestation verify <file> --repo SkunkWerkx/HyperTabular \
  --source-ref refs/tags/v1.0.0 --predicate-type https://github.com/SkunkWerkx/.github/predicates/release/v1

The native libraries, the Python wheels and the archives inside the NuGet package and the
jar also carry build provenance from CI run 40253806642
at commit b9f3a23, the parent of the tagged commit, signed by the Forge:

gh attestation verify <file> --repo SkunkWerkx/HyperTabular --signer-repo SkunkWerkx/.github --source-digest b9f3a234968869a324bb8b53cb28263e6e570401

inventory.json lists every file with its size and SHA-256 and, for each file with build
provenance, the commit it was built from and the repository that signed it; it carries
build provenance of its own at the tagged commit. PyPI additionally holds a PEP 740 publish attestation for every wheel.