Repository navigation
The first stable release. Every package's public API and the platforms in the README's
Platform support table are now a commitment under Semantic
Versioning: breaking or dropping either takes a new major version. Every binding reads Apple
Numbers beside XLSX and ODS; a live stream hands over each header and batch as it arrives,
in Rust, Python and Swift; a plan can no longer ask for unbounded memory, with one ordinal
cap, 2^20, in every binding; the Rust API is fixed at what a 1.0 can promise
(#[non_exhaustive], a kernel surface of only what a binding calls); what was never used
goes (Java's GraalWasm backend, Python's BACKEND); and every shipped file verifies from
the release tag alone.
Added
- Rust —
Batch::values::<T>is public. It lends any typed column whole — decimals,
UUIDs, timestamps, dates, date-times and durations as well as the primitives — where only
the primitives had accessors (batch.i64(col)and its siblings, which it now underlies).
The slice borrows the reader as theirs do, and a cell that did not cast holds the door's
zero beside its verdict, as theirs does. (Rust) - C#, Java and Swift lend more columns whole. C#'s
Batch.Values<T>lends a decimal
column as HyperCast'sRawDecimal, a timestamp, Unix or Excel-serial column as
RawTimestamp, a date column asRawDate, a date-time column asRawCiviland a
duration column asRawDuration, zero-copy, and a time column asulongnanoseconds; a
UUID column is still read withGet<Guid>, since the core's RFC 9562 bytes are not a
Guid's layout. Java'svalues(column)returns the column segment for every door but
text, read with HyperCast'sNativeValuesreaders at their*_BYTESstride. Swift's
values(_:as: UInt64.self)lends a time column's nanoseconds; HyperCast's Swift package
has no value types with the core's layout, so the other doors stay a cell at a time.
(C#, Java, Swift) - Apple Numbers. Every binding reads a
.numbersdocument — Apple's IWA packaging:
Snappy-compressed chunks of Protocol Buffers objects — the way it reads XLSX and ODS,
told by what the container holds, never by its name. Each table is a sheet, listed
under its sheet's name (<sheet> - <table>when a sheet holds several, as Numbers'
own Excel export names them) and hidden when its sheet is; cells are read from the
v5 storage Numbers has written since 2018 — decimals and currencies as numbers, dates
as wall clocks or, under a format that shows no date (a custom format's pattern looked
up in the document's list), times of day, durations as spans, booleans, text and rich
text with the runs joined, error cells as#VALUE!
(the file does not record which error), the cells a merge covers as empty — and a
row a tile leaves out is an empty row. The core's newkernel/workbook/{iwa,snappy, proto,numbers}.rsstay inside the no-panic, no-allocation proofs; the window holds
only the chunks of the object in hand, and a deflated part is inflated into its far
end.FormatgainsNumbers(3),Opened.strings_bytesis what the
data-list parts decode to, and a new failure kindiwa(26, part10) names
a Numbers part that is not what the packaging says —found1 a broken chunk, 2 a
malformed message, 3 an object a reference names that is not there (expectedits
identifier), 4 cell storage of a version the core does not read (expectedthe
version). Thenot_a_workbookmessage now says "not an XLSX, ODS or Numbers
workbook". The corpus gainsnumbers-data.numbers, written by Numbers 26.3.1 on
macOS, its Excel exportnumbers-data.xlsx, and two generated Numbers packages in the
layouts Numbers never writes small (chunks of a few bytes, seven-row tiles, wide
offsets, a segmented string list, rows left out of their tile);workbook.jsongrows
from 198 to 246 cases, every one of them new. (Rust, C#, Java, Go, Python, Ruby, PHP, Swift) - A core freshness check for the suites that link committed binaries. Go's, Swift's
and PHP's suites refuse to run againstgo/staticlib, the Swift artifact bundle or
php/src/nativewhenrust/src,rust/Cargo.tomlorrust/Cargo.lockdiffer from the
commit those binaries were staged from, and name the local-core command to run instead
(Go and PHP stop before any test; Swift failsCoreFreshnessTests). CI, the local core,
shallow clones and a checkout with fresh binaries written into the tree skip it.
.github/scripts/local-core.shfinds a Forge checkout at../HyperForgeas well as
../.github. (repository only)
Changed
- The release flow, before its first run here. The wheels are uploaded by a step of
release.yml's ownpypi-publishjob, so their PEP 740 publish attestations name
release.yml, as 0.9.0's do (the Forge's composite action nested the upload, where it
cannot find its own container image; HyperUuid's and HyperCast's 1.0.0 wheels went up
from another workflow for it); staging starts by itself once the release-mode CI on the
preparation commit is green, and a second request for a commit already staged is skipped
with a notice; the Forge commit's step output isrevision, which keeps CodeQL's
actions/cache-poisoning/poisonable-stepquiet. (repository only) - Built on HyperCast 1.0.0. Every binding depends on HyperCast's first stable release,
within its major: the crate1.0.0, NuGet1.0.0, Maven1.0.0, Gov1.0.0, Python
>=1.0.0,<2, Ruby~> 1.0, PHP^1.0.0and Swiftfrom: "1.0.0", where it was0.9.0
(Swift: a commit pin, which no tagged package may carry). (all packages) - Swift — on HyperCast 1.0.0, whose doors do not throw. HyperCast's Swift doors and
Cast.nativeVersion()droppedthrows(bad data was always the verdict'sFault), so a
call needs notry; one left in place compiles with a warning. (Swift) - Breaking — Rust: the kernel's public surface is what a binding calls, and no more.
kernel::abi,kernel::door,kernel::exports,kernel::delimited::fill, and
kernel::workbook's state, memory, format constants andbookandrowsmodules stay
public; the delimited engine, scanner and unescape, the inflate, and the workbook's zip,
XML, styles, number, part, sort, cell, Snappy, protobuf, IWA and Numbers modules are
private to the crate, so 1.0.0 does not freeze them. The tests that drove
them directly are compiled into the crate fromtests/and every integration target is
listed in the manifest. (Rust) - Breaking — Rust:
Column,Dialect,SheetOptions,Door,Error,Formatand
FailureKindare#[non_exhaustive]. A setting, a door, a format the core learns to
read or a failure it can report is an addition without a new major version once 1.0.0 is
out. The three configuration structs are built with their constructors, constants and
builder methods rather than struct literals; a match on an enum outside the crate names a
wildcard arm, and a failure code this binding does not know reads asContainer, as it
already did. (Rust) - C# —
HyperTabular.targetsno longer lists anEmccExportedFunctionper export. The
NativeFileReferenceis the whole Blazor WebAssembly wiring: the SDK's generated pinvoke
table calls each native function by symbol, so the link resolves them against the archive
andLibraryImport("*")reaches them through that table. The smoke test reaches all
fourteen exports with the list gone (on SDK 11.0.100-rc.2) and fails at the link with the
archive unlinked, so the list was dead weight. (C#) - The release is two acts at one tag, and every shipped file verifies from the tag.
PushingvX.Y.Zpacks every registry input, attests each package and the release as a
whole, preserves the bytes and plans every upload without publishing; a dispatch of
release.ymlwithprepared_run_idpublishes from the preserved bytes through what was
the recovery path. The release attestation is one statement signed at the tag whose
subjects are every registry package and every native library committed for Go, Swift and
PHP, and whose predicate is the release inventory, so
gh attestation verify <file> --repo SkunkWerkx/HyperTabular --source-ref refs/tags/vX.Y.Z --predicate-type https://github.com/SkunkWerkx/.github/predicates/release/v1answers for
any of them, including the wheels and the committed natives that build provenance alone
tied to the preparation commit on master (SkunkWerkx/HyperUuid#35). The GitHub Release
notes come from this file's section for the version plus the verification commands, and a
tag whose CHANGELOG lacks the section is refused before anything publishes. The job bodies
ofrelease.yml,prepare-release.yml,stage-native-binaries.ymland every static check
inci.ymlmoved to the Forge (SkunkWerkx/.github, called at master, composite actions
where the repository's own identity signs or publishes), with this repository described to
it by.github/hyper.jsonand.github/scripts/abi_probe.c; CI gainedcheck-semver
against the last crates.io release, the no_std consumer build and the shared library's
export check on every RID. Dependabot bumps every ecosystem in one monthly pull request,
every third-party action is pinned to a commit, and CodeQL runs weekly. CI also runs the
Rust suite under wasmtime onwasm32-wasip1(test-wasi) and SwiftLint'smissing_docs
over the Swift package's public API (swift/.swiftlint.yml), on every pull request.
(repository only)
Removed
- Java — the GraalWasm backend. The jar no longer carries the core a second time as a
wasm32-wasip1module, andTabular.backend(), thehypertabular.backendproperty and the
fall-back to the module on a platform without a native build are gone: the binding is FFM
downcalls into the library it bundles. The module served a platform nobody had asked for,
and replaying the suite through it in interpreter mode on five CI legs cost more than every
other step of a leg together once the Numbers reader was in it (23 minutes against 15 on
linux-x64).cargo wasm-moduleand thewasm32-wasip1export flags leave
rust/.cargo/config.tomlwith it. (Java) - Python —
hypertabular.BACKEND. It was always"native"once the PyO3 extension
became the only backend, so it named no choice.hypertabular.native_version()still
reports the core the extension was built from. (PyPI)
Fixed
- A worksheet cut off between two rows is a broken part, not a shorter sheet. An XLSX
worksheet whose XML ended after a whole</row>without closingsheetData— cut off by
the file, or by a zip directory that declared the part that long — read as a shorter
sheet: the rows past the cut were gone with no failure. The part reader now counts the
elements a part has opened and not closed, and the end of a part inside one is
FailureKind::Xml(found6, an unterminated element), raised after every intact row
before it, final for the sheet, in every part the core reads — a string table cut
between two strings is refused at load. (all packages) - A string table that claims a gigabyte is not given one. The size a workbook's zip
directory declared forsharedStrings.xmlwas what every binding sized its string arena
by, before the part was read, capped only at 256 MiB; a kilobyte of XLSX could ask for
all of it. The core now holds that figure to what the part's compressed bytes can inflate
to (1032 to one, deflate's own ceiling), so the arena is never larger than the part
could fill, and an honest file is sized as before. (all packages) - Rust — the delimited row ceiling no longer depends on the buffer the reader began
with. A stream's buffer started atbuffer_bytesand doubled unclamped, so a record
longer thanDelimitedReader::MAX_ROW_BYTESwas read when the buffer happened to hold it
and refused when it did not. The buffer now starts at most at the ceiling and grows to it
and no further, throughReadand theasyncfeature'sAsyncReadalike, so a record
that fills the ceiling without ending isFailureKind::RowTooLongwhatever the buffer
began at — as the other seven bindings already read. (Rust) - Rust — a live delimited stream no longer holds back a header or batch that has arrived.
A refill, throughReador theasyncfeature'sAsyncRead, used to read until the
buffer was full or the source ended, so a pipe or a socket that paused after a few
records held them back; it now goes back to parsing after any read that returns bytes.
Interrupted reads, records split across reads, cancellation and terminal errors behave
as before. (Rust) - Python — the same, for a file and a file object. A file
DelimitedReader.openopened
(a FIFO, say) was read until the buffer was full, and a buffered file object's
read(size)over a pipe or a socket waits for all ofsize. The reader now parses after
any read that returns bytes, calls a file object'sread1(size)where it has one, and
opens the path with the interpreter released, so a FIFO whose writer is another Python
thread no longer deadlocks the open. (PyPI) - Rust — a slice of 2 GiB or more is read.
from_slicehanded the core the whole slice
in one call, and the core takes under 2 GiB a call (its offsets are 31 bits), so a larger
slice failed at once asError::Separator. It is handed over in windows under 2 GiB now,
as Python already hands overbytes, and only a single record that long is
FailureKind::RowTooLong. (Rust) - Swift — a live asynchronous sequence is read as it arrives, in whole batches. A reader
opened from anAsyncSequenceawaited its bytes until the read buffer filled or the
sequence ended, so a header or row that had arrived from a live source (URLSession.bytes)
was held back. A task owned by the reader now awaits the sequence, up tobufferBytes
ahead, and a read takes everything that has arrived, waiting only when nothing has: a
batch isbatchRowsrows when that many are in, and a row from a paused source is read
within 200 µs. A cancelled read still loses nothing, a sequence's error still follows
every byte before it, and the task stops when the reader is released. The sequence is
passedsending. (Swift) - Swift — a FIFO or pipe opened by path is read as it is written.
DelimitedReader(contentsOfFile:)read throughFileHandle.read(upToCount:), which on a
FIFO waits for the whole count; off Windows it makes oneread(2)per refill. (Swift) - Every binding refuses a plan ordinal of 2^20 (1,048,576) or more, as its own plan
error. No workbook format has that many columns (XLSX and ODS stop at 16,384, Numbers
at 1,000), and at the cap one row of cell table is at most 8 MiB and a sheet's row slots
at most 16 MiB, so no plan can ask for more. Before, the bounds differed and some broke:
Rust took any ordinal that fit au32, so onwasm32Column::text(usize::MAX)
overflowed and panicked and on a 64-bit host an ordinal nearu32::MAXsized a cell table
of tens of terabytes and aborted on the allocation; Swift crashed on a precondition for an
ordinal of 2³¹ − 1 or more; C# overflowedOrdinal + 1atint.MaxValueand bound a
corrupt width; Java took everyint; Python, Ruby, Go and PHP stopped at 2³¹ − 1. Now an
ordinal of 2^20 or more is refused for a delimited plan and a sheet's alike, before
anything is allocated: RustError::Plan, C#ArgumentException, Java
IllegalArgumentException, Swift the newPlanError.outOfReach(column:), Python
ValueError, RubyArgumentError, PHPInvalidArgumentException, Go's plan error.
(Rust, C#, Java, Go, Python, Ruby, PHP, Swift) - A plan column's ordinal no longer sizes a batch of cell table. A delimited reader's
cell table took the plan's widest ordinal plus two entries for every row of a batch, and
blanked them all on every row, so on a two-row fileColumn::i64(100_000)took 3.2 GB,
2 000 000 took 8 GB, and 1 000 000 000 was killed past 10 GB with nothing read. Now,
once a header has fixed the width every record has,bind— or a constructor given the
plan, as soon as it has read the header — refuses a column at or past that width as the
binding's own plan error naming it (RustError::Plan, C#ArgumentException, Java
IllegalArgumentException, SwiftPlanError.outOfReach(column:), PythonValueError,
RubyArgumentError, PHPInvalidArgumentException, Go's plan error): such a column
could only ever be empty. Without a header, where the width is not known at bind, the
table holds as many rows as 4096 × 257 entries (8 MiB, a default batch of a plan
reaching 256 source columns) allow, and never less than one: a plan reaching further
reads shorter batches, and its far columns read as empty in the memory one row's reach
takes, at most 8 MiB under the 2^20 cap. An ordinary plan's batches are full as before, and a sheet is unchanged — its cell
table is a batch of plan columns, not of reach, and its header does not fix the width of
the rows below. The corpus case "a plan is a projection, in its own order", which read
ordinal 9 behind a four-name header as empty, reads the same record without a header.
(Rust, C#, Java, Go, Python, Ruby, PHP, Swift)
The full list is in CHANGELOG.md.
Verifying
Every file in this release verifies from the tag alone. One attestation, signed by this
repository's release.yml at v1.0.0, names all 102 of them: every registry
package (NuGet, RubyGems, PyPI, crates.io and Maven Central), the 34 native
libraries committed for Go, Swift and PHP, and the 8 metadata files staged beside
them. Its predicate is the release inventory attached below.
gh attestation verify <file> --repo SkunkWerkx/HyperTabular \
--source-ref refs/tags/v1.0.0 --predicate-type https://github.com/SkunkWerkx/.github/predicates/release/v1The native libraries, the Python wheels and the archives inside the NuGet package and the
jar also carry build provenance from CI run 40253806642
at commit b9f3a23, the parent of the tagged commit, signed by the Forge:
gh attestation verify <file> --repo SkunkWerkx/HyperTabular --signer-repo SkunkWerkx/.github --source-digest b9f3a234968869a324bb8b53cb28263e6e570401inventory.json lists every file with its size and SHA-256 and, for each file with build
provenance, the commit it was built from and the repository that signed it; it carries
build provenance of its own at the tagged commit. PyPI additionally holds a PEP 740 publish attestation for every wheel.