v0.2.5

All six crates released together at 0.2.5. The version is the workspace's, so they move as one.

Crate
ytsaurus-yson	YSON codec, text and binary
ytsaurus-job	job runtime
ytsaurus-client	HTTP API v4 launcher
ytsaurus-helpers	#[derive(TableRow)] — first release
ytsaurus-skiff	Skiff schema and codec — first release, pre-release
ytsaurus-format	DataFormat — first release, pre-release

ytsaurus-skiff and ytsaurus-format carried publish = false until now. They are on the registry because ytsaurus-job and ytsaurus-client depend on them and could not be published otherwise; the Skiff ship gates are still not all green and their API may change in a patch release.

What is in it

Making a managed installation runnable from the environment (#50) — nine issues found by running the whole example suite against a real multi-node cluster instead of a local Docker one.

• Client::with_heavy_proxies_under(domains) — the configured address's domain plus the ones named. An installation that publishes its heavy proxies in a zone of its own had every one of them refused, and the only answers were writing all 79 names out or removing the rule.
• Four new Client::from_env variables — YT_PROXY_SUFFIX, YT_HEAVY_PROXY_DOMAINS, YT_HEAVY_PROXIES_ANYWHERE, YT_FILE_CACHE — each inert when unset. Every example builds its client with from_env, so a policy settable only in Rust is a policy an example cannot be run under.
• invalid peer certificate: UnknownIssuer now names the two things that fix it, YT_CA_BUNDLE and the platform-verifier feature.
• Both /hosts refusals offer all three settings with their environment spellings, and name a domain that could not be used rather than dropping it in silence.
• cached_upload brings its own file cache; profile builds its caption from the run and defaults to five rounds.
• Docs: a runbook for a cluster that is not the local one, a full environment-variable table, --features tls for selfrun against https, and the decoding share recorded as open — 10.6 % on a local cluster against 36.2 % on a production one, either side of the 30 % threshold the Skiff question turns on.

Full detail in each crate's CHANGELOG.

v0.2.0 — API fixes from the pilot, plus a launcher

Everything in this release came from using 0.1.0 rather than reviewing it: a
production-shaped pilot (sessionize) and a
new launcher were written against the API, and every place it forced a
workaround was filed as an issue. All four are closed here.

New: ytsaurus-client

A thin HTTP API v4 client — run a job with no Python installation:

let client = Client::from_env()?;
client.upload_worker("target/…/my_job", "//tmp/my_job")?;
let id = client.start_map(&spec)?;
client.wait_for_operation(&id)?;

Verified against a local cluster with nothing Python on PATH: creates tables,
uploads the worker as executable, writes rows, runs a map, polls to completion,
reads both tables back and compares them byte for byte.

Parameters and specs are encoded with this project's own YSON codec rather than
JSON, which keeps the dependency list short and exercises ytsaurus-yson
against a real cluster on every request.

Two limits are documented rather than hidden: heavy commands are not routed via
/hosts, and ureq 3.3 exposes no trailers, so a failure the proxy reports
mid-stream cannot be seen. read_table compensates by rejecting a response that
is not a complete YSON list fragment.

ytsaurus-job

Reduce keys (#2). A reducer no longer re-derives the key from its first
row:

let mut groups = reader.groups_by(["user_id"]);
while let Some(mut group) = groups.next_group()? {
    let user = group.key().bytes("user_id").unwrap_or_default();
    // ...
}

YTsaurus does not transmit the key — key_switch carries no payload — so this
reads it from the group's first row: the same work, done once instead of in
every reducer. Accessors are byte-first, because reduce keys routinely are not
UTF-8.

Error classification (#1). JobError::kind() gives a stable,
allocation-free identifier for a reason column; is_row_local() separates
"quarantine this row" from "the stream is broken, stop". On the pilot's cluster
run the rejects table collapsed to 3x invalid_yson instead of three distinct
formatted messages.

Named output tables (#4):

let (mut writer, [events, rejects]) = JobWriter::named(["events", "rejects"])?;
writer.write(rejects, &row)?;

Two output tables of different meaning is exactly where transposing 0 and 1
gives a job that runs happily and fills each table with the other's rows.

Documentation

The guide now covers the output side (#3): an output row may borrow from
the input row, so a rejects table needs no to_vec(). Filed as an API gap, it
turned out to be a documentation gap — verified by compiling the borrowing form
against 0.1.0 before writing the issue up.

Breaking

JobError::UnknownTable gained a names field, so an out-of-range write
reports output table 9 does not exist; this job has 2 output table(s): events, rejects rather than a bare index. Add .. to any exhaustive pattern.

JobWriter::write and write_raw now take impl Into<TableId>. This is
source-compatible: write(0, &row) still resolves.

Verified

202 tests and 19 doctests, clippy clean under -D warnings. On a live cluster:
the pilot's 888 input rows give 877 kept / 11 quarantined and 143 sessions across
60 users, matching an independent Python recomputation of the whole expectation;
cat still reproduces its input byte for byte; wordcount still matches a
hand-computed result.

v0.1.0 — YSON codec and MapReduce job runtime

First release. Write YTsaurus MapReduce workers in Rust instead of C++.

[dependencies]
ytsaurus-job = "0.1"

Crate	Version
ytsaurus-job	0.1.0	job runtime — docs
ytsaurus-yson	0.1.0	YSON codec — docs

A YTsaurus job is an ordinary executable: it reads rows from fd 0 and writes
output tables to fds 1, 4, 7… in binary YSON. There is no official Rust SDK, so
this provides the stack — and the guide takes you from
an empty file to a running operation.

What it does

• Streams. The reader holds one buffer no matter how large the input is —
  2 GB flows through at 1.9 MiB peak RSS.
• Zero-copy rows. Decode into &str / &[u8] straight out of the read
  buffer; borrowed decoding measures ~15 % faster than owned.
• Byte-exact pass-through. Row::raw() returns the original bytes, so an
  identity job reproduces its input exactly.
• Control records. table_index, row_index and range_index are applied
  and surfaced per row; key_switch becomes per-key iterators for reduce.
• Multi-table output. One descriptor per table (3k + 1), or a single
  stream with <table_index=N># switch records.
• Static workers. scripts/build-worker.sh produces fully static
  x86_64-unknown-linux-musl binaries, and works on macOS without a
  cross-toolchain.

Downloads

The attached binaries are the example workers from this release, built exactly
as scripts/build-worker.sh builds them — fully static
x86_64-unknown-linux-musl, stripped, ready to hand to yt map --local-file.
Verify with shasum -a 256 -c SHA256SUMS.

• cat-x86_64-unknown-linux-musl — identity map; copies input to output unchanged
• wordcount-x86_64-unknown-linux-musl — wordcount map / wordcount reduce

They are examples, not the library. To use the library, depend on the crates above.

Verified against a real cluster

Not just against a reading of the specification — a local YTsaurus in Docker ran:

• the identity map, output table byte-identical to the input (309 688 bytes);
• table switching across two input and two output tables;
• a wordcount map-reduce matching a hand-computed result.

The offline end-to-end fixtures are captured from that cluster, so CI keeps a
meaningful signal without Docker.

158 tests. Fuzzing ran 6.5 M iterations across both YSON formats without a crash.

Upstream

ytsaurus-yson is a fork of ss123she/yson-rs
@ ba2044c, taken under Apache-2.0 (upstream offers MIT OR Apache-2.0). Nearly
all of the codec is the original author's work.

Vendoring it surfaced three defects, all fixed here and listed in the
changelog:

1. An infinite loop on a stray / in text input. Input as short as /a
   never returned. It allocates nothing, so no memory watchdog catches it.
2. Non-UTF-8 map keys were rejected, though the DOM stores keys as Vec<u8>.
3. Non-UTF-8 attribute names were silently replaced with "".

The last two matter because YTsaurus strings and attribute names are arbitrary
byte strings, not text.

Known limitations

Documented in the codec README. The two that
bite most: maps round-trip as values rather than bytes (BTreeMap sorts keys),
and decoding into String fails on non-UTF-8 columns — use serde_bytes.

The public API has not had an independent review yet; expect it to move before
1.0.