You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Breaking
Python — record.leader is now a property, not a method (use record.leader,
not record.leader()). Leader constructs from a 24-character string or as Leader() with properties assigned; the old field keyword arguments
(Leader(record_type=..., ...)) are removed.
Python — Record.get_fields() returns control fields in record order (was
fixed ascending-tag order).
Rust — MarcError and the PipelineError, RecoveryMode, ValidationLevel, EncodingAnalysis, RdfFormat, and IndicatorValidation enums are #[non_exhaustive]: exhaustive matches need a wildcard arm, and external
construction goes through the new MarcError constructors.
Rust — from_path constructors now return MarcReader<BufReader<File>> (was MarcReader<File>).
Rust — recovery::try_recover_record is removed; truncated-record salvage now
runs through the ISO 2709 skeleton walk.
Build — MSRV is now Rust 1.88; both crates use edition 2024.
Added
Community files: issue and pull request templates, and a security policy (SECURITY.md)
pointing at GitHub private advisories.
parse_record_from_bytes: parse one complete MARC record from in-memory bytes with
no reader I/O and no per-record copies. The Python MARCReader read path now uses it,
collapsing the former chain of per-record buffer copies between the source and the
parser to a single pymarc-compatibility stash (current_chunk).
parse_record_from_shared_bytes: parse from a buffer the caller already holds behind an Arc, without taking ownership. The Python MARCReader uses it so the current_chunk
stash shares one allocation with the parser instead of cloning, and current_chunk is now
read lazily — iterating without inspecting it copies no record bytes into Python.
Criterion benches for the serialization formats that had no CI perf signal — CSV, MODS,
Dublin Core, BIBFRAME (Turtle), and MARC-in-JSON (both directions for MODS and
MARC-in-JSON) — plus a single-thread parser-pool bench that tracks the
producer-consumer pipeline's per-record cost deterministically under simulation.
The pymarc parity oracle now executes in CI and check.sh instead of silently skipping:
a new oracle extra pins pymarc in uv.lock (Dependabot adjudicates behavior changes on
bump PRs), and the oracle extends beyond iteration shape to value-level comparisons —
title, format_field(), value(), and as_marc() byte-equality over the 1k corpus.
MarcError::metadata() returns an ErrorMetadata snapshot of every structured field an
error carries. The per-field accessors, Display/detailed() rendering, JSON output, and
the Python exception mapping all read from this single per-variant table now; rendered and
serialized output is unchanged.
MarcError (enum and variants) and the grower enums (PipelineError, RecoveryMode, ValidationLevel, EncodingAnalysis, RdfFormat, IndicatorValidation) are now #[non_exhaustive], so adding a variant or field is no longer a breaking change. Downstream matches need a wildcard arm; MarcError construction outside the crate goes through new
public constructors (invalid_field, truncated_record, record_length_invalid, fatal_reader_error) and with_* positional setters.
copy.deepcopy() support for Record, Field, and Leader: deep copies are fully
independent of the original (and of any record), while copy.copy() stays shallow, matching
pymarc. Deep-copying a live field handle yields a detached snapshot of its current data.
A reproducible three-way benchmark (scripts/benchmark_comparison.py, with examples/benchmark_native.rs for the native column) and published figures
(docs/benchmarks/): the Python wrapper reads ~7× pymarc per record, ~30× on the
parallel bulk path, ~1.6× extract, and ~6.5× roundtrip, against a native-Rust
ceiling of ~9×/~35× on the reference host. Replaces the prior "needs
re-measurement" hedge.
Changed
The parallel parse helpers (parse_batch_parallel / parse_batch_parallel_limited) and the
blocking ProducerConsumerPipeline reads now release the GIL during the parallel parse and the
channel wait, so other Python threads keep running instead of being blocked for the duration.
(The parallel helpers now take ownership of the buffer, required for sound GIL release.)
Record.get_fields() with no arguments is faster: it made one PyO3 call per control tag
(nine) plus one per data field, and now fetches all control fields in a single call.
Control fields come back in record order rather than fixed ascending-tag order (a
difference only for records whose control fields are stored out of order; record order
matches pymarc).
Iterating a MARCReader is faster: wrapping each parsed record no longer builds a
throwaway inner _Record and two _Leader objects only to discard them, lifting
per-record read throughput (path and bytes inputs alike). Output is unchanged.
Writing ISO 2709 records (MARCWriter and the authority/holdings writers) allocates less:
each directory entry's length and start-position digits are written straight into the
output buffer instead of through a per-field format!.
MARCXML deserialization and 880-linkage parsing no longer recompile their regexes on
every call (hoisted to LazyLock statics). Parsing MARCXML records one at a time — e.g. parse_xml_to_array in a loop — is substantially faster; batch/whole-document parsing
already amortized the cost and is unchanged.
Reading from a Python file-like object (MARCReader(open(path, "rb")), BytesIO, etc.)
now reads the source in 256 KiB chunks and slices records out in Rust, instead of two file.read() calls plus a getattr("read") per record. The read method is bound once
and each chunk is borrowed via PyBytes rather than copied. Output is unchanged; file-path
and bytes inputs were already chunked and are unaffected.
Cargo.lock is now committed, so CI, wheel builds, and local checkouts resolve the
same dependency versions; Dependabot manages version bumps as reviewable diffs.
Removed unused dependencies (bytes, nom, encoding_rs, csv, bindings-crate tempfile) and demoted example-only anyhow/flate2 to dev-dependencies, shrinking
the published crate's dependency tree. check.sh now runs cargo machete to keep it so.
The version is now declared once, in [workspace.package]: both crates inherit it and pyproject.toml reads it via dynamic = ["version"]. Shared dependencies and rust-version are also workspace-inherited, so they cannot drift between crates.
BIBFRAME/RDF support is now behind the default-on bibframe cargo feature. Default
builds and the Python wheel are unchanged; --no-default-features drops the
oxrdf/oxrdfio dependency tree for MARC-only users.
Record parsing no longer copies every record's bytes into the error-diagnostics
buffer: the parse buffer is shared by refcount instead, deleting a per-record
alloc+memcpy that every read path paid so that the under-1% of records that error
could render hex dumps. Error bytes_near output is unchanged.
Tuned the release profile: fat LTO, codegen-units = 1, and debuginfo stripping.
Wheels get ~2-3% higher read throughput and a 21% smaller extension binary.
The Python leader surface now matches pymarc: record.leader is a property (was a
method) and assignable from a Leader or 24-character string; Leader renders and
compares as its 24-character MARC 21 string (str(), len(), == with str, repr)
and constructs from one (Leader('00136nam a2200061 4500')). The oracle now
compares leaders against pymarc value-for-value. Rust Leader gains a Display impl.
Migrated both crates to Rust edition 2024 and raised the MSRV from 1.87 to 1.88
(let-chains require 1.88). The workspace now uses resolver v3, so dependency
resolution respects rust-version instead of breaking the MSRV build on upgrades.
Releases now publish a source distribution and macOS universal2 wheels, so Intel Macs
and platforms outside the wheel matrix can install mrrc (previously "no matching
distribution"). PyPI metadata completed: real author, PEP 639 license expression, Typing :: Typed classifier, and project URLs (homepage, docs, changelog, issues).
File-path readers now buffer their reads (64 KiB): Python MARCReader(path) (and the
authority/holdings readers) and Rust from_path previously issued two-plus read(2)
syscalls per record. Rust from_path constructors now return MarcReader<BufReader<File>> instead of MarcReader<File>.
Updated pyo3 to 0.29, resolving RUSTSEC-2026-0176 (out-of-bounds read in nth/ nth_back on PyList/PyTuple iterators in pyo3 ≤0.28).
Updated quick-xml to 0.40. MARCXML text and attribute decoding now applies XML 1.0
end-of-line and attribute-value normalization explicitly (previously the XML 1.1 EOL
set, which additionally folded NEL/LSEP — those are no longer normalized, matching
MARCXML's XML 1.0 reality).
Corrected the declared Rust MSRV from 1.71 to 1.87 — the floor the dependency tree
already required, so 1.71 never actually built; a new CI job now verifies the
workspace compiles on the declared MSRV so the claim cannot drift again. The accurate
MSRV unlocked clippy-driven cleanups: the MARC-8 tables now use std::sync::LazyLock
(dropping the lazy_static dependency) and I/O error construction uses std::io::Error::other.
Truncated records no longer allocate or zero-pad a buffer of the leader's claimed
length: the reader grows the record buffer as bytes arrive (8 KiB steps) and lenient
salvage parses the short body directly, so allocation is bounded by actual input size
instead of a claimed maximum-length record on a small stub.
Removed
Deleted four dead manual-profiling bench harnesses (profiling_harness, detailed_profiling, rayon_profiling, rayon_file_io_profiling). They produced no
CI or CodSpeed signal, referenced documents that no longer exist, and one wrote
scratch files to /tmp. The bench target now contains only criterion benches.
The placeholder mrrc binary (src/main.rs, a banner print) and the orphaned tests/create_sample_data.py (fixtures come from scripts/generate_benchmark_fixtures.py).
The docs.rs front page no longer embeds the README (whose CI badges and repo-relative
links 404 off GitHub); it now renders the curated crate-level docs instead.
recovery::try_recover_record: truncated-record salvage now runs through the shared
ISO 2709 skeleton's clamped directory walk, which slices fields at their data-area
offsets — intact fields ahead of the truncation point are now recovered instead of
dropped. Non-digit directory bytes on the salvage walk keep their InvalidField (E106)
shape but now also count against the reader's recovered-error cap.
Fixed
Record is now iterable: for field in record yields each field as a wrapped mrrc.Field
(control and data, in record order), matching pymarc. It previously raised TypeError.
Record.remove_field_at() now returns the mrrc.Field wrapper (with __getitem__ and the
other pymarc conveniences) instead of the bare _mrrc.Field extension type.
copy.copy() of a Record no longer raises RecursionError: the wrapper's attribute
delegation now stops cleanly when the inner record is absent during copy reconstruction.
Corrected Python documentation errors that broke copy-pasted code: the README and mrrc.read() examples called record.title() (it is a property); docs showed imports
and reader constructors that do not exist in Python (MarcError, MARCReader.from_path(), .with_source()); the API reference stated the wrong recovery_mode default ("strict"; it is "permissive") and omitted the validation_level and max_errors keyword arguments.
Corrected documentation drift: the Rust API reference showed parse_batch_parallel with
the wrong arity and a stale 5-variant MarcError table; parse_batch_parallel / parse_batch_parallel_limited were missing from the Python API reference; the release
procedure understated the wheel count and used retired issue-tracker commands.
