v0.6.1 - Documented unsafe, tracked lockfile, and overflow-checked tests

v0.6.1 - Documented unsafe, tracked lockfile, and overflow-checked tests

TL;DR

Hardening release with no API or behavior change. Every unsafe block in rypipe-core now carries a SAFETY note and the crate denies undocumented ones, Cargo.lock is tracked for reproducible resolution and audits, and the core test suite runs in release mode under -C overflow-checks=on.

Changed

• Every unsafe block is documented, and undocumented ones fail the build. #![deny(clippy::undocumented_unsafe_blocks)] is enabled in rypipe-core, and the SIMD mask helpers and their callers, the counting allocator, the memory map, and the post-validation UTF-8 borrow each carry a SAFETY (or # Safety) note describing its invariant.
• Cargo.lock is tracked. CI and downstream audits resolve the exact dependency set instead of re-resolving on every build; Dependabot keeps it current.

CI

• Core tests run under overflow checks. A release-mode cargo test -p rypipe-core with -C overflow-checks=on fails on integer overflow that the normal release profile would wrap silently.

Full commit log: v0.6.0...v0.6.1

v0.6.0 - Single-pass bounded streaming, fast-path primitives, and an integrations page

v0.6.0 - Single-pass bounded streaming, fast-path primitives, and an integrations page

TL;DR

Bounded streaming is now a single pass: BoundedExecutor spills finished RecordBatches from one TableBuilder instead of parsing each pre-split chunk into its own builder and merging them, which removes a second full copy of all column data. Two fast-path primitives land: multi-delimiter BlockMasks with a quote-aware next_unquoted, and a ColumnarSink::put_str_at string path. The docs gain an ecosystem overview page. Delivered rows, schema-union behavior, StreamStats, and the memory-budget contract are unchanged.

Features

• Single-pass bounded streaming (SpillSink). BoundedExecutor's run_stream, run_bytes_stream, their _with_stats variants and the consumer variants parse the whole input through one TableBuilder and spill finished RecordBatches to the consumer through the new SpillSink, replacing the previous chunk parse plus merge::extend. On the xmlstreamer CDATA feed this moved single-thread bounded streaming from about 0.6x to about 1.0x of a raw quick-xml scan; flat feeds from about 1.0x to about 1.2x. Delivered rows, schema-union behavior, StreamStats, budget accounting, and the memory-budget contract are unchanged.
• ColumnarSink::should_stop. A default-false hook a parser can poll between records to abort the current parse early. Bounded single-pass sinks raise it when the consumer has gone away, so a dropped streaming iterator stops the worker promptly instead of finishing the whole input.
• ColumnarSink::put_str_at string fast path. Push a known &str into a known slot without the Value round trip; TableBuilder writes straight into the column, falling back when a strict memory budget or a filter row buffer is active.
• Multi-delimiter BlockMasks and quote-aware next_unquoted. Every delimiter's bitmask is computed from one 64-byte load, and next_unquoted derives an inside-quotes mask with a SWAR prefix-XOR, so delimiters inside quoted values are skipped with no per-byte state machine.

Changed

• Mapped streaming parses in place. run_mapped_stream reads the memory-mapped input directly instead of dropping the mapping and re-reading each chunk through a reusable buffer.
• Oversize-regime batching preserved. When the budget cannot hold a single row the spilling sink keeps the MIN_ROWS_OVERSIZE_BATCH floor and does not flush early on capacity, so per-row batches cannot explode consumer cost.

Documentation

• Ecosystem overview page. A new integrations page lists adapters and links them from the documentation navigation.

Full commit log: v0.5.1...v0.6.0

v0.5.1 - rypipe-python crate README and packaging metadata

TL;DR

Patch release that adds the missing README.md to the rypipe-python crate and corrects stale metadata in the rypipe-core README. It is also the first release where rypipe-python publishes to crates.io through trusted publishing.

Bug Fixes

• rypipe-python crate README. The crate shipped without a README.md, so its crates.io page reported a missing file. Added the README plus the manifest readme, homepage, documentation, keywords, and categories fields.
• Stale rypipe-core README references. The dependency example now reads 0.5, and the pinned Arrow version was corrected from =59.2.0 to =59.3.0.

CI

• rypipe-python trusted publishing. rypipe-python now publishes to crates.io through OIDC trusted publishing, like rypipe-core; this is the first release to use it.

Full commit log: v0.5.0...v0.5.1

v0.5.0 - Streaming parsers, stage profiling, and chunk-size control

TL;DR

Incremental streaming parsers: a feature-gated StreamingRecordParser plus RecordStream surface that lets an adapter consume arbitrarily chunked, non-seekable feeds and emit adapter-defined records one at a time, with a ParseDiagnostics channel for recoverable damage. A stage profiler reports per-stage timing across workers from Rust and Python. An opt-in min_chunk_bytes floor lets adapters tune split planning for heterogeneous CPUs. Two executor fixes: bounded batch count under sub-row budgets, and a duplicate leading split point on tiny inputs. Adapter crates can now depend on rypipe-python from crates.io.

Features

• Incremental streaming parsers (streaming-parser feature). StreamingRecordParser and RecordStream let an adapter consume arbitrarily chunked input (including non-seekable feeds). The engine owns the buffer, compaction, EOF handshake, and diagnostics, while the adapter keeps format-specific framing. Parsers emit an adapter-defined Record through a callback instead of only ColumnarSink field values, so records can carry raw bytes or per-record metadata (replaced/false markers, collisions); parse_available receives eof so a parser can flush or report truncation. ParseDiagnostics gives adapters a channel for recoverable events (malformed record, truncated input) without failing the feed. This surface was extracted from the xmlstreamer adapter, whose streaming engine previously lived outside rypipe: the driver, buffer compaction, EOF handshake, and diagnostics channel move into core, while the adapter keeps its format-specific scanner (section policy, delimiter validation, item isolation). No existing API changes; the modules are behind the streaming-parser feature.
• Collecting diagnostics sink. CollectingDiagnostics implements ParseDiagnostics, collecting warnings and counters behind a has_events() cheap poll and a take_events() drain, so streaming drivers do not allocate a list per record. ParseDiagnostics gains a default has_events(). The callback signature and the cheap-poll-and-drain shape were taken from the xmlstreamer adapter's diagnostics sink (has_logs()), promoted into core so every streaming adapter gets the same low-overhead path.
• Stage profiler. rypipe_core::profiling records per-stage timing (split, validate, parse, export, merge) across all workers; disabled by default (a relaxed atomic load per stage). Exposed to Python as rypipe.reset_stage_profile(), rypipe.disable_stage_profile(), and rypipe.stage_profile(), so adapter authors can see where a read spends its time without a system profiler.
• Chunk-size floor knob. ExecutionPlan gains an opt-in min_chunk_bytes (and with_min_chunk_bytes), with a matching min_chunk_bytes plan kwarg in the Python bindings and adapter readers. It overrides the default 2 MiB split-planning floor so adapters can tune the chunk count for heterogeneous CPUs and small/large files; the default is unchanged. Engines call the new Splitter::find_split_points_with, which defers to an adapter's find_split_points at the default floor. Requested by the xmlstreamer adapter while tuning a laptop CPU with 2 performance cores and 8 efficiency cores; xmlstreamer now forwards the kwarg through every read entry point.

Bug Fixes

• Duplicate leading split point on tiny inputs. When bytes.len() was smaller than the requested chunk count, every nominal offset collapsed to 0 and the prepended leading split point duplicated it, violating the documented split contract (sorted, unique, first 0, last bytes.len()). Zero candidates are now dropped before the prepend.
• Unbounded batch count under budgets smaller than one row. A budget below a single input row collapsed rows_per_batch to 1, so the bounded executor emitted one batch per row and consumers that concatenate batches (such as arrow::ConcatenateTables) degraded quadratically (10 MB at a 1 KB budget took about 12 s versus 0.4 s unbounded). The oversize regime now floors rows_per_batch at 64, skips the early flush, and only emits on the row-count condition; the same workload runs in about 0.03 s.

Documentation

• Plan and API references updated for min_chunk_bytes. The plan architecture, adapter walkthrough, fusion notes, memory and chunking guide, and the Python and Rust API references document the new floor, its default, and the Python kwarg mapping.

CI

• crates.io publishing for rypipe-python. The release workflow now publishes rypipe-python after rypipe-core, so adapter crates can depend on the bindings as a registry dependency instead of a local path. rypipe-python is still the PyO3 source of the rypipe PyPI wheel, not a separate Python distribution.

Full commit log: v0.4.0...v0.5.0

v0.4.0 - Memory budget observability, stateful parsing, and hardening

TL;DR

Memory budget observability: a new public budget module with StreamStats peak-memory reporting on streaming runs. Strict memory budget contract across the executors, mapped to Python MemoryError. Declarative stateful parsing for multi-line records. A broad hardening wave: decompression bomb protection, regex and allocation caps, bounds checks, and a supply chain audit. Docs fully overhauled, with a new integrations section and example adapters.

Features

• budget module: BudgetPartition, BudgetLedger, StreamStats. New public module deriving per-component limits from a MemoryBudget (with proof tests that the worst case sums within budget), a capacity-accurate ledger of live engine bytes, and a per-run statistics struct. BoundedExecutor gains run_stream_with_stats and run_bytes_stream_with_stats, reporting batches, rows, oversize_batches (single record larger than the batch target), and peak_tracked_bytes. Sizing and enforcement are unchanged; the accounting is observational.
• Memory budget contract. MemoryBudget.with_strict(true) turns the budget into a hard limit: capacity-accurate per-builder checks (TableBuilder::set_memory_budget) error with Error::Memory { used, limit } when an allowance is exceeded. Parallel streaming derives per-worker chunk size and slot allowances from the budget.
• Memory errors surface as Python MemoryError. The engine memory error maps to MemoryError in the bindings instead of a generic exception.
• Declarative stateful parsing. Splitter gains RecordBoundary, continuation_char, and comment_prefixes for multi-line record formats (line continuations, comment lines), plus find_next_record_boundary and is_continued_newline helpers.
• _rypipe.errors module. Typed exception module for the Python bindings.
• Fuzz targets and seed corpora. cargo-fuzz targets with documented seeds and CI smoke campaigns.
• Expression API string transforms. strip, lstrip, rstrip, lower, upper, replace, and length on col() predicates, fused into the Rust parse loop.
• Fused is_type, is_null, contains ops, and partial CastTypes pushdown. More filters stay in the Rust parse loop; non-pushable casts are skipped during fusion instead of forcing a full Python pass.
• engine_probe example and benchmark harness update. Probe binary for engine internals, plus recorded memory and boundary benchmark results.

Security

• Decompression bomb protection. 1 GiB limit on decompressed output via LimitReader; oversize compressed inputs are rejected mid-stream.
• Regex size limit. Compiled regex patterns capped at 1 MiB to reduce ReDoS risk.
• Allocation caps. In/NotIn value lists capped at 100,000 elements, StrColumn::with_capacity capped at 1M rows, Decimal128 scale capped at 38.
• Replace filter empty-old rejection. Replace with an empty old string is rejected to prevent per-row memory amplification.
• Supply chain audit. cargo-deny advisory, license, and ban checks in CI; all GitHub Actions pinned to full commit SHAs.
• Path and arithmetic hardening. Path canonicalization before file reopen (TOCTOU), overflow-safe chunk split point calculation.

Bug Fixes

• Filter comparisons on unparseable values. Typed comparisons now fail instead of silently comparing as 0; division by zero in arithmetic filters returns false.
• Dictionary robustness. Merge paths use from_utf8_lossy to preserve corrupt data; HashMap lookups and code indexing no longer panic on missing keys or out-of-bounds codes.
• Bounds checks. find_literal, in_skip_region, and dictionary code indexing check ranges instead of panicking.
• Degenerate splitter guards. Bounded and parallel paths guard against empty chunk ranges.
• RowObserver panic safety. Observer hooks in the bounded parse path are wrapped in catch_unwind, matching the parallel path.
• RenameFields collisions. Mapping multiple source fields to the same target name now raises ValueError instead of silently losing data.
• ParquetWriter resource leak. to_parquet closes the writer on write errors via try/finally.
• Poisoned lock recovery. Schema cache and parallel stream locks recover instead of panicking.
• Input validation. threads=0, empty/negative memory strings, and out-of-range plan parameters (auto_dict_threshold, auto_dict_max_size, max_split_chunks) are rejected with clear errors.

Documentation

• Docs overhaul. Reworked tutorial, building-adapters guide, advanced topics, and architecture sections, plus a compact top-level nav with dropdowns.
• New integrations section. Guides for Arrow, DuckDB, Polars, pandas, Parquet, Delta Lake, Iceberg, Spark, direct SQL databases (ADBC and SQLAlchemy), and Excel.
• New examples section. Benchmark pages, an engine probe walkthrough, report fixture, and three example adapters (INI, LDIF, Properties).
• Memory contract documented. What the budget covers, what it does not, and how strict mode behaves.
• Page titles fixed. Front-matter titles across all pages; the documentation homepage no longer shows a code comment as its browser tab title.

CI

• Adapter and template job. Native example adapters are built and tested as wheels, and the cargo-generate template is generated and smoke-tested.
• Linux ARM64 regression job.
• Dependabot updates. GitHub Actions and cargo dependency bumps.

Full commit log: v0.3.1...v0.4.0

v0.3.1 - Test fix and CI test blocker

TL;DR

Fixes flaky schema cache test and adds test blocker to publishing CI workflows.

Bug Fixes

• Flaky schema cache test. Removed global counter assertions from schema_cache_hits_misses_and_plan_changes that were affected by parallel test interference. The test now validates schema discovery correctness without depending on exact cache hit/miss counts.

CI

• Publishing workflows: test blocker. Both PyPI (publishing.yml) and crates.io (publishing-cargo.yml) publishing workflows now run the full test suite as a required gate before building and publishing. A release will not be published if tests fail.

Full commit log: v0.3.0...v0.3.1

v0.3.0 - Expression API, streaming sinks, and observer hooks

Sorry, I fucked up the version numbers (jumped from 0.1.x to 2.0.0). Everything is now corrected to 0.x.

TL;DR

Expression API with composable col() predicates that fuse into the Rust parse loop, observer hooks for per-row instrumentation, strict type checking, bounded-memory streaming across all sinks, and a property-based testing crate for adapter development. Breaking changes: to_dataframe() removed, lambda compiler sunset, ColumnarSink and Pipeline traits expanded.

Features

• Expression API with col() for fusable filter predicates. col("amount") > 100 builds a predicate that fuses into the Rust parse loop, just like keyword-form filters. Supports & (and), | (or), ~ (not) composition, and methods: isin, not_in, startswith, endswith, contains, matches, between, is_null, is_not_null, is_type.
• FilterRowsAny, FilterRowsAll, FilterRowsNot combinators. Build OR, AND, and NOT logic over keyword-form filters. Combinators are fusable and nestable: FilterRowsAny(A, FilterRowsAll(B, FilterRowsNot(C))) expresses A or (B and not C).
• RowObserver trait and Python observer hooks. Per-row instrumentation fired during parsing. The Rust RowObserver trait has methods on_begin_row, on_put_field, on_row_accepted, on_row_rejected, on_chunk_finished. Python passes observer={"on_row_rejected": fn, ...} in plan kwargs, wrapping callables in a PyObserver.
• strict_types mode. strict_types=True on Source.init or ExecutionPlan aborts the read when a non-null value fails to parse into its declared type, instead of silently nulling. Genuine nulls remain null.
• max_split_chunks parameter. Caps the number of bounded-streaming chunks. Defaults to 100,000. Exposed on Source.init and ExecutionPlan.
• MemoryBudget struct and BatchConsumer trait. MemoryBudget::new(bytes) sets a streaming budget. BatchConsumer is a callback trait for streaming reads without batch collection: Pipeline::read_bytes_stream_consumer(bytes, budget, consumer).
• parse_decimal128 support. FieldType::Decimal128 and CastTypes now parse decimal values.
• rypipe-test crate. Property-based testing helpers for adapter development: proptest strategies (arb_field_name, arb_field_value, arb_record, arb_records, arb_malformed_utf8, arb_nested_quotes), fixtures (MALFORMED_UTF8, NESTED_QUOTES, EMPTY_VALUES), and test helpers (parse_test_bytes, assert_batches_equal, KeyValueParser, NewlineSplitter).
• cargo-generate template for adapter scaffolding. cargo generate emiliano-go/rypipe template creates a ready-to-build adapter package.
• memory= kwarg on all sink functions. collect(pipeline, memory="64MiB"), to_pandas(pipeline, memory=...), to_polars(pipeline, memory=...), to_parquet(pipeline, path, memory=...) all support bounded-memory streaming.
• discover_schema() function. Scans a file once and returns column names after applying field_mapping, drop_fields, etc. Avoids per-file discovery overhead.
• Pipeline.to_pandas(), to_polars(), to_parquet() methods. Materialize pipeline results directly without importing sink functions.
• Binary memory units. iter_record_batches and Source.to_pandas/to_polars/to_parquet accept KiB, MiB, GiB, TiB in addition to KB, MB, GB, TB.
• starts_with, ends_with, contains filter operators. Fusable keyword-form operators for string prefix, suffix, and substring matching.
• schema() respects projection. When schema=["Name", "Amount"] is passed, Source.schema() returns ["Name", "Amount"] instead of the full column list.
• Splitter::find_split_points default method. Finds safe split points via nominal offsets, next_record_start, skip-region rejection, dedup, and sort. Adapters rarely need to override.
• FrozenSchema::from_partial_plan and is_exact. from_partial_plan builds a schema from a partial column list. is_exact reports whether the schema matches exactly.
• Schema cache functions. insert_schema_cache, clear_schema_cache, schema_cache_stats, dynamic_window_count, dynamic_window_size, layout_signature for schema discovery optimization.
• Timestamp format variants. FieldType::Timestamp(TimeUnit) supports Second, Millisecond, Microsecond, Nanosecond, plus custom chrono formats: timestamp[ms,format=%Y%m%d %H:%M].
• Predicate composable trait. Python objects with _to_spec() returning a spec dict can be passed as FilterRows(predicate=obj).

Breaking Changes

• to_dataframe() removed. Use to_pandas() instead. The alias existed for backward compatibility with crxml.
• Lambda compiler sunset. The in-tree lambda bytecode compiler is removed. Use col() expressions or keyword-form filters instead; for arbitrary logic, use plain callables (Python fallback, not fusable).
• ColumnarSink trait expanded. 10 new methods: put_row, resolve_raw, resolve_and_put_raw, put_field_resolved, put_field_at, resolve_and_put, needs_value, needs_resolve, wanted_mask, expect_slot.
• Pipeline expanded. 3 new methods: read_bytes_stream_consumer, read_path_stream_consumer, read_path_stream_par.
• ExecutionPlan gained fields. strict_types: bool, max_split_chunks: Option, observer: Option<Arc>.

Bug Fixes

• ArithmeticCompare.cmp_value type. Was documented as f64, corrected to String.
• Length.value type. Was documented as i64, corrected to String.
• Compound filter specs fall back to Python with full plan override. FilterRowsAny/All/Not and unsupported ops (is_null, is_type, regex, comparisons) now correctly apply rename, drop, and cast transformations in the Python fallback path.
• iter_record_batches memory passthrough. Memory strings are parsed to bytes before passing to the Rust extension, which only accepts integer byte counts.
• schema() projection. Source.schema() now returns self._schema when set, respecting the projection.

Full commit log: v0.2.2...v0.3.0

v0.2.2 - Contiguous dictionary storage and data integrity tests

TL;DR

Contiguous dictionary storage reduces per-entry memory overhead by eliminating String allocations. Dictionary values are now stored in a single contiguous byte buffer instead of Vec, enabling zero-copy Arrow export and removing the Vec to SeedDict conversion during unification. Comprehensive data integrity tests added covering all ColumnBuilder variants, compare filters, filter predicate trees, bounded executor, splitter/recordparser edge cases, and thread safety.

Performance

• Dictionary memory: Eliminates 24 bytes String overhead per dictionary entry (pointer + length + capacity).
• Arrow export: Zero-copy wrap of contiguous data+offsets into StringArray (no String-to-&str conversion).
• Unification: No Vec to SeedDict conversion needed; already in contiguous layout.

Testing

• 22 new data integrity tests (52 total) covering:
  • Dictionary: push, codes, nulls, empty, single value, many values, all modes.
  • Column-to-column compare filters.
  • Filter predicate trees (And, Or, Not) edge cases.
  • Bounded executor with different budget sizes.
  • Splitter edge cases (empty, single newline, no newlines, many newlines).
  • RecordParser edge cases (invalid UTF-8, empty lines, partial lines).
  • All ColumnBuilder variants via Pipeline.
  • Schema order with many columns.
  • Combined plans (rename+drop+filter+types+dictionary).
  • Cross-parser consistency (put_field vs resolve_and_put).
  • Mmap vs Owned equivalence.
  • Thread safety (concurrent reads).

Full commit log: v0.2.1...v0.2.2

v0.2.1 - Bug fixes and expanded documentation

TL;DR

Bug fix release addressing the wanted_mask projection short-circuit that truncated rows when field_map or drop_fields was active. macOS ARM build support via cfg-guarded is_x86_feature_detected. Sitemap Content-Type fix for Cloudflare Pages. Expanded adapter documentation with a complete Hello World example, fusion diagram, and error handling guidance. All em dashes replaced with ;:,()-.

Bug Fixes

• wanted_mask projection short-circuit (table_builder.rs): When field_map or drop_fields was non-empty, wanted_mask built its bitmask from field_index (columns seen so far) instead of all expected columns. After processing just the first field, wanted_mask matched row_dirty and row_satisfied() returned true prematurely, causing the scanner to skip remaining fields. Fix: only short-circuit when schema_order is explicitly set.
• is_x86_feature_detected! (scan/mod.rs): Guard with cfg(any(target_arch = "x86", target_arch = "x86_64")) to fix macOS ARM build failure.
• bench_tier return type (bench/mod.rs): Return type (f64, usize) did not match actual return value (f64, f64, usize).
• RowBuffer::new (predicate.rs): Allow dead_code on unused constructor.
• sitemap.xml (docs/_headers): Add Content-Type: application/xml header for Cloudflare Pages.
• Writing Adapters nav (zensical.toml): Fix link to point to writing-adapters/index.md directory, not missing .md file.

Documentation

• Hello World adapter: Complete runnable LogSplitter/LogParser example in writing-adapters/index.md.
• plan_overrides example: Dedicated section showing how adapters receive fused stage kwargs.
• Fusion diagram: Visual ASCII diagram of the fusion pipeline flow in advanced/fusion.md.
• Parallel streaming: Added in-flight management subsection with backpressure and ordering detail in advanced/execution-modes.md.
• Adapter design: Expanded from 120 to 280+ lines with Splitter example, RecordParser detail, push method hierarchy, and error handling guidance.
• Performance page: Trimmed from 178 to 116 lines, deduplicated with advanced pages.
• Python-only path: Added note that adapters require Rust but Python users consume without writing Rust.
• Em dash cleanup: Replaced all --- with ;:,()- across 12 architecture and writing-adapters files.

Full commit log: v0.2.0...v0.2.1

v0.2.0 - Framework rebrand and performance milestone

TL;DR

Framework rebrand and performance milestone. rypipe reframed as a "data ingestion framework" with a full architecture rewrite: streaming pipeline, parallel execution, bounded-memory mode, incremental dictionary unification, projection short-circuit, and SIMD-accelerated scanning. Comprehensive performance profiling harness and documentation overhaul.

Architecture

• Streaming pipeline: 64KB true streaming via BatchConsumer + StreamingBatchIterator with dynamic bytes_used and split_off.
• Parallel streaming: In-flight management with backpressure and ordering.
• Incremental dictionary unification: auto_dict parallel gap closed from 45% to 16%. Per-chunk upgrade in parallel, then O(dict_size) unification.
• row_satisfied projection short-circuit: Scanner byte-jumps to row close when all wanted columns arrive. +64% on drop_half parallel.
• expect_slot layout prediction: memcmp raw bytes instead of full attribute scan + hash lookup. ~25 ns to ~8 ns per field.
• F1 precomputed close_finder: Eliminated per-row Vec+Finder allocation. +10% single-thread.
• F2 fast-path find_attr_value: Single-attribute fast path. +3% single-thread.
• Engine-provided Splitter default: Eliminates bug class that caused TSV and crxml regressions.
• Splitter SIMD optimization: next_row_start() searches for <tag in one memmem::find pass instead of memchr(b'<') (24M iterations to 465k matches). Split phase 40% faster.
• find_special_regions() single-pass: Scans once for b"<!" prefix instead of two separate scans.
• concat_tables() schema fast path: Skips promote_options='default' when schemas already match.

Performance

• Single-thread: ~953 MB/s on 533 MB Crystal Reports XML (up from ~800 MB/s).
• Parallel projected: ~6,879-7,630 MB/s drop_half/rename/schema on 533 MB.
• PrimColumn validity bitpack: Replaced Vec<Option> with bitmap for better cache performance.
• Cow-based Value::Str: Closed filtered-path use-after-free in entity-decoded strings.

Bug Fixes

• Sparse-column KeyError (from 1.2.1): to_arrow() crashed when later rows had fields the first row lacked. Fixed to union all row keys.
• Compare filter docs corrected: Compare filters do not force the merge path; they are applied per-row during parse AND re-applied post-export.
• auto_dict docs corrected: Incremental dict path preserves fast export when schemas consistent.
• wanted_mask projection short-circuit: Fires incorrectly when field_map or drop_fields were active.
• is_x86_feature_detected! guard: Added cfg guard for non-x86 platforms.
• bench_tier return type: Corrected from (f64, usize) to (f64, f64, usize).
• All clippy -D warnings resolved.

Documentation

• 134 audit findings fixed across 5 rounds (rypipe + crxml).
• Architecture docs expanded from 1,058 to 2,000 lines.
• Writing-adapters reworked into 7 dedicated pages.
• All em dashes replaced with ;:,()-.

Dependencies

• rypipe-core upgraded to 0.2.0 (was 0.1.1).
• Arrow pinned at =55.2.0 (FFI_ArrowSchema not stable across minor versions).

Full commit log: v0.1.1...v0.2.0