Skip to content

Fix unsound Delta unsigned-widening cast#8195

Merged
joseph-isaacs merged 1 commit into
developfrom
claude/blissful-ramanujan-7S6HL
Jun 1, 2026
Merged

Fix unsound Delta unsigned-widening cast#8195
joseph-isaacs merged 1 commit into
developfrom
claude/blissful-ramanujan-7S6HL

Conversation

@joseph-isaacs
Copy link
Copy Markdown
Contributor

@joseph-isaacs joseph-isaacs commented Jun 1, 2026

Fixes #8193.

Problem

The Delta cast kernel reused the encoded bases/deltas across primitive-type changes, which is unsound. Delta stores deltas as wrapping_sub at the source bit width and decodes with wrapping_add at the array's bit width, so widening corrupted every value after a delta that wrapped at the source width:

[200u8, 50, 75, 10, 255] -> cast u32 -> [200, 306, 331, 522, 767]   (expected [200, 50, 75, 10, 255])

Two further latent bugs in the same kernel:

  • u32 -> f32 passed the guard, entered the reuse path, then errored in Delta::try_new (float dtype) instead of falling back.
  • Casting a nullable Delta to non-nullable at the same ptype eagerly rebuilt a non-nullable DeltaArray, silently discarding real nulls.

Fix

Reuse the encoded components only for a nullability change at the same primitive type (adding/keeping nullability, where validity lives in the deltas child). Every ptype change, and dropping nullability, defers to the generic decompress-and-re-encode path, which decodes correctly and validates nulls.

Nothing more is recoverable from metadata: widening would need a wrap/monotonicity flag (value range is insufficient — [0,200] and [200,0] share min/max but only one is safe to widen); narrowing/signedness need the value range, which can't be derived from the components.

Tests

  • test_cast_delta_unsigned_widening_wraps — widening corruption regression.
  • test_cast_delta_same_width_float_targetu32 -> f32 falls back instead of erroring.
  • test_cast_delta_add_nullability — same-ptype add-nullability fast path.
  • test_cast_delta_nullability_preserves_nulls — null positions round-trip.
  • test_cast_delta_drop_nullability_with_nulls_errors — dropping nullability with real nulls errors.
  • existing test_cast_delta_conformance, test_cast_delta_u8_to_u32, test_cast_delta_nullable.

Checks

  • cargo test -p vortex-fastlanes — 291 + 3 doctests pass (delta::compute::cast 11/11).
  • cargo clippy -p vortex-fastlanes --all-targets --all-features — clean.
  • cargo +nightly fmt -p vortex-fastlanes -- --check — clean.

Rebased onto the latest develop and squashed to a single commit.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Jun 1, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 1 improved benchmark
❌ 2 regressed benchmarks
✅ 1272 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime cuda/bitpacked_u8/unpack/3bw[100M] 300.2 µs 349.9 µs -14.19%
Simulation chunked_varbinview_canonical_into[(100, 100)] 273.1 µs 307.8 µs -11.28%
Simulation chunked_varbinview_opt_canonical_into[(1000, 10)] 225.3 µs 188 µs +19.87%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/blissful-ramanujan-7S6HL (cf09b33) with develop (fa2c35a)

Open in CodSpeed

Delta stores deltas as `wrapping_sub` at the source bit width and
reconstructs values with `wrapping_add` at the array's bit width, so
reusing the encoded bases and deltas across a primitive-type change is
unsound. The previous kernel did this for any unsigned widening, which
corrupted every value after a delta that wrapped at the source width
(e.g. [200u8, 50] cast to u32 decoded as [200, 306] instead of [200, 50]).

It also had two latent bugs: a u32 -> f32 cast entered the reuse path and
then errored in `Delta::try_new` instead of falling back, and casting a
nullable DeltaArray to non-nullable at the same ptype silently discarded
real nulls.

The kernel now reuses the encoded components only for a nullability change
at the same primitive type (adding or keeping nullability, where validity
lives in the deltas child). Every ptype change, and dropping nullability,
defers to the generic decompress-and-re-encode path, which decodes
correctly and validates nulls.

Adds regression tests for the widening corruption, the u32 -> f32
fallback, adding nullability, preserving nulls across a same-ptype
nullability cast, and erroring when dropping nullability with real nulls.

Fixes #8193.

Signed-off-by: Claude <noreply@anthropic.com>
@joseph-isaacs joseph-isaacs force-pushed the claude/blissful-ramanujan-7S6HL branch from c32a13f to cf09b33 Compare June 1, 2026 15:07
@joseph-isaacs joseph-isaacs added the changelog/fix A bug fix label Jun 1, 2026 — with Claude
@joseph-isaacs joseph-isaacs enabled auto-merge (squash) June 1, 2026 15:24
@joseph-isaacs joseph-isaacs requested a review from robert3005 June 1, 2026 15:24
@joseph-isaacs joseph-isaacs merged commit ad01aea into develop Jun 1, 2026
65 of 68 checks passed
@joseph-isaacs joseph-isaacs deleted the claude/blissful-ramanujan-7S6HL branch June 1, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unsound Delta cast: unsigned-widening corrupts wrapped deltas

3 participants