test_memtable_generation_zero_panics SIGABRT on macOS with Apple ld from Xcode 15.0 CLT

[summaryzb]

Symptom

Running cargo test --workspace on macOS, the entire lance crate test process is killed by SIGABRT when executing test_memtable_generation_zero_panics:

test dataset::mem_wal::scanner::data_source::tests::test_memtable_generation_zero_panics - should panic ...
libunwind: stepWithCompactEncoding - invalid compact unwind encoding
error: test failed, to rerun pass `-p lance --lib`

Caused by:
  process didn't exit successfully (signal: 6, SIGABRT: process abort signal)

The lance crate has 1406 tests in total. Because the SIGABRT kills the whole test binary, ~1127 tests after this point are never executed, severely impacting local development iteration on macOS.

All other crates (lance-core, lance-encoding, lance-file, lance-index, etc., totaling 1623 tests) pass without issue.

Trigger Conditions

This crash requires two conditions to be met simultaneously:

1. Large test binary: The lance crate test binary is ~429 MB (__TEXT segment ~226 MB, __unwind_info ~4.6 MB). Apple ld from Xcode 15.0.x generates invalid compact unwind encoding entries for binaries of this size. Smaller crate binaries (e.g., lance-encoding ~64 MB, lance-core ~12 MB) are not affected.

2. Stack unwinding must occur: Only #[should_panic] tests guarantee a panic, which triggers catch_unwind → libunwind stack unwinding → reads the corrupted __unwind_info entries → abort(). Normal passing tests never panic, so they never trigger unwinding and are unaffected.

test_memtable_generation_zero_panics is the only #[should_panic] test in the lance crate, making it the sole point of failure. Other crates have their own #[should_panic] tests (lance-encoding, lance-core, lance-table, lance-file), but their smaller binary sizes avoid the linker bug entirely.

Environment

Component	Version
macOS	26.0 Tahoe (also reproducible on Sonoma 14.x)
Arch	arm64 (Apple Silicon)
Dev tools	Command Line Tools (not full Xcode)
Apple Clang	15.0.0 (clang-1500.0.40.1)
Apple ld	1015.7 (LLVM 15.0.0, Aug 2023)
Rust	1.91.0

Root Cause

This is a known Apple linker bug — the ld shipped with Xcode 15.0.x / CLT 15.0 generates invalid compact unwind encoding entries in the __unwind_info Mach-O section for large binaries. When libunwind encounters these entries during stack unwinding (triggered by #[should_panic]'s catch_unwind), it prints stepWithCompactEncoding - invalid compact unwind encoding and calls abort().

Upstream references:

• Rust: Panic unwinding fails on macOS targets when using the Xcode 15 beta rust-lang/rust#113783
• Lance CI fix (selecting Xcode 15.4): commit 0dbfc7cf / PR ci: fix stepWithCompactEncoding error on MacOS #2531

The project CI already works around this by using sudo xcode-select -s /Applications/Xcode_15.4.app in .github/workflows/rust.yml.

Proposed Solutions

Option A: Skip the test on macOS (recommended)

Add #[cfg_attr(target_os = "macos", ignore)] to avoid the SIGABRT on any macOS dev machine regardless of toolchain version. The test still runs on Linux CI where the bug does not exist.

// Ignored on macOS: the lance test binary (~429MB) triggers an Apple ld bug
// that generates invalid compact unwind encoding, causing #[should_panic]
// tests to SIGABRT. See: https://github.com/rust-lang/rust/issues/113783
#[test]
#[should_panic(expected = "MemTable generation must be >= 1")]
#[cfg_attr(target_os = "macos", ignore = "may SIGABRT due to Apple ld compact unwind bug")]
fn test_memtable_generation_zero_panics() {
    LsmGeneration::memtable(0);
}

Option B: Replace assert! with Result return

Change LsmGeneration::memtable() to return Result<Self, LanceError> instead of panicking, then test with a normal assert!(result.is_err()). This avoids unwinding entirely but requires an API change.

Option C: Developer documentation

Document that local macOS development requires Xcode 15.4+ or newer Command Line Tools. This is already the CI policy but not enforced locally.