fix: bump duckdb 1.10501->1.10505 to unbreak Windows MSVC CI build - #1
Merged
Conversation
The bundled DuckDB in duckdb 1.10501.0 (engine 1.5.1) vendors an old fmt
whose format.h:326 declares
`checked_ptr = stdext::checked_array_iterator<T*>`. MSVC 14.51 (VS 2026,
now on GitHub windows-latest) *removed* stdext::checked_array_iterator, so
the C++ compile of libduckdb-sys fails with C2653 ('stdext' is not a
class or namespace name) plus a cascade of syntax errors — reddening the
`test (windows-latest)` job on `cargo test --all-features` (duckdb-output).
macOS/Linux (clang) are unaffected.
DuckDB dropped that usage in PR #23261/#23239, shipped in engine 1.5.5 =
crate 1.10505.0. Verified against the bundled sources: 1.10501.0's
format.h contains the removed symbol at line 326 (matching the CI error
line); 1.10505.0's format.h has zero references. The `^1.4.4` requirement
already allowed the fix, but the committed lock held CI at the broken
1.10501.0 — bump the lock and pin the manifest floor at the fixed version.
Only the app's optional duckdb-output feature is affected; the published
blazehash-core library has no duckdb dependency.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
….5 bump The duckdb 1.10501->1.10505 bump changed the vetted versions (duckdb/ libduckdb-sys 1.10505.0, ureq 3.3.0) and pulled two new transitives (ureq-proto 0.6.0, utf8-zero 0.8.1), leaving 6 deps unvetted: comfy-table, duckdb, libduckdb-sys, ureq, ureq-proto, utf8-zero. These were previously covered by version-PINNED exemptions, which the bump invalidated. Replace them with version-AGNOSTIC publisher trust (cargo vet trust): duckdb/libduckdb-sys -> Mytherin (DuckDB creator), ureq/ureq-proto/utf8-zero -> algesten (ureq author), comfy-table -> Nukesor (comfy-table author). This is the bam-forensic pattern and survives future version bumps, so the release/CI vet gate stops re-breaking on every duckdb/ureq bump. cargo vet auto-removed the now superseded pinned exemptions (comfy-table 7.2.2, duckdb/libduckdb-sys 1.10501.0, ureq 2.12.1). imports.lock caches the publisher metadata. cargo vet --locked: Vetting Succeeded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…sitive) The advisory DB now flags instant 0.1.13 as unmaintained. This is an unmaintained-ONLY advisory (not a vulnerability, no reachable defect in a hashing workload), reachable in the audited graph solely via the GPU feature: wgpu 22 -> parking_lot 0.11.2 -> instant. parking_lot is held on the 0.11 line by sled (opendal/remote stack, already excluded from this audit). No safe upgrade exists (instant is abandoned; parking_lot 0.12 dropped it), and the only true removals -- a wgpu major bump (rewrites the GPU backend) or updating the excluded opendal/sled remote tree -- are out of scope for this Windows-CI fix. Narrow, annotated ignore per the fleet adler2-law exception for an unmaintained-not-vulnerable transitive with no successor; grouped with the existing wgpu-transitive paste ignore and carrying its removal condition. Pre-existing on main (identical crates), surfaced by the advisory DB, not introduced by the duckdb bump. cargo deny check: advisories ok, bans ok, licenses ok, sources ok. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Windows `test` job OOMs once the C++ build succeeds: cargo test compiles/links ~100 integration-test binaries, each statically linking ~800 crates plus bundled C++ (duckdb/rocksdb/wasmtime/yara-x). With full `-C debuginfo=2` this exhausts the runner's RAM — rustc aborts with `handle_alloc_error` (compiling watch_tests) and link.exe dies with `LNK1102: out of memory` (linking merkle_tests). Set CARGO_PROFILE_DEV_DEBUG=0 / CARGO_PROFILE_TEST_DEBUG=0 on the Windows leg only, which strips the CodeView debug tables driving the memory blowup. Windows-scoped, mirroring the existing Linux mold mitigation for the same OOM class; macOS, Linux, coverage, MSRV and local dev are untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
blazehash was the fleet outlier with NO CI caching, so every run recompiled the bundled C++ engines (DuckDB/RocksDB/wasmtime/arrow/yara-x) from source across 3 OSes and ~9 feature-test passes — the ~1h runtimes and the Windows OOM both trace to this. Add Swatinem/rust-cache (the fleet standard, used by issen + most repos) to the three heavy jobs (test / coverage / check-msrv). It caches ~/.cargo + the cleaned target/ (incl. the bundled C++ objects in target/*/build/*/out), so steady-state runs skip the C++ recompile. Auto-keys per-OS/arch + rustc + Cargo.lock; the test matrix's native legs each get their own key (the OS/arch-not---target gotcha doesn't apply here — no cross-compile). deny/vet skipped (metadata-only, ~25s). Uses the GHA cache (now 20 GB). First run is cold; the payoff lands on subsequent runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The duckdb OOM fix let the Windows test job finally reach the clippy step, exposing a
pre-existing clippy::collapsible_match in walk_windows_mft.rs (Windows-cfg'd, so
macOS never linted it). Converted 'MFT_ATTR_DATA => { if cond { … } }' to a match
guard 'MFT_ATTR_DATA if cond => { … }' — behavior-preserving given the '_ => {}'
fallthrough. Verified via cross-target windows-gnu clippy (--lib --no-default-features
to skip the bundled C++). ubuntu-latest on the prior run was a fail-fast cancellation
of this same failure, not a separate issue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Linux test job hung ~6h on 'cargo test --features hashdb'. Diagnosis across configs showed no runtime reproduction on aarch64 in any mode; the leading hypothesis is an x86_64 mold link-time deadlock during the hashdb relink over a shared target/. Stop using mold for the Linux test leg (link with the default ld) and add timeout-minutes: 45 so any future hang fails fast with a fresh, readable log instead of burning the 6h ceiling. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ostic) The x86_64-linux hashdb-only leg still hangs after dropping mold, while the linux --all-features leg (which includes hashdb), the Windows hashdb leg, and aarch64 all pass. GitHub discards logs for a job cancelled by timeout-minutes, so this run makes the hang self-diagnosing: - Split the Linux hashdb step into link (--no-run) and run phases, each wrapped in 'timeout --verbose' so a hang fails that STEP with a preserved log and the last Compiling/Running line names the culprit (relink vs runtime test). - Raise the job cap to 90min for headroom on the cold bundled-C++ build. - Add rust-cache cache-on-failure so a later-step failure still persists the already-built all-features artifacts, breaking the cold-cache-every-run trap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause, proven from a preserved CI link log: linking the ~100 giant bundled-C++ test binaries with full debuginfo (debuginfo=2) exhausts the 7 GB runner's memory. mold DEADLOCKS on it (the original 6 h hang); the default GNU ld SIGBUSes instead — 'collect2: fatal error: ld terminated with signal 7 [Bus error], core dumped' on 4 hashdb-feature test binaries. The linux --all-features leg, Windows, and aarch64 all pass, which is why it looked feature/arch-specific; it is really a linker-memory ceiling hit by the hashdb-only relink after the all-features build. Fix (two complementary, root-cause reductions of linker memory): - Drop dev/test debuginfo on Linux too (was Windows-only) — removes the DWARF tables that drive the blowup; this is the same cure already proven for the Windows LNK1102 OOM in this repo. - Link with lld instead of mold — memory-efficient on huge binaries, without mold's deadlock. Also kept from the diagnosis: timeout-minutes on the test job (fail fast, not 6 h) and rust-cache cache-on-failure (persist the all-features artifacts so a later-step failure doesn't force a cold rebuild next run). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
test (windows-latest)CI job onmainis RED, while release-plz keeps publishingblazehash-core— an honest-gate gap this PR closes.The failure is not a Rust test failure. It is a native C++ build break of the bundled DuckDB (
libduckdb-sys) under GitHub's newest MSVC toolchain (Visual Studio 18 / MSVC 14.51.36231, now onwindows-latest):(followed by a cascade of C2988/C2143/C7525 parse errors). Triggered by
cargo test --all-features→duckdb-output→duckdb/bundled. macOS/Linux (clang) are unaffected.Root cause
MSVC 14.51 removed the long-deprecated non-standard
stdext::checked_array_iterator. Thefmtvendored inside DuckDB 1.5.1 (crateduckdb 1.10501.0, the version the committed lock pinned) still declareschecked_ptr = stdext::checked_array_iterator<T*>atformat.h:326— the exact line MSVC now rejects.Fix
DuckDB dropped that usage in PR #23261 / #23239, shipped in engine 1.5.5 = crate
1.10505.0. The^1.4.4requirement already permitted it; the committedCargo.lockheld CI at the broken1.10501.0. This bumps the lock and pins the manifest floor at the fixed version.Verification (on macOS — windows-latest cannot be run locally)
format.hfrom both crates'duckdb.tar.gz—1.10501.0containsstdext::checked_array_iteratorat line 326 (matches the CI error line exactly);1.10505.0contains zero references. The bump deletes the offending line.cargo check --features duckdb-outputon macOS compileslibduckdb-sys 1.10505.0and type-checksduckdb 1.10505.0cleanly (1.5.1→1.5.5 is a patch bump).cargo fmt --checkpasses.Only the app's optional
duckdb-outputfeature is affected; the publishedblazehash-corelibrary has no duckdb dependency. Lock churn is confined to DuckDB's transitive tree.🤖 Generated with Claude Code