[CI] add LLVM IR bloat regression check#1083
Conversation
Add an llvm-lines job to CI that compares LLVM IR line counts between the current branch and main to detect monomorphization bloat regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a CI job that compares LLVM IR line counts for diskann-benchmark between the PR branch and main to detect monomorphization bloat, plus a standalone bash script that produces a markdown delta report from two cargo llvm-lines outputs.
Changes:
- New
llvm-linesjob in.github/workflows/ci.yml(gated byneeds: basics) that installscargo-llvm-lines, runs it on both checkouts, computes overall growth, and writes a delta report to$GITHUB_STEP_SUMMARY. - New
.github/scripts/compare-llvm-lines.shthat parses twocargo llvm-linesoutputs and emits a per-function markdown comparison table sorted by delta.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
.github/workflows/ci.yml |
Adds an advisory llvm-lines job that builds baseline/current cargo llvm-lines output, checks total growth against LLVM_LINES_GROWTH_THRESHOLD, and surfaces a report to the job summary. |
.github/scripts/compare-llvm-lines.sh |
New bash script that diffs two cargo llvm-lines outputs and prints a sorted markdown table of per-function regressions. |
Comments suppressed due to low confidence (2)
.github/workflows/ci.yml:529
- With GitHub Actions' default shell options (
bash --noprofile --norc -eo pipefail, whichdefaults.run.shell: bashat the top of this file selects),cargo llvm-lines ... | head -100 | tee ...is very likely to fail the step. Onceheadhas read 100 lines it closes its stdin;cargo llvm-linesthen dies with SIGPIPE (exit code 141), andpipefailpropagates that failure even thoughheadandteesucceeded. Capture the full output first (e.g.cargo llvm-lines ... | tee ../current-llvm-lines.txt) and only applyheadlater when feeding the report, or disable pipefail locally for this command.
The same issue applies to the head -100 on line 541.
run: cargo llvm-lines --package diskann-benchmark --release | head -100 | tee ../baseline-llvm-lines.txt
- name: Generate current LLVM lines
working-directory: diskann_rust
run: cargo llvm-lines --package diskann-benchmark --release | head -100 | tee ../current-llvm-lines.txt
.github/workflows/ci.yml:539
- If
baseline-llvm-lines.txtdoes not contain a(TOTAL)line (e.g.cargo llvm-linesoutput format changed, the build was warning/error-filtered, or the SIGPIPE issue above produces an empty file),baseline_totalwill be the empty string and$(( ... / baseline_total ))will fail with a bash arithmetic error, taking down the step with a confusing message.compare-llvm-lines.shdefends againstbaseline_total -eq 0on line 23 but this inline computation does not. Consider either delegating the threshold check to the script or adding the same guard here (and defaulting empty values to 0).
baseline_total=$(awk '/\(TOTAL\)/{print $1}' baseline-llvm-lines.txt)
current_total=$(awk '/\(TOTAL\)/{print $1}' current-llvm-lines.txt)
growth=$(( (current_total - baseline_total) * 100 / baseline_total ))
if [ "$growth" -gt "$LLVM_LINES_GROWTH_THRESHOLD" ]; then
echo "::warning::LLVM IR grew ${growth}% ($baseline_total → $current_total lines)"
fi
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1083 +/- ##
==========================================
- Coverage 89.46% 89.45% -0.01%
==========================================
Files 459 458 -1
Lines 85482 85398 -84
==========================================
- Hits 76475 76392 -83
+ Misses 9007 9006 -1
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
# DiskANN v0.53.0 Release Notes ## Breaking Changes An AI generated, human reviewed list of changes is summarized below. ### Paged search overhauled — channel-based API ([#1078](#1078)) `PagedSearchState` and its `'static`-bound pause/resume model have been replaced with an async, channel-based interface. The recommended way to drive paged search is now via a `tokio::sync::mpsc` channel, with the searcher embedded in an otherwise-`'static` future. See the [rendered RFC](https://github.com/microsoft/DiskANN/blob/main/rfcs/01078-paged-search.md) for the new shape. Callers wired against `PagedSearchState` must migrate to the channel API. Users of paged search via `wrapped_async::DiskANNIndex` that know their inner futures will never suspend can use the new `wrapped_async::DiskANNIndex::paged_search_no_await`; this will efficiently run paged searches with minimal synchronization overhead. ### `DiskANNIndex::flat_search` removed ([#1076](#1076)) `DiskANNIndex::flat_search` and the `IdIterator` trait have been removed from the `diskann` crate. Equivalent functionality lives on the new inherent method `DiskIndexSearcher::flat_search` in `diskann-disk`. This unblocks the experimental directions in #1067 and #983. ```rust // Before diskann_index.flat_search(query, ...)?; // After disk_index_searcher.flat_search(query, ...).await?; ``` ### `DiskIndexSearcher::flat_search` now batched ([#1097](#1097)) The new `DiskIndexSearcher::flat_search` uses the bulk `pq_distances` path instead of one-vector-at-a-time `Accessor::build_query_computer` + `evaluate_similarity`. Downstream behavior is equivalent but tighter resource bounds apply. ### `centroid` removed from PQ interfaces ([#1010](#1010)) The dataset-centroid argument has been removed from `FixedChunkPQTable` construction, `populate`, and most other PQ APIs. The shift only ever worked for L2 distance and was silently ignored for inner-product / cosine, so passing it was a footgun. When an L2 shift is required, fold it into the PQ pivots instead (the library now does this internally). ```rust // Before let table = FixedChunkPQTable::new(.., centroid, ..); // After — drop the centroid argument let table = FixedChunkPQTable::new(.., ..); ``` ### Flat search interface ([#983](#983)) A new `flat` module in `diskann` adds a provider-agnostic brute-force search surface, mirroring the shape of graph search. Backends implement a single trait, `DistancesUnordered<C>` (in `flat/strategy.rs`), which fuses iteration and distance computation, allowing any backend (in-memory, quantized, disk, remote) to plug into a shared algorithm. See the [rendered RFC](https://github.com/microsoft/DiskANN/blob/main/rfcs/00983-flat-search.md). This is additive but is the new canonical surface — direct ad-hoc flat-search call sites should migrate. ### `bf_tree` extracted into `diskann-bftree` crate ([#1020](#1020)) The bf_tree provider has been moved out of `diskann-providers` (previously at `diskann-providers/src/model/graph/provider/async_/bf_tree/`) into a new standalone `diskann-bftree` crate. Along with the move: - Switched from PQ to spherical quantization. - Dropped dependencies on `DeletionCheck`, `AsDeletionCheck`, and `RemoveDeletedIdsAndCopy`. - Simplified generics. Consumers must update their `Cargo.toml` to depend on `diskann-bftree` and update import paths. ### `direct_distance_impl` and `inner_product_raw` re-exposed ([#1081](#1081)) `direct_distance_impl` (free function) and `FixedChunkPQTable::inner_product_raw` are `pub` again after being privatized in #1044. Restored to unblock a downstream user. Not breaking in the typical direction — this restores previously available API surface. ### MinMax `recompress` takes a grid-scale parameter ([#1109](#1109)) The MinMax `recompress` API now accepts a grid-scale parameter. ## New Features - SIMD-optimized L2-squared norm ([#1107](#1107)) - Significantly faster bitmap computation ([#1099](#1099)) - Large speedup on the bitmap construction path used by filtered search. - LLVM IR bloat regression check in CI ([#1083](#1083)) - CI now flags regressions in generated LLVM IR size, helping catch unintended monomorphization blow-ups. - Recall computation fix for under-k groundtruth ([#1069](#1069)) ## Merged PRs * Revise README for DiskANN3 by @harsha-simhadri in #1046 * [CI] Try to fix publishing step by @hildebrandmw in #1057 * [benchmark] Remove `DispatchRule` by @hildebrandmw in #1064 * [benchmark] Automatic Input Registration by @hildebrandmw in #1066 * Remove centroid from most PQ interfaces by @hildebrandmw in #1010 * [diskann/disk] Remove `flat_search` from `DiskANNIndex` by @hildebrandmw in #1076 * macos build and miri check to nightly by @harsha-simhadri in #1058 * [API] Make some methods public again by @hildebrandmw in #1081 * [benchmark] Simply `Inputs` more by @hildebrandmw in #1077 * Turn on stack protection for the diskann-garnet NuGet build by @jackmoffitt in #1082 * Fix options for diskann-garnet nuget pipeline by @jackmoffitt in #1091 * [CI] add LLVM IR bloat regression check by @arazumov in #1083 * Bump openssl from 0.10.79 to 0.10.80 by @dependabot[bot] in #1093 * [Disk CI benchmarks] Use 1ES.Pool=diskann-github by @arazumov in #869 * Fix recall computation for fewer than k groundtruth results by @magdalendobson in #1069 * bf_tree migration away from diskann-providers by @JordanMaples in #1020 * [RFC/diskann] Overhaul paged search by @hildebrandmw in #1078 * Remove unsafe code from compute_vec_l2sq by @arazumov in #1094 * Remove direct accessor call in `diskann-garnet` by @hildebrandmw in #1098 * Refactor `DiskIndexSearcher::flat_search` to use batching by @hildebrandmw in #1097 * [flat index] Flat Search Interface by @arkrishn94 in #983 * migrating multi-hop tests from diskann-providers to diskann by @JordanMaples in #928 * Significantly speed up bitmap computation by @magdalendobson in #1099 * `compute_vecs_l2sq`: Replace scalar L2 Squared norm with SIMD-optimized FastL2NormSquared by @arazumov in #1107 * [minmax] Add grid scaling to recompress API by @arkrishn94 in #1109 **Full Changelog**: v0.52.0...v0.53.0
This pull request introduces an automated check in the CI workflow to monitor and report on LLVM IR code size growth, helping to detect regressions in monomorphization cost. The main changes include adding a new CI job and a supporting shell script for comparing LLVM IR line counts between the current branch and the baseline.
CI workflow enhancements:
llvm-linesjob to.github/workflows/ci.ymlthat compares the LLVM IR line counts of the current branch against themainbranch usingcargo-llvm-lines. This job is not required for PR merges but provides regression reports and enforces a configurable growth threshold.Supporting scripts:
.github/scripts/compare-llvm-lines.sh, a Bash script that parses and compares the output ofcargo-llvm-linesfor the baseline and current builds, generating a markdown-formatted regression report for easier review.