feat: support system-installed OpenBLAS via VSAG_USE_SYSTEM_DEPS by wxyucs · Pull Request #2117 · antgroup/vsag

wxyucs · 2026-05-28T09:37:31Z

Summary

Introduce a small system-dependency policy framework (VSAG_USE_SYSTEM_DEPS = AUTO / ON / OFF, with per-dependency VSAG_USE_SYSTEM_<DEP> overrides) in cmake/VSAGOptions.cmake and cmake/VSAGHelpers.cmake.
Wire OpenBLAS only into the new framework. Additional dependencies will be migrated incrementally in follow-up PRs (see Roadmap: reduce bundled third-party dependencies in core VSAG #1949 Phase 5 roadmap).
Keep the existing USE_SYSTEM_OPENBLAS=ON switch working with its previous "try system, fall back to bundled" semantics — no behavioural regression for current users.

This is a focused replacement for #1958, which tried to migrate all bundled dependencies at once and accumulated 36 files / +638 LoC of churn. Here only 4 files change.

Resolution order for OpenBLAS

When the resolved policy is not OFF, the build resolves OpenBLAS in this order and stops at the first hit:

an OpenBLAS::OpenBLAS target already defined by a parent project,
find_package(OpenBLAS CONFIG),
a manual search for libopenblas plus cblas.h / lapacke.h under common system prefixes,
the bundled ExternalProject_Add build (AUTO / OFF only; ON fails configuration here).

Test plan

Local CMake configure + build smoke tests (no system OpenBLAS installed on the dev box):

cmake -S . -B build -DENABLE_LIBAIO=OFF — AUTO policy falls back to the bundled OpenBLAS, cmake --build build --target vsag succeeds end-to-end.
cmake -S . -B build -DENABLE_LIBAIO=OFF -DVSAG_USE_SYSTEM_OPENBLAS=ON — fails configuration with the expected actionable error from vsag_fail_missing_system_dep.
cmake -S . -B build -DENABLE_LIBAIO=OFF -DVSAG_USE_SYSTEM_OPENBLAS=OFF — forces the bundled build.
cmake -S . -B build -DENABLE_LIBAIO=OFF -DVSAG_USE_SYSTEM_DEPS=OFF — global override forces bundled build.
cmake -S . -B build -DENABLE_LIBAIO=OFF -DUSE_SYSTEM_OPENBLAS=ON (legacy switch) — falls back to bundled (no FATAL), matching pre-change behaviour.

Notes

No CI workflow changes; a follow-up PR can add a System Deps Smoke job once we have more than one dependency to exercise.
The find_package(OpenBLAS CONFIG) / OpenBLAS::OpenBLAS paths are mirrored from the previously CI-validated logic in feat: support external third-party dependencies #1958, with the reviewer fixes folded in (no double-defined imported targets, no FORCE on cache variables).

Closes: #2116
Refs: #1949

🤖 Generated with Claude Code

Introduce a small policy framework (VSAG_USE_SYSTEM_DEPS=AUTO/ON/OFF with per-dependency VSAG_USE_SYSTEM_<DEP> overrides) and wire only OpenBLAS into it. Additional dependencies will be migrated in follow-up changes. For OpenBLAS the resolution order is: 1. an existing OpenBLAS::OpenBLAS target, 2. find_package(OpenBLAS CONFIG), 3. a manual search for libopenblas + cblas.h / lapacke.h, 4. the bundled ExternalProject build (AUTO / OFF only). The legacy USE_SYSTEM_OPENBLAS option keeps its previous "try system, fall back to bundled" semantics for backward compatibility. Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: ClaudeCode:claude-opus-4.7

mergify · 2026-05-28T09:38:18Z

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Require kind label

Wonderful, this rule succeeded.

label~=^kind/

🟢 Require version label

Wonderful, this rule succeeded.

label~=^version/

🟢 Require linked issue for feature/bug PRs

Wonderful, this rule succeeded.

body~=(?im)(?:^|[\s\-\*])(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s+(?:#\d+|[\w.\-]+/[\w.\-]+#\d+|https?://github\.com/[\w.\-]+/[\w.\-]+/issues/\d+)

gemini-code-assist

Code Review

This pull request introduces a unified mechanism for managing system third-party dependencies in VSAG, replacing the legacy USE_SYSTEM_OPENBLAS option with a more flexible VSAG_USE_SYSTEM_DEPS policy (supporting AUTO, ON, and OFF modes) and per-dependency overrides like VSAG_USE_SYSTEM_OPENBLAS. It updates the OpenBLAS configuration to check for pre-existing targets, use find_package in CONFIG mode, or perform a manual search. The review feedback highlights that using NO_DEFAULT_PATH in the manual search for OpenBLAS and LAPACKE libraries restricts CMake from searching standard system locations on other architectures or respecting user-specified paths (like CMAKE_PREFIX_PATH). Removing NO_DEFAULT_PATH is recommended to improve portability.

Copilot

Pull request overview

Adds a small CMake policy framework to optionally reuse system-provided third-party dependencies (starting with OpenBLAS), while keeping the legacy USE_SYSTEM_OPENBLAS option working with its prior “try system, fall back to bundled” behavior.

Changes:

Introduce VSAG_USE_SYSTEM_DEPS (AUTO/ON/OFF) plus per-dependency overrides like VSAG_USE_SYSTEM_OPENBLAS.
Update OpenBLAS resolution to prefer an existing OpenBLAS::OpenBLAS target / find_package(OpenBLAS CONFIG) before falling back to a manual search, then bundled build when allowed.
Document the new system-deps options and the OpenBLAS resolution order.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
extern/openblas/openblas.cmake	Implements policy-driven system OpenBLAS discovery and fallback to bundled build.
DEVELOPMENT.md	Documents new system dependency policy options and OpenBLAS resolution order.
cmake/VSAGOptions.cmake	Adds cache options for global and OpenBLAS-specific system-deps policies.
cmake/VSAGHelpers.cmake	Adds helper functions to resolve/validate policies and fail with actionable messages.

- Drop NO_DEFAULT_PATH from the manual OpenBLAS / LAPACKE search so CMake also consults CMAKE_PREFIX_PATH and standard multiarch locations on less common architectures, after VSAG's curated path list (gemini-code-assist). - Re-publish BLAS_LIBRARIES as a cache STRING so superbuild / downstream consumers see it on both BLAS backends, matching MKL's behaviour (Copilot). - Clarify in DEVELOPMENT.md that the legacy USE_SYSTEM_OPENBLAS switch maps to AUTO ("try system, fall back to bundled"), not ON (Copilot). Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: ClaudeCode:claude-opus-4.7

Repoint every x86 CI job at the appropriate variant tag introduced in the previous PR and turn on `-DUSE_SYSTEM_OPENBLAS=ON` for the OpenBLAS-backed jobs so they reuse the `libopenblas-dev` package shipped in `vsaglib/vsag:ci-x86-openblas` instead of rebuilding OpenBLAS from source on every run. Image routing: * `pr-ci.yml`, `coverage.yml`, `lint.yml`, `check_compatibility.yml`, `performance.yml`, `generate_old_version_index.yml` → `vsaglib/vsag:ci-x86-openblas`. * `asan_build_and_test.yml`, `tsan_build_and_test.yml`, `asan_with_simd_option.yml`, and the SIMD / TSAN matrices in `daily_test.yml` → `vsaglib/vsag:ci-x86-mkl` (these jobs already build with `VSAG_ENABLE_INTEL_MKL=ON`). * Daily x86 ASan in `daily_test.yml` uses the `-openblas` image but intentionally does NOT set `USE_SYSTEM_OPENBLAS`, so it keeps exercising `ExternalProject_Add(openblas)` end-to-end — that is now the dedicated guard for the from-source build path. `USE_SYSTEM_OPENBLAS=ON` is forwarded via `EXTRA_DEFINED` (existing Makefile plumbing) rather than introducing a new `make` flag, so this change does not touch `Makefile` and does not collide with the in-flight `VSAG_USE_SYSTEM_OPENBLAS` CMake option being added in antgroup#2117. CircleCI's `prepare_env_and_create_swap_file` step now also installs `liblapacke-dev` so CMake's `find_path(LAPACKE_INCLUDE NAMES lapacke.h)` probe succeeds against the system OpenBLAS. Release artifacts (`scripts/release/dist.sh`, `docker/Dockerfile.dist_*x86`) and `make pyvsag` are intentionally not changed: they continue to build OpenBLAS from source and statically link it. Refs: antgroup#2118 Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: OpenCode:claude-opus-4.7

Repoint every x86 CI job at the BLAS-variant image introduced in the previous PR and pass `-DUSE_SYSTEM_OPENBLAS=ON` (via the existing `EXTRA_DEFINED` Makefile knob) to the OpenBLAS-backed jobs so they reuse the pre-installed `libopenblas-dev` instead of rebuilding OpenBLAS from source on every run. Routing: * `pr-ci`, `coverage`, `lint`, `check_compatibility`, `performance`, `generate_old_version_index` → `ci-x86-openblas`. All except `generate_old_version_index` also set `USE_SYSTEM_OPENBLAS=ON`; the latter builds historical release tags whose CMake does not understand the flag, so the source-build path is kept there. * `asan_build_and_test`, `tsan_build_and_test`, `asan_with_simd_option`, daily SIMD / TSAN → `ci-x86-mkl` (these jobs already build with `VSAG_ENABLE_INTEL_MKL=ON`). * Daily x86 ASan → `ci-x86-openblas` but deliberately leaves `USE_SYSTEM_OPENBLAS` unset, so `ExternalProject_Add(openblas)` keeps getting exercised end-to-end as the dedicated guard for the from-source build path. CircleCI's `prepare_env_and_create_swap_file` step also installs `liblapacke-dev` so CMake's `find_path(LAPACKE_INCLUDE NAMES lapacke.h)` probe succeeds against the system OpenBLAS. Forwarding `USE_SYSTEM_OPENBLAS` via `EXTRA_DEFINED` keeps `Makefile` untouched and avoids colliding with the in-flight `VSAG_USE_SYSTEM_OPENBLAS` CMake option in antgroup#2117. Release artifacts (`scripts/release/dist.sh`, `docker/Dockerfile.dist_*x86`) and `make pyvsag` are unchanged and still build OpenBLAS from source. Refs: antgroup#2118 Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: OpenCode:claude-opus-4.7

LHT129

LGTM

Repoint every x86 CI job at the BLAS-variant image introduced in the previous PR and pass `-DUSE_SYSTEM_OPENBLAS=ON` (via the existing `EXTRA_DEFINED` Makefile knob) to the OpenBLAS-backed jobs so they reuse the pre-installed `libopenblas-dev` instead of rebuilding OpenBLAS from source on every run. Routing: * `pr-ci`, `coverage`, `lint`, `check_compatibility`, `performance`, `generate_old_version_index` → `ci-x86-openblas`. All except `generate_old_version_index` also set `USE_SYSTEM_OPENBLAS=ON`; the latter builds historical release tags whose CMake does not understand the flag, so the source-build path is kept there. * `asan_build_and_test`, `tsan_build_and_test`, `asan_with_simd_option`, daily SIMD / TSAN → `ci-x86-mkl` (these jobs already build with `VSAG_ENABLE_INTEL_MKL=ON`). * Daily x86 ASan → `ci-x86-openblas` but deliberately leaves `USE_SYSTEM_OPENBLAS` unset, so `ExternalProject_Add(openblas)` keeps getting exercised end-to-end as the dedicated guard for the from-source build path. CircleCI's `prepare_env_and_create_swap_file` step also installs `liblapacke-dev` so CMake's `find_path(LAPACKE_INCLUDE NAMES lapacke.h)` probe succeeds against the system OpenBLAS. Forwarding `USE_SYSTEM_OPENBLAS` via `EXTRA_DEFINED` keeps `Makefile` untouched and avoids colliding with the in-flight `VSAG_USE_SYSTEM_OPENBLAS` CMake option in antgroup#2117. Release artifacts (`scripts/release/dist.sh`, `docker/Dockerfile.dist_*x86`) and `make pyvsag` are unchanged and still build OpenBLAS from source. Refs: antgroup#2118 Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: OpenCode:claude-opus-4.7

wxyucs requested review from LHT129 and inabao as code owners May 28, 2026 09:37

Copilot AI review requested due to automatic review settings May 28, 2026 09:37

wxyucs requested a review from jiaweizone as a code owner May 28, 2026 09:37

wxyucs added kind/feature New feature version/0.18 labels May 28, 2026

pull-request-size Bot added the size/L label May 28, 2026

Copilot started reviewing on behalf of wxyucs May 28, 2026 09:37 View session

wxyucs mentioned this pull request May 28, 2026

feat: support external third-party dependencies #1958

Draft

mergify Bot added the module/thirdparty label May 28, 2026

gemini-code-assist Bot reviewed May 28, 2026

View reviewed changes

Comment thread extern/openblas/openblas.cmake Outdated

Comment thread extern/openblas/openblas.cmake Outdated

Copilot AI reviewed May 28, 2026

View reviewed changes

Comment thread DEVELOPMENT.md Outdated

Comment thread extern/openblas/openblas.cmake

Comment thread extern/openblas/openblas.cmake

This was referenced May 28, 2026

[improve](build): split CI image by BLAS backend and reuse system OpenBLAS in fast-path jobs #2118

Open

ci: switch fast-path jobs to system OpenBLAS and split-image tags #2120

Open

wxyucs added version/1.0 and removed version/0.18 labels May 29, 2026

LHT129 approved these changes May 29, 2026

View reviewed changes

wxyucs merged commit 2aa8c82 into main May 29, 2026
20 of 21 checks passed

wxyucs deleted the support-system-deps-openblas branch May 29, 2026 05:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support system-installed OpenBLAS via VSAG_USE_SYSTEM_DEPS#2117

feat: support system-installed OpenBLAS via VSAG_USE_SYSTEM_DEPS#2117
wxyucs merged 2 commits into
mainfrom
support-system-deps-openblas

wxyucs commented May 28, 2026

Uh oh!

mergify Bot commented May 28, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LHT129 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wxyucs commented May 28, 2026

Summary

Resolution order for OpenBLAS

Test plan

Notes

Uh oh!

mergify Bot commented May 28, 2026

Merge Protections

🟢 Require kind label

🟢 Require version label

🟢 Require linked issue for feature/bug PRs

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LHT129 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants