Error while vendoring tantivy-binding dependencies to build Milvus

[lucasmlp]

Hello,

I noticed that two versions of rust-stemmers were added as dependency of tantivy-binding in the Milvus 2.6.36 release, those are:

• 1.2.0 - commit
• 1.3.0 - commit

This broke our offline reproducible build of Milvus because it's not possible to run cargo vendor for the tantivy-binding dependecy:

[2026-10-27 07:47:25] INFO - 2026-10-27T06:47:25.147585Z ERROR obs_service_cargo::cli: err=Custom { kind: Interrupted, error: "error: failed to sync\n\nCaused by:\n  found duplicate version of package `rust-stemmers v1.2.0` vendored from two sources:\n\n  \tsource 1: https://github.com/milvus-io/rust-stemmers.git?rev=3b2545218f76e797baecbf91fa1cbd2c0db60726#3b254521\n  \tsource 2: https://github.com/milvus-io/rust-stemmers.git#3b254521\n" }
[2026-10-27 07:47:25] INFO - 2026-10-27T06:47:25.152912Z ERROR cargo_vendor: error: failed to sync
[2026-10-27 07:47:25] INFO - 
[2026-10-27 07:47:25] INFO - Caused by:
[2026-10-27 07:47:25] INFO -   found duplicate version of package `rust-stemmers v1.2.0` vendored from two sources:
[2026-10-27 07:47:25] INFO - 
[2026-10-27 07:47:25] INFO -   	source 1: https://github.com/milvus-io/rust-stemmers.git?rev=3b2545218f76e797baecbf91fa1cbd2c0db60726#3b254521
[2026-10-27 07:47:25] INFO -   	source 2: https://github.com/milvus-io/rust-stemmers.git#3b254521
[2026-10-27 07:47:25] INFO - 
[2026-10-27 07:47:25] INFO - Error: Custom { kind: Interrupted, error: "error: failed to sync\n\nCaused by:\n  found duplicate version of package `rust-stemmers v1.2.0` vendored from two sources:\n\n  \tsource 1: https://github.com/milvus-io/rust-stemmers.git?rev=3b2545218f76e797baecbf91fa1cbd2c0db60726#3b254521\n  \tsource 2: https://github.com/milvus-io/rust-stemmers.git#3b254521\n" }
[2026-10-27 07:47:25] INFO - Command exited with return code 1

Which version should be used, 1.2.0 or 1.3.0?

Thank you in advance.

[majidninja]

You need both, actually. 1.2.0 and 1.3.0 aren't the conflict: cargo vendor puts them in rust-stemmers-1.2.0 and rust-stemmers-1.3.0 without trouble. The problem is that 1.2.0 is in Cargo.lock twice, from the same commit but with two different source strings:

• tantivy-binding depends on it directly with rev = "3b25452..." → git+...rust-stemmers.git?rev=3b25452...#3b25452...
• the tantivy-5 (0.21.1-fix4) fork depends on it as a plain git dependency → git+...rust-stemmers.git#3b25452...

Cargo treats those as two different packages, and cargo vendor refuses to put two packages with the same name and version into one directory. Tantivy 0.23 uses 1.3.0 (rev = ac4aee9...), which is unrelated.

I reproduced your error on the 3.0 branch (where #54096/#54097 landed). Dropping the rev from the direct dependency fixes it:

-rust-stemmers = { git = "https://github.com/milvus-io/rust-stemmers.git", rev = "3b2545218f76e797baecbf91fa1cbd2c0db60726" }
+rust-stemmers = { git = "https://github.com/milvus-io/rust-stemmers.git" }

Then cargo vendor in internal/core/thirdparty/tantivy/tantivy-binding succeeds. The lockfile change is only the duplicate entry going away. Both users now resolve to the existing git+https://github.com/milvus-io/rust-stemmers.git#3b25452... entry, so it is still exactly the same commit and the analyzer output doesn't change. You end up with rust-stemmers (1.2.0 @ 3b25452) and rust-stemmers-1.3.0 in vendor/.

For an offline build you could carry that as a small patch. Upstream, the cleaner fix is to make the direct dependency and the tantivy fork specify the git source the same way (both with rev or both without), so the lockfile only has one 1.2.0 entry.

[lucasmlp]

Thank you, @majidninja.

I was able to vendor the Cargo dependencies for Tantivy after creating a patch for Cargo.toml.