Skip to content

fix(index): log vector training sampling progress#6724

Merged
Xuanwo merged 1 commit into
lance-format:mainfrom
hfutatzhanghb:codex-vector-training-sampling-progress-logs
May 11, 2026
Merged

fix(index): log vector training sampling progress#6724
Xuanwo merged 1 commit into
lance-format:mainfrom
hfutatzhanghb:codex-vector-training-sampling-progress-logs

Conversation

@hfutatzhanghb
Copy link
Copy Markdown
Contributor

@hfutatzhanghb hfutatzhanghb commented May 10, 2026

Summary

Adds progress logging while loading vector training data so long IVF/PQ training data sampling does not appear stuck.

Logs per-batch or per-chunk progress for nullable FSL sampling, fallback multivector sampling, uniform FSL sampling, and fragment-limited sampling.

Testing

  • cargo fmt --all
  • cargo check -p lance --tests

Examples

Just like below:

image

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added the bug Something isn't working label May 10, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 10, 2026

Codecov Report

❌ Patch coverage is 87.87879% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance/src/index/vector/utils.rs 87.87% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@hfutatzhanghb
Copy link
Copy Markdown
Contributor Author

@Xuanwo Hi, could you please review this pr when have free time? Thanks.

Copy link
Copy Markdown
Collaborator

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice change, thank you!

@Xuanwo Xuanwo merged commit 68aec51 into lance-format:main May 11, 2026
30 checks passed
@hfutatzhanghb hfutatzhanghb deleted the codex-vector-training-sampling-progress-logs branch May 11, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants