Skip to content

perf(scan): reduce system column pipeline overhead - #8717

Merged
jiaoew1991 merged 2 commits into
lance-format:jiaoew/stack-system-column-base-v4from
jiaoew1991:perf/system-column-scan-pipeline
Aug 25, 2026
Merged

perf(scan): reduce system column pipeline overhead#8717
jiaoew1991 merged 2 commits into
lance-format:jiaoew/stack-system-column-base-v4from
jiaoew1991:perf/system-column-scan-pipeline

Conversation

@jiaoew1991

@jiaoew1991 jiaoew1991 commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Superseded

This PR was merged only into its temporary integration base while the dependency stack was being refreshed; it was not merged into main.

Please review #8747, which carries the focused system-column pipeline diff on the refreshed dependency base.

lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-changes Latest Gatekeeper recommendation requests changes. label Aug 24, 2026
@jiaoew1991
jiaoew1991 force-pushed the perf/system-column-scan-pipeline branch from ce92c1d to 23a1f6c Compare August 24, 2026 12:10
@jiaoew1991
jiaoew1991 force-pushed the jiaoew/stack-system-column-base branch from e004e20 to 90c4cdb Compare August 24, 2026 12:10
@lance-gatekeeper lance-gatekeeper Bot removed the K-changes Latest Gatekeeper recommendation requests changes. label Aug 24, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. and removed K-approved Latest Gatekeeper recommendation permits acceptance. labels Aug 24, 2026
@jiaoew1991
jiaoew1991 force-pushed the perf/system-column-scan-pipeline branch from 23a1f6c to 9cdd0dc Compare August 24, 2026 13:53
@github-actions github-actions Bot added A-python Python bindings A-deps Dependency updates A-encoding Encoding, IO, file reader/writer labels Aug 24, 2026
@jiaoew1991
jiaoew1991 changed the base branch from jiaoew/stack-system-column-base to jiaoew/stack-system-column-base-v2 August 24, 2026 13:54
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 24, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 24, 2026
@jiaoew1991
jiaoew1991 force-pushed the perf/system-column-scan-pipeline branch from 9cdd0dc to 4432222 Compare August 24, 2026 14:33
@github-actions github-actions Bot added the A-java Java bindings + JNI label Aug 24, 2026
@jiaoew1991
jiaoew1991 changed the base branch from jiaoew/stack-system-column-base-v2 to jiaoew/stack-system-column-base-v3 August 24, 2026 14:33
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 24, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 24, 2026
@jiaoew1991
jiaoew1991 force-pushed the perf/system-column-scan-pipeline branch from 4432222 to f47b8b9 Compare August 25, 2026 01:09
@github-actions github-actions Bot added A-index Vector index, linalg, tokenizer A-ci CI / build workflows labels Aug 25, 2026
@jiaoew1991
jiaoew1991 changed the base branch from jiaoew/stack-system-column-base-v3 to jiaoew/stack-system-column-base-v4 August 25, 2026 01:09
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 25, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The base rebase leaves the reviewed implementation patch-equivalent. Bounded read-ahead and consolidated system-column assembly continue to preserve scan and error semantics while delivering the verified performance improvement.

@lance-gatekeeper lance-gatekeeper Bot added K-approved Latest Gatekeeper recommendation permits acceptance. and removed K-approved Latest Gatekeeper recommendation permits acceptance. labels Aug 25, 2026
@jiaoew1991
jiaoew1991 merged commit 73df0d9 into lance-format:jiaoew/stack-system-column-base-v4 Aug 25, 2026
11 checks passed
@jiaoew1991

Copy link
Copy Markdown
Contributor Author

This PR was merged only into its temporary integration base while the dependency stack was being refreshed; it was not merged into main. The independently reviewable replacement is #8747, based on the refreshed #8713 + #8716 + #8715 stack.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci CI / build workflows A-deps Dependency updates A-encoding Encoding, IO, file reader/writer A-index Vector index, linalg, tokenizer A-java Java bindings + JNI A-python Python bindings K-approved Latest Gatekeeper recommendation permits acceptance. performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant