Skip to content

perf(compute): vectorize mixed int32 filters with AVX2 - #1284

Merged
zeroshade merged 1 commit into
apache:mainfrom
fallintoplace:perf/compute-filter-avx2-compress
Sep 3, 2026
Merged

perf(compute): vectorize mixed int32 filters with AVX2#1284
zeroshade merged 1 commit into
apache:mainfrom
fallintoplace:perf/compute-filter-avx2-compress

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add an AVX2 path for filtering 32-bit fixed-width values with mixed selection masks.
  • Compress two groups of four values with vpshufb and use masked stores for the compact output.
  • Keep the current path for nulls, short or unaligned inputs, dense masks, unsupported CPUs, and noasm builds.
  • Add tests and a benchmark for mixed masks, offsets, and 1K, 64K, and 1M rows.

Benchmark

The benchmark is in arrow/compute/vector_selection_bench_test.go.

It covers alternating, random 25/50/75%, clustered 50%, all-selected, and none-selected masks at 1K, 64K, and 1M rows.

A native AVX2 run needs an x86_64 machine with AVX2. The benchmark also runs on the local Apple M1, but that only exercises the existing arm64 fallback, so no local AVX2 timing is included.

Tests

  • go test ./arrow/compute/internal/kernels ./arrow/compute
  • go test -tags noasm ./arrow/compute/internal/kernels ./arrow/compute
  • go test -race ./arrow/compute/internal/kernels ./arrow/compute
  • amd64 cross-build for both packages

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dispatch guards are sound (AVX2 probe, length%8, byte-aligned filter offset, bounds check, null-free inputs only, mixed-mask heuristic) and GetSpanValues applies span offsets, so the sliced-input case is safe. The c2goasm source is included per repo convention. The AVX2 path itself will be exercised by the AMD64 CI jobs I've just approved.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Arrow Go maintainer.

@zeroshade
zeroshade merged commit e9cc75c into apache:main Sep 3, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants