Skip to content

Add selectable io_uring buffer backends - #3

Open
whzruc wants to merge 1 commit into
rebuild/cpp-dynamic-buffer-poolfrom
rebuild/cpp-io-uring-multi-backend
Open

Add selectable io_uring buffer backends#3
whzruc wants to merge 1 commit into
rebuild/cpp-dynamic-buffer-poolfrom
rebuild/cpp-io-uring-multi-backend

Conversation

@whzruc

@whzruc whzruc commented Sep 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a non-fixed io_uring backend that uses ordinary reads without buffer registration
  • retain registered fixed-buffer implementations for the legacy, dynamic, and static buffer-pool modes
  • expose backend selection through pixel.bufferpool.mode
  • route reader initialization, submission, completion, and cleanup through the selected backend
  • add correctness coverage and reproducible benchmarks for both ClickBench q24 and raw I/O scanning
  • support shared interleaved scheduling and one-thread-per-device scheduling in the raw scanner

Stack

This PR is based on rebuild/cpp-dynamic-buffer-pool and should be reviewed after PR #2.

Performance conclusions

The committed reports summarize the results in English and Chinese. Raw outputs and per-run measurements are intentionally excluded from Git.

ClickBench q24 end-to-end

The test used 24 NVMe SSDs, 12/24/48 threads, one warm-up and seven measured runs per backend. All 63 measured runs returned identical results without errors.

Threads non-fixed dynamic static
12 391.134 s 387.452 s 386.226 s
24 213.743 s 212.269 s 213.898 s
48 147.892 s 150.814 s 151.959 s

The largest difference was 2.75%, below the predefined 5% practical-significance threshold. q24 therefore shows no material end-to-end regression from the selectable backends, but its filtering, sorting, and materialization dilute differences in the I/O submission path.

Raw I/O scan

The raw scanner bypasses DuckDB, PixelsReader, decoding, and materialization. Each measured run scanned 15,360 files and 1,916,931,234,624 bytes with 1,048,576-byte requests and queue depth 32 per thread.

Threads non-fixed (GB/s) dynamic (GB/s) static (GB/s)
12 66.18 68.80 68.74
24 109.57 109.57 109.70
48 109.87 110.14 110.34

Registered fixed buffers were about 4% faster at 12 threads, before platform saturation. At 24 and 48 threads, all backends converged within 0.5% at an aggregate ceiling of roughly 110 GB/s.

With 24 threads and 24 devices, forcing one thread per device produced 108.79–109.15 GB/s and was 0.38%–0.83% slower than the shared interleaved queue. The shared queue is not a material bottleneck and slightly improves load balancing.

A single SSD reached 6.14 GB/s, while all 24 averaged about 4.55 GB/s. The 1/4/8/12/24-device scaling curve and the four PCIe root groups indicate that the aggregate ceiling is in the shared PCIe/CPU I/O, DMA, or memory path rather than in buffer registration or file scheduling.

Interpretation

  • fixed buffers provide a small throughput benefit when aggregate I/O bandwidth is not saturated
  • backend choice does not materially affect peak throughput once the platform reaches its shared I/O ceiling
  • dynamic and static are equivalent in steady-state scanning; initialization, grow/update cost, and memory use remain separate evaluation dimensions
  • the known dynamic-backend performance concern is intentionally left for follow-up work

Validation

  • built pixels_extension, PixelsIoUringScanBenchmark, DirectUringNonFixedTest, DynamicBufferPoolTest, and GlobalStaticBufferPoolTest
  • ran buffer-pool correctness tests; io_uring cases skip cleanly where ring initialization is unavailable
  • validated identical q24 query results across all measured backend runs
  • validated identical file, byte, and request coverage across raw-scan backends
  • Python syntax checks and git diff --check pass

@whzruc
whzruc force-pushed the rebuild/cpp-io-uring-multi-backend branch from 9235824 to c50bb08 Compare September 2, 2026 10:01
@whzruc
whzruc force-pushed the rebuild/cpp-io-uring-multi-backend branch from c50bb08 to cccc4fc Compare September 2, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant