Skip to content

Expose LayoutStrategy::buffered_bytes through the jni boundary#8909

Merged
robert3005 merged 4 commits into
developfrom
rk/bufferedbytes
Jul 24, 2026
Merged

Expose LayoutStrategy::buffered_bytes through the jni boundary#8909
robert3005 merged 4 commits into
developfrom
rk/bufferedbytes

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

This is useful to track write progress since flushed bytes are an underestimate

Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005
robert3005 marked this pull request as ready for review July 23, 2026 09:41
@robert3005 robert3005 added changelog/feature A new feature changelog/chore A trivial change and removed changelog/feature A new feature labels Jul 23, 2026
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done af5a1e5 1 Explore Profiling Data
Previous Runs (2)
Status Commit Job Attempt Link
🟢 Done f71d090 1 Explore Profiling Data
🟢 Done 71b18b5 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: Vortex queries 📖

Verdict: No clear signal (medium confidence)
Attributed Vortex impact: +7.7%
Engines: DataFusion No clear signal (+11.1%, environment too noisy confidence) · DuckDB No clear signal (+4.4%, low confidence)
Vortex (geomean): 1.087x ➖
Parquet (geomean): 1.048x ➖
Shifts: Parquet (control) +4.8% · Median polish +7.1%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.171x ❌, 0↑ 2↓)
name PR af5a1e5 (ns) base 7654891 (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 🚨 9735078 8574618 1.14
vortex_q01/datafusion:vortex-file-compressed 🚨 6314904 5229639 1.21
datafusion / parquet (1.054x ➖, 0↑ 1↓)
name PR af5a1e5 (ns) base 7654891 (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 21712677 21568209 1.01
vortex_q01/datafusion:parquet 🚨 5331263 4830857 1.10
duckdb / vortex-file-compressed (1.087x ➖, 0↑ 0↓)
name PR af5a1e5 (ns) base 7654891 (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 11666479 10627404 1.10
vortex_q01/duckdb:vortex-file-compressed 7054921 6551542 1.08
duckdb / parquet (1.042x ➖, 0↑ 0↓)
name PR af5a1e5 (ns) base 7654891 (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 24437304 23559866 1.04
vortex_q01/duckdb:parquet 9985572 9545147 1.05

No file size changes detected.

Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@codspeed-hq

codspeed-hq Bot commented Jul 23, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 26.92%

⚡ 1 improved benchmark
✅ 1846 untouched benchmarks
⏩ 46 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation decode_varbin[(1000, 4)] 34.3 µs 27 µs +26.92%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/bufferedbytes (af5a1e5) with develop (7654891)

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@robert3005
robert3005 enabled auto-merge (squash) July 23, 2026 12:01
@robert3005
robert3005 merged commit 9c10b73 into develop Jul 24, 2026
75 checks passed
@robert3005
robert3005 deleted the rk/bufferedbytes branch July 24, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants