Skip to content

Prevent non empty unions and clarify null docs#8882

Merged
connortsui20 merged 1 commit into
developfrom
ct/union-non-empty
Jul 21, 2026
Merged

Prevent non empty unions and clarify null docs#8882
connortsui20 merged 1 commit into
developfrom
ct/union-non-empty

Conversation

@connortsui20

Copy link
Copy Markdown
Member

Rationale for this change

Tracking issue: #8769

What changes are included in this PR?

As discussed offline, we have decided to disallow empty unions because we do not think there is a good usecase for it. And as you can see in the diffs here, that meant we could not choose a good default for the "zero value" of scalars.

This PR prevents empty unions, as well as clarifies some of the null docs around the union scalar accessors.

What APIs are changed? Are there any user-facing changes?

Some name changes to make things more obvious

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 75d61fd 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Vortex queries 📖

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +4.6%
Engines: DataFusion No clear signal (+7.6%, low confidence) · DuckDB No clear signal (+1.8%, low confidence)
Vortex (geomean): 1.009x ➖
Parquet (geomean): 0.986x ➖
Shifts: Parquet (control) -1.4% · Median polish +1.0%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.045x ➖, 0↑ 0↓)
name PR 75d61fd (ns) base 62af4a1 (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 9795641 9464067 1.04
vortex_q01/datafusion:vortex-file-compressed 6255843 5931343 1.05
datafusion / parquet (0.971x ➖, 0↑ 0↓)
name PR 75d61fd (ns) base 62af4a1 (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 19999395 20451849 0.98
vortex_q01/datafusion:parquet 4475686 4637790 0.97
duckdb / vortex-file-compressed (1.019x ➖, 0↑ 0↓)
name PR 75d61fd (ns) base 62af4a1 (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 10021423 9877811 1.01
vortex_q01/duckdb:vortex-file-compressed 6058631 5919041 1.02
duckdb / parquet (1.001x ➖, 0↑ 0↓)
name PR 75d61fd (ns) base 62af4a1 (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 23289169 23193585 1.00
vortex_q01/duckdb:parquet 9359157 9377576 1.00

No file size changes detected.

@connortsui20
connortsui20 enabled auto-merge (squash) July 21, 2026 16:32
@connortsui20
connortsui20 merged commit f68d60d into develop Jul 21, 2026
111 of 114 checks passed
@connortsui20
connortsui20 deleted the ct/union-non-empty branch July 21, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants