Skip to content

fix(bench): avoid OOM in array_replace bench#22120

Merged
alamb merged 1 commit into
apache:mainfrom
kumarUjjawal:fix/array_expression_memory_issue
May 12, 2026
Merged

fix(bench): avoid OOM in array_replace bench#22120
alamb merged 1 commit into
apache:mainfrom
kumarUjjawal:fix/array_expression_memory_issue

Conversation

@kumarUjjawal
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The array_replace benchmark allocated ~90GB of memory before running (3 × 100M Expr literals), causing OOM on normal machines.

What changes are included in this PR?

  • Reduce array_len in array_expression bench from 100_000_000 to 100_000.
  • Remove a broken assert_eq! (and its unused expected_array) that compared the ScalarFunction Expr returned by array_replace_all against the unmodified input — the two Expr trees are never equal. The OOM previously hid the failing assertion.

Are these changes tested?

Yes, cargo bench -p datafusion-functions-nested --bench array_expression passes locally

Are there any user-facing changes?

No

@github-actions github-actions Bot added the functions Changes to functions implementation label May 12, 2026
@alamb
Copy link
Copy Markdown
Contributor

alamb commented May 12, 2026

Thanks @kumarUjjawal

@alamb alamb added this pull request to the merge queue May 12, 2026
Merged via the queue into apache:main with commit a1b788c May 12, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

array_expression benchmark uses too much memory and is sigkill'd by the OS

2 participants