Skip to content

perf: array comparison reference equality + numeric fast path#674

Merged
stephenamar-db merged 1 commit intodatabricks:masterfrom
He-Pin:perf/array-ref-equality
Apr 8, 2026
Merged

perf: array comparison reference equality + numeric fast path#674
stephenamar-db merged 1 commit intodatabricks:masterfrom
He-Pin:perf/array-ref-equality

Conversation

@He-Pin
Copy link
Copy Markdown
Contributor

@He-Pin He-Pin commented Apr 4, 2026

Motivation

Array comparison in the evaluator compares element-by-element. Two optimizations are possible:

  1. If both arrays are the same object (reference equality), they are trivially equal
  2. For numeric elements (the most common comparison type), a direct double comparison is faster than full Val dispatch

Key Design Decision

Add reference equality short-circuit and inline numeric comparison fast path in the array comparison loop.

Modification

  • Evaluator.scala: Added reference equality check for arrays, inline numeric fast path for element comparison

Benchmark Results

JMH (JVM, 3 iterations)

Benchmark Master (ms/op) This PR (ms/op) Change
bench.02 50.427 ± 38.906 46.361 ± 6.238 -8.1%
comparison2 85.854 ± 188.657 70.597 ± 22.435 -17.8%
realistic2 73.458 ± 66.747 64.775 ± 1.436 -11.8%

References

  • Upstream exploration: he-pin/sjsonnet jit branch array comparison optimization

Result

-8% to -18% JVM improvement for array comparison via reference equality short-circuit and numeric fast path.

Add reference equality check (xi eq yi) in the array comparison loop
before type dispatch. When arrays share elements (e.g., long_array + [1]
< long_array + [2] where long_array has 1M shared elements), this skips
type dispatch for all identical references.

Also inline numeric fast path (double compare without polymorphic
compare() dispatch) for the common case where both elements are Num.

Upstream: jit branch commits f7ad961, 62437d8
@He-Pin He-Pin force-pushed the perf/array-ref-equality branch from fa49ccf to b9d479c Compare April 4, 2026 17:51
@He-Pin He-Pin changed the title perf: array comparison reference equality short-circuit perf: array comparison reference equality + numeric fast path Apr 4, 2026
@He-Pin He-Pin marked this pull request as ready for review April 4, 2026 21:49
@stephenamar-db stephenamar-db merged commit cfbb1a1 into databricks:master Apr 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants