Skip to content

perf: avoid slow exponentiation and repeated element reads - #23

Merged
lpatiny merged 4 commits into
mainfrom
faster-distances
Jul 30, 2026
Merged

perf: avoid slow exponentiation and repeated element reads#23
lpatiny merged 4 commits into
mainfrom
faster-distances

Conversation

@lpatiny

@lpatiny lpatiny commented Jul 26, 2026

Copy link
Copy Markdown
Member

Speeds up the hot loops of the distance and similarity functions.

Algorithmic changes

  • kumarJohnson: prod ** 1.5prod * Math.sqrt(prod). No engine specializes a 1.5 exponent, so this is the biggest win, on V8 and JavaScriptCore alike.
  • minkowski: special-case p = 1 and p = 2. V8 sends x ** 1 through generic pow at ~9x the cost of a plain read; p = 2 is worth ~1.5x when called monomorphically.
  • similarity.pearson: drop the two arrays it allocated only to mean-centre before calling cosine, folding the work into two passes.

Measured with beforeAfter.js, before → after, per element (n = 10000, Float64Array, node 24):

function before after
kumarJohnson 15.50 ns 2.41 ns 6.4x
similarity.pearson 19.06 ns 3.23 ns 5.9x
minkowski(p=1) 11.51 ns 2.02 ns 5.7x

For the other functions defining const like for example from:

  for (let i = 0; i < a.length; i++) {
    ans +=
      a[i] * Math.log((2 * a[i]) / (a[i] + b[i])) +
      b[i] * Math.log((2 * b[i]) / (a[i] + b[i]));
  }

to

 for (let i = 0; i < a.length; i++) {
    const ai = a[i];
    const bi = b[i];
    const sum = ai + bi;
    ans += ai * Math.log((2 * ai) / sum) + bi * Math.log((2 * bi) / sum);
  }

has no impact, neither with bun or with nodeJS except in the case of polymorphism (once a typed array, once a number array for example) and in this case we win a factor 1.5

But as a side observation changing the type of an array is really a bad idea and we are loosing a factor 2 in speed. I wonder if we can reach this kind of issue in some of our projects.

So practically I would still make this 'const' definition even if we win speed only in case of polymorphism. @targos WDYT ?

@lpatiny
lpatiny force-pushed the faster-distances branch from 8bd41e7 to ee5f0a2 Compare July 26, 2026 19:33
@lpatiny lpatiny changed the title Faster distances perf: avoid slow exponentiation and repeated element reads Jul 26, 2026
lpatiny added 2 commits July 26, 2026 21:42
Built on benchmark.js, the library the other mljs repositories use:

- all.js lists the cost of every exported function
- beforeAfter.js compares two implementations of the same function,
  both copied into one process so they see identical data
- exponentiation.js isolates the cost of the `**` operator
- elementReads.js isolates the cost of repeated `a[i]` reads
- equivalence.js checks `x ** 1.5` against `x * Math.sqrt(x)`

Baseline cost of the current implementations, measured with beforeAfter.js
on node 24 (n = 10000, Float64Array). These are per-element costs of each
function, not speedups:

  topsoe              21.99 ns/element
  similarity.pearson  19.06 ns/element
  kumarJohnson        15.50 ns/element
  minkowski(p=1)      11.51 ns/element
  squared              4.51 ns/element
  cosine               4.38 ns/element
  pearson              3.74 ns/element
  clark                3.22 ns/element

The next commit optimizes these and reports the same measurements again.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three algorithmic changes, each large and stable across engines:

- kumarJohnson: `prod ** 1.5` -> `prod * Math.sqrt(prod)`. No engine
  specializes a 1.5 exponent, so this is the single biggest win.
- minkowski: special-case p = 1 and p = 2. V8 runs `x ** 1` through generic
  pow at ~9x the cost of a plain read; p = 2 is worth ~1.5x.
- similarity.pearson: drop the two arrays it allocated only to mean-centre
  before calling cosine, and fold the work into two passes.

Measured with beforeAfter.js, before -> after (n = 10000, Float64Array,
node 24), against the baseline recorded in the previous commit:

  kumarJohnson        15.50 -> 2.41 ns/element    6.4x faster
  similarity.pearson  19.06 -> 3.23 ns/element    5.9x faster
  minkowski(p=1)      11.51 -> 2.02 ns/element    5.7x faster
  squared              4.51 -> 2.22 ns/element    2.0x faster
  cosine               4.38 -> 2.19 ns/element    2.0x faster
  pearson              3.74 -> 2.14 ns/element    1.7x faster
  clark                3.22 -> 2.17 ns/element    1.5x faster
  topsoe              21.99 -> 17.21 ns/element   1.3x faster

The remaining change caches repeated `a[i]` reads in locals. Its value is
much smaller and depends on what the load site actually observes at run
time, not on any TypeScript type: V8 often eliminates the repeated loads by
itself. elementReads.js measures anywhere from 1.00x when a site only ever
sees one Float64Array shape up to ~1.5x once it has seen several array
kinds, and the exact figure moves with how the harness is built. The rows
above are the higher end of that range because beforeAfter.js exercises
every function with both array kinds in one process. It is never a
regression, so it stays, but it is not a reliable 2x.

JavaScriptCore eliminates those loads outright: on bun the caching is
1.00x everywhere and only kumarJohnson (4x) and similarity.pearson (6x)
improve.

kumarJohnson moves by at most 1 ulp, and `(-Infinity) ** 1.5` is Infinity
where `-Infinity * Math.sqrt(-Infinity)` is NaN; both are outside the
non-negative domain the metric is defined on. Its test now uses toBeCloseTo.

ml-array-mean is no longer used.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lpatiny
lpatiny force-pushed the faster-distances branch from ee5f0a2 to 6f39522 Compare July 26, 2026 19:43
@lpatiny
lpatiny marked this pull request as draft July 26, 2026 20:04
@lpatiny
lpatiny requested a review from targos July 27, 2026 09:24
@lpatiny
lpatiny marked this pull request as ready for review July 27, 2026 09:24
@targos

targos commented Jul 29, 2026

Copy link
Copy Markdown
Member

The specialized changes LGTM but I would avoid to change all methods. I've run the benchmarks myself and I am not convinced that anything is meaningfully improved.

➤ node elementReads.js
Float64Array only  before      5.47 ns/element  ±5.1%
Float64Array only  after       5.54 ns/element  ±4.3%
Array only         before      5.28 ns/element  ±5.8%
Array only         after       5.45 ns/element  ±4.9%
both kinds         before      5.65 ns/element  ±3.5%
both kinds         after       5.41 ns/element  ±5.7%

gain from caching the element reads:
  Float64Array only  0.99x
  Array only         0.97x
  both kinds         1.04x

@lpatiny

lpatiny commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

Here is another benchmark that shows this 1.5 factor on the polymorphism (and does not rely on another library). But for sure the real issue is polymorphism and we need to think about it because even if we define const we still loose a factor 2 if we change of type.

A single switch of the type at the end of the warm-up has a full impact on the number of operation per seconds.

measured warm-up cached direct
array array 1506 M/s 1533 M/s
array typed 704 M/s 464 M/s <- 2.1x / 3.3x slower
typed array 694 M/s 458 M/s <- 2.1x / 3.6x slower
typed typed 1437 M/s 1645 M/s
mixed mixed 711 M/s 493 M/s

8401b5f

distance [faster-distances●] % bash benchmark/arrayKinds.sh

  (ai * bi) / (ai + bi) on 10000 elements, arguments swapped at each call

  kind             cached         direct      ratio
  -------- -------------- -------------- ----------
  array           1558 M/s        1571 M/s      0.99x
  typed           1488 M/s        1679 M/s      0.89x
  mixed            740 M/s         506 M/s      1.46x

@targos

targos commented Jul 29, 2026

Copy link
Copy Markdown
Member

It seems to show that the code is faster when there is no polymorphism, so do we really want to lose this?

@lpatiny
lpatiny merged commit a020077 into main Jul 30, 2026
8 checks passed
@lpatiny
lpatiny deleted the faster-distances branch July 30, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants