perf: avoid slow exponentiation and repeated element reads - #23
Conversation
8bd41e7 to
ee5f0a2
Compare
Built on benchmark.js, the library the other mljs repositories use: - all.js lists the cost of every exported function - beforeAfter.js compares two implementations of the same function, both copied into one process so they see identical data - exponentiation.js isolates the cost of the `**` operator - elementReads.js isolates the cost of repeated `a[i]` reads - equivalence.js checks `x ** 1.5` against `x * Math.sqrt(x)` Baseline cost of the current implementations, measured with beforeAfter.js on node 24 (n = 10000, Float64Array). These are per-element costs of each function, not speedups: topsoe 21.99 ns/element similarity.pearson 19.06 ns/element kumarJohnson 15.50 ns/element minkowski(p=1) 11.51 ns/element squared 4.51 ns/element cosine 4.38 ns/element pearson 3.74 ns/element clark 3.22 ns/element The next commit optimizes these and reports the same measurements again. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three algorithmic changes, each large and stable across engines: - kumarJohnson: `prod ** 1.5` -> `prod * Math.sqrt(prod)`. No engine specializes a 1.5 exponent, so this is the single biggest win. - minkowski: special-case p = 1 and p = 2. V8 runs `x ** 1` through generic pow at ~9x the cost of a plain read; p = 2 is worth ~1.5x. - similarity.pearson: drop the two arrays it allocated only to mean-centre before calling cosine, and fold the work into two passes. Measured with beforeAfter.js, before -> after (n = 10000, Float64Array, node 24), against the baseline recorded in the previous commit: kumarJohnson 15.50 -> 2.41 ns/element 6.4x faster similarity.pearson 19.06 -> 3.23 ns/element 5.9x faster minkowski(p=1) 11.51 -> 2.02 ns/element 5.7x faster squared 4.51 -> 2.22 ns/element 2.0x faster cosine 4.38 -> 2.19 ns/element 2.0x faster pearson 3.74 -> 2.14 ns/element 1.7x faster clark 3.22 -> 2.17 ns/element 1.5x faster topsoe 21.99 -> 17.21 ns/element 1.3x faster The remaining change caches repeated `a[i]` reads in locals. Its value is much smaller and depends on what the load site actually observes at run time, not on any TypeScript type: V8 often eliminates the repeated loads by itself. elementReads.js measures anywhere from 1.00x when a site only ever sees one Float64Array shape up to ~1.5x once it has seen several array kinds, and the exact figure moves with how the harness is built. The rows above are the higher end of that range because beforeAfter.js exercises every function with both array kinds in one process. It is never a regression, so it stays, but it is not a reliable 2x. JavaScriptCore eliminates those loads outright: on bun the caching is 1.00x everywhere and only kumarJohnson (4x) and similarity.pearson (6x) improve. kumarJohnson moves by at most 1 ulp, and `(-Infinity) ** 1.5` is Infinity where `-Infinity * Math.sqrt(-Infinity)` is NaN; both are outside the non-negative domain the metric is defined on. Its test now uses toBeCloseTo. ml-array-mean is no longer used. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ee5f0a2 to
6f39522
Compare
|
The specialized changes LGTM but I would avoid to change all methods. I've run the benchmarks myself and I am not convinced that anything is meaningfully improved. |
|
Here is another benchmark that shows this 1.5 factor on the polymorphism (and does not rely on another library). But for sure the real issue is polymorphism and we need to think about it because even if we define const we still loose a factor 2 if we change of type. A single switch of the type at the end of the warm-up has a full impact on the number of operation per seconds. measured warm-up cached direct distance [faster-distances●] % bash benchmark/arrayKinds.sh
(ai * bi) / (ai + bi) on 10000 elements, arguments swapped at each call
kind cached direct ratio
-------- -------------- -------------- ----------
array 1558 M/s 1571 M/s 0.99x
typed 1488 M/s 1679 M/s 0.89x
mixed 740 M/s 506 M/s 1.46x |
|
It seems to show that the code is faster when there is no polymorphism, so do we really want to lose this? |
Speeds up the hot loops of the distance and similarity functions.
Algorithmic changes
kumarJohnson:prod ** 1.5→prod * Math.sqrt(prod). No engine specializes a 1.5 exponent, so this is the biggest win, on V8 and JavaScriptCore alike.minkowski: special-casep = 1andp = 2. V8 sendsx ** 1through generic pow at ~9x the cost of a plain read;p = 2is worth ~1.5x when called monomorphically.similarity.pearson: drop the two arrays it allocated only to mean-centre before callingcosine, folding the work into two passes.Measured with
beforeAfter.js, before → after, per element (n = 10000, Float64Array, node 24):kumarJohnsonsimilarity.pearsonminkowski(p=1)For the other functions defining const like for example from:
to
has no impact, neither with bun or with nodeJS except in the case of polymorphism (once a typed array, once a number array for example) and in this case we win a factor 1.5
But as a side observation changing the type of an array is really a bad idea and we are loosing a factor 2 in speed. I wonder if we can reach this kind of issue in some of our projects.
So practically I would still make this 'const' definition even if we win speed only in case of polymorphism. @targos WDYT ?