Skip to content

Conversation

cesco69
Copy link
Contributor

@cesco69 cesco69 commented Oct 9, 2025

TLTR: the current benchmark is not reliable
see #801 (comment)

In this PR have migrated the current benchmark based on benchmark.js (abbandoned, last update 7 years ago) to tinybench.
I have also preserved the same output

PR (tinybench)

short string............................................. x 11,148,323 ops/sec ±96.84% (449942 runs sampled)
unsafe short string...................................... x 17,983,965 ops/sec ±0.41% (1559790 runs sampled)
short string with double quote........................... x 8,865,937 ops/sec ±7.80% (668640 runs sampled)
long string without double quotes........................ x 9,749 ops/sec ±35.67% (471 runs sampled)

MASTER (benchmark.js)

short string............................................. x 25,309,923 ops/sec ±2.39% (180 runs sampled)
unsafe short string...................................... x 95,582,481 ops/sec ±6.63% (157 runs sampled)
short string with double quote........................... x 16,337,301 ops/sec ±2.08% (182 runs sampled)
long string without double quotes........................ x 52,829 ops/sec ±2.25% (182 runs sampled)

Benchmark compare: PR vs PR

image

Benchmark compare: MASTER VS MASTER

image

Raw result patch-10 VS patch-10

Checking out "patch-10"
Execute "npm run bench"

> fast-json-stringify@6.1.1 bench
> node --expose-gc ./benchmark/bench.js

short string............................................. x 11,706,183 ops/sec ±0.51% (1490282 runs sampled)
unsafe short string...................................... x 13,448,873 ops/sec ±0.39% (1751476 runs sampled)
short string with double quote........................... x 9,955,676 ops/sec ±1.22% (1042841 runs sampled)
long string without double quotes........................ x 58,338 ops/sec ±2.10% (5234 runs sampled)
unsafe long string without double quotes................. x 14,098,104 ops/sec ±0.70% (1838793 runs sampled)
long string.............................................. x 32,270 ops/sec ±2.72% (2562 runs sampled)
unsafe long string....................................... x 15,952,407 ops/sec ±0.35% (2090631 runs sampled)
number................................................... x 16,597,008 ops/sec ±0.33% (2172467 runs sampled)
integer.................................................. x 12,844,829 ops/sec ±0.45% (1659085 runs sampled)
formatted date-time...................................... x 1,296,819 ops/sec ±0.63% (125221 runs sampled)
formatted date........................................... x 1,114,027 ops/sec ±1.02% (108293 runs sampled)
formatted time........................................... x 1,106,145 ops/sec ±0.56% (107010 runs sampled)
short array of numbers................................... x 78,312 ops/sec ±1.59% (7337 runs sampled)
short array of integers.................................. x 78,772 ops/sec ±1.55% (7361 runs sampled)
short array of short strings............................. x 21,574 ops/sec ±3.07% (1811 runs sampled)
short array of long strings.............................. x 22,968 ops/sec ±2.94% (1982 runs sampled)
short array of objects with properties of different types x 11,311 ops/sec ±2.55% (1070 runs sampled)
object with number property.............................. x 15,830,834 ops/sec ±0.38% (2075975 runs sampled)
object with integer property............................. x 14,275,757 ops/sec ±0.37% (1872645 runs sampled)
object with short string property........................ x 10,257,803 ops/sec ±0.70% (1198907 runs sampled)
object with long string property......................... x 42,880 ops/sec ±2.65% (3489 runs sampled)
object with properties of different types................ x 1,302,833 ops/sec ±2.31% (104535 runs sampled)
simple object............................................ x 8,235,082 ops/sec ±1.18% (675282 runs sampled)
simple object with required fields....................... x 7,885,421 ops/sec ±1.86% (608887 runs sampled)
object with const string property........................ x 17,662,142 ops/sec ±0.35% (2313928 runs sampled)
object with const number property........................ x 14,926,448 ops/sec ±0.37% (1960630 runs sampled)
object with const bool property.......................... x 14,163,588 ops/sec ±0.34% (1852894 runs sampled)
object with const object property........................ x 14,341,321 ops/sec ±0.35% (1875053 runs sampled)
object with const null property.......................... x 13,666,385 ops/sec ±0.30% (1785938 runs sampled)

Checking out "patch-10"
Execute "npm run bench"

> fast-json-stringify@6.1.1 bench
> node --expose-gc ./benchmark/bench.js

short string............................................. x 11,570,845 ops/sec ±0.46% (1469542 runs sampled)
unsafe short string...................................... x 11,351,172 ops/sec ±0.57% (1430163 runs sampled)
short string with double quote........................... x 9,530,411 ops/sec ±1.35% (941579 runs sampled)
long string without double quotes........................ x 53,824 ops/sec ±2.38% (4621 runs sampled)
unsafe long string without double quotes................. x 14,104,935 ops/sec ±0.52% (1842289 runs sampled)
long string.............................................. x 48,667 ops/sec ±1.26% (4648 runs sampled)
unsafe long string....................................... x 12,879,691 ops/sec ±0.85% (1668303 runs sampled)
number................................................... x 13,393,812 ops/sec ±0.79% (1732412 runs sampled)
integer.................................................. x 14,141,119 ops/sec ±0.54% (1845326 runs sampled)
formatted date-time...................................... x 1,280,996 ops/sec ±0.71% (124255 runs sampled)
formatted date........................................... x 1,090,780 ops/sec ±0.95% (106092 runs sampled)
formatted time........................................... x 1,094,414 ops/sec ±0.62% (105745 runs sampled)
short array of numbers................................... x 76,887 ops/sec ±1.51% (7195 runs sampled)
short array of integers.................................. x 78,839 ops/sec ±1.65% (7345 runs sampled)
short array of short strings............................. x 24,590 ops/sec ±1.24% (2378 runs sampled)
short array of long strings.............................. x 21,519 ops/sec ±2.97% (1806 runs sampled)
short array of objects with properties of different types x 11,028 ops/sec ±2.55% (1045 runs sampled)
object with number property.............................. x 15,872,653 ops/sec ±0.32% (2084372 runs sampled)
object with integer property............................. x 17,123,302 ops/sec ±0.32% (2249903 runs sampled)
object with short string property........................ x 11,338,634 ops/sec ±0.34% (1426428 runs sampled)
object with long string property......................... x 41,316 ops/sec ±3.13% (3156 runs sampled)
object with properties of different types................ x 1,840,636 ops/sec ±1.55% (152921 runs sampled)
simple object............................................ x 8,278,110 ops/sec ±0.95% (683770 runs sampled)
simple object with required fields....................... x 7,021,421 ops/sec ±1.75% (507834 runs sampled)
object with const string property........................ x 16,731,578 ops/sec ±0.43% (2188026 runs sampled)
object with const number property........................ x 14,244,972 ops/sec ±0.64% (1864377 runs sampled)
object with const bool property.......................... x 16,654,446 ops/sec ±0.32% (2179446 runs sampled)
object with const object property........................ x 16,938,277 ops/sec ±0.42% (2227930 runs sampled)
object with const null property.......................... x 16,442,077 ops/sec ±0.34% (2154620 runs sampled)

short string..............................................+1.17%
unsafe short string......................................+18.48%
short string with double quote............................+4.46%
long string without double quotes.........................+8.39%
unsafe long string without double quotes..................-0.05%
long string..............................................-33.69%
unsafe long string.......................................+23.86%
number...................................................+23.92%
integer...................................................-9.17%
formatted date-time.......................................+1.24%
formatted date............................................+2.13%
formatted time............................................+1.07%
short array of numbers....................................+1.85%
short array of integers...................................-0.08%
short array of short strings.............................-12.27%
short array of long strings...............................+6.73%
short array of objects with properties of different types.+2.57%
object with number property...............................-0.26%
object with integer property.............................-16.63%
object with short string property.........................-9.53%
object with long string property..........................+3.79%
object with properties of different types................-29.22%
simple object.............................................-0.52%
simple object with required fields.......................+12.31%
object with const string property.........................+5.56%
object with const number property.........................+4.78%
object with const bool property..........................-14.96%
object with const object property........................-15.33%
object with const null property..........................-16.88%
Back to bench f2c2c6f

Signed-off-by: francesco <francesco.bagnoli.69@gmail.com>
@cesco69 cesco69 changed the title feat: new benchmar with tinybench feat: new benchmark with tinybench Oct 9, 2025
@cesco69 cesco69 marked this pull request as ready for review October 9, 2025 10:38
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

mcollina commented Oct 9, 2025

@Uzlopak why do you think there is a difference?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants