Brainstorming Features

[WebReflection]

Now that this module is fully covered and production ready, I’d like to write down thoughts around its application and how this can improve further to better tackle all use cases.

Recursion

---

Update it's in and it works wonderfully

---

It’s wonderful and super fast but it inevitably retains a lot in memory. This is not really an issue in JS, those references will be kept in memory anyway until the returned reference exists, but it makes lower level serialization (WASM, C, Zig) more convoluted for little benefits when:

• data is created out of SQLite. It’s true that strings repetition is common, each row key, but it’s also true that there could be thousand different strings or numbers as value … is it worth to track all of them in memory?
• streamed data. With a pre-defined static buffer it’s all good but with data streamed on demand, which is potentially huge, the amount of RAM needed to track recursion might be huge and not friendly with constrained devices (MicroPython to name a use case). The same goes for decoding such data. If the stream is huge, there is no way to free memory while decoding each part of it until the stream ends.
• cases where recursion is neither expected nor desired for complex data. With JSON that’s not possible but in the other hand, this is the reason flatted has million downloads per week.

In other libraries recursion is only for non primitives but here there is a chance to fine tune its capabilities:

• all means recursion for anything as it is now, except Boolean, null and empty strings
• some could mean only non primitive data
• none could mean no recursion expected, will eventually throw out of max stack because if anything is retained in memory to track non primitives there is little gain … worth it?

The question is if the implementation should be bitwise based, so that grouping cases or excluding these can be ad hoc, or generic enough which feels more appropriate.

Once generic: should the default be more relaxed (some) or does it matter, since at deciding time, unless is none, it either works or throw if none is expected?

Hooks while encoding

Having a toBufferedClone symbol feels the right thing to do but should I think about a way to also decode the returning value later?

This could be a method passed as option in both encoding and decoding, where the different data could be transformed in and back.

Performance

---

Update ... conclusions

---

Need to test:

• raw JSON comparison - update ✅ done ... and it currently sucks
• postMessage - update ✅ done, it's up to 100X faster on postMessage over complex data ... and it is currently as fast on single pass, and once hot, but X times faster on multiple pass of the same data around
• worth writing a native node thing too to see how much this would improve performance if it was a native API?

[WebReflection]

Web Worker Performance

Well, I'd be damned ... when I've heard "structuredClone is very slow cross workers" I wasn't expecting this slower compared to bufferedClone ... here some benchmark you can test live (open the console to see results)

---

MISLEADING RESULTS - the benchmark was all wrong so nothing in here made sense.

A new benchmark is in the making and I will share results once it's completed.

A work in progress can be tested live

---

~~It's amazng this library is ~10X slower on the same thread to perform a structured clone but up to 100X faster in complex postMessage based applications 🥳~~

---

edit conclusions

[WebReflection]

OK, for correctness I'd like to also add WebKit and Firefox results ... yeah, we're down to 2X up to 10X there, but it's clear there is something really weird in postMessage and structuredClone combination, as it's super fast inline, it's deadly slow for anything else.

---

MISLEADING RESULTS - the benchmark was all wrong so nothing in here made sense.

A new benchmark is in the making and I will share results once it's completed.

A work in progress can be tested live

---

edit conclusions

[WebReflection]

To whoever is following: an issue has been updated with these details in Chromium repository because I find it absurd it's so slow compared to other browsers, and that slower than my own user-land library to create a buffer view of any JS serializable type.

[jorroll]

I'm not sure how you created the benchmarks you've shared above, but my own benchmarking (in Chrome) indicates that using this library with postMessage is slower than postMessage without using this library @WebReflection.

• https://stackblitz.com/edit/vitejs-vite-jsakzcgj?file=src%2Fmain.ts
• Edit: I forked the stackblitz project and that seems to have caused a bug in stackblitz that's preventing that demo from running now (says it can't find a file). You should be able to copy the code and run it locally as a test though.

[titoBouzout]

@jorroll your test doesn't seem to be using the transfer option on the postMessage.

I found the original benchmark too clever and a bit hard to understand, I am seeing of refactoring it to double-check the results. btw the number on first load is similar to Andreas numbers, but subsequent refreshes show a smaller albeit considerable gap.

[jorroll]

@jorroll your test doesn't seem to be using the transfer option on the postMessage.

It is. See the worker.ts file for "encoded-query". Note that the test only encodes the response from the worker to the main thread. I didn't bother encoding the request from the main thread to the worker. This shouldn't affect the conclusion though.

[jorroll]

Also, strange. I didn't change the stackblitz example at all, but now it appears to be working again 🤷. Hopefully you can open up the link and view the console logs to see the results.

[titoBouzout]

I kind of hardcoded an alternative benchmark trying to avoid unnecessary things. coding/decoding both sides, seems like structure clone is faster, or we may be missing something

worker2.html

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <script type="module">
      import { decode, encode } from "../src/index.js";
      import { data } from "./data.js";

      const worker = new Worker(new URL("./worker2.js", import.meta.url), {
        type: "module",
      });

      let promise;

      worker.addEventListener("message", (event) => {
        const { type, data } = event.data;

        switch (type) {
          case "structured-clone": {
            promise.resolve(data);
            break;
          }
          case "buffered-clone": {
            promise.resolve(decode(data));
            break;
          }
          case "buffered-clone-echo": {
            promise.resolve(decode(data));
            break;
          }
          case "buffered-clone-recursion-all": {
            promise.resolve(decode(data, { recursion: "all" }));
            break;
          }
          case "buffered-clone-recursion-some": {
            promise.resolve(decode(data, { recursion: "some" }));
            break;
          }
        }
      });

      function get(type, data) {
        promise = Promise.withResolvers();
        switch (type) {
          case "structured-clone": {
            worker.postMessage({ type, data });
            break;
          }
          case "buffered-clone": {
            worker.postMessage({ type, data: encode(data) });
            break;
          }
          case "buffered-clone-echo": {
            worker.postMessage({ type, data: encode(data) });
            break;
          }
          case "buffered-clone-recursion-all": {
            worker.postMessage({
              type,
              data: encode(data, { recursion: "all" }),
            });
            break;
          }
          case "buffered-clone-recursion-some": {
            worker.postMessage({
              type,
              data: encode(data, { recursion: "some" }),
            });
            break;
          }
        }
        return promise.promise;
      }

      async function test(type, data) {
        // warm
        for (let i = 1; i < 5; i++) {
          await get(type, { 0: [data] });
        }

        // time
        const iterations = 10;

        const start = performance.now();

        for (let i = 0; i < iterations; i++) {
          await get(type, { 0: [data] });
        }

        const total = performance.now() - start;

        console.log(
          type + " - total",
          +total.toFixed(2),
          "avg",
          +(total / iterations).toFixed(2),
        );
      }

      async function main() {
        await test("structured-clone", data);
        await test("buffered-clone", data);
        await test("buffered-clone-echo", data);
        await test("buffered-clone-recursion-all", data);
        await test("buffered-clone-recursion-some", data);

        console.log("--- with alternative data ---");
        await test("structured-clone", dataAlternative);
        await test("buffered-clone", dataAlternative);
        await test("buffered-clone-echo", dataAlternative);
        await test("buffered-clone-recursion-all", dataAlternative);
        await test("buffered-clone-recursion-some", dataAlternative);
      }

      main().catch((error) => console.error("error running tests", error));

      var dataAlternative = {
        _id: "5973782bdb9a930533b05cb2",
        isActive: true,
        balance: "$1,446.35",
        age: 32,
        eyeColor: "green",
        name: "Lsadafe Kegwewgewg",
        gender: "gwgwegweg",
        company: "Gewwewgewg",
        email: "234g3gergreg@aaaaaaaaaaaaaaaaaaaa.com",
        phone: "+0 (000) 000-0000",
        friends: [
          {
            id: 0,
            name: "Casaasdn Ssadsad",
          },
          {
            id: 1,
            name: "Fsdasdasd Msasd",
          },
          {
            id: 2,
            name: "Csadf M3434g43g",
          },
        ],
        favoriteFruit: "banana",
      };
    </script>
  </head>
</html>

worker2.js

import { encode, decode } from "../src/index.js";

self.addEventListener("message", (event) => {
  const { data, type } = event.data;

  switch (type) {
    case "structured-clone": {
      self.postMessage({ type, data });
      break;
    }
    case "buffered-clone": {
      const encodedData = encode(decode(data));

      self.postMessage({ type, data: encodedData }, [encodedData.buffer]);
      break;
    }
    case "buffered-clone-echo": {
      self.postMessage({ type, data }, [data.buffer]);
      break;
    }
    case "buffered-clone-recursion-all": {
      const encodedData = encode(decode(data, { recursion: "all" }), {
        recursion: "all",
      });

      self.postMessage({ type, data: encodedData }, [encodedData.buffer]);
      break;
    }
    case "buffered-clone-recursion-some": {
      const encodedData = encode(decode(data, { recursion: "some" }), {
        recursion: "some",
      });

      self.postMessage({ type, data: encodedData }, [encodedData.buffer]);
      break;
    }
    default: {
      throw new Error(`unknown message type ${type}`);
    }
  }
});

[WebReflection]

Ok, I think I know what’s the issue … basically the first round trip adds the Worker initialization time + network. I swear I’ve inverted order at some point and still see similar results but it makes sense, the benchmark is wrong.

Now it’s 2am here but tomorrow I will create a better benchmark that:

• creates a worker and waits for its first interacion
• post data as is, change it on the other side somewhere, post back and validate changes
• encode data and transfer it, decode and change it somewhere, encode and transfer it back
• do the same per each method twice or up to 5 times to see if there’s any cold VS hot perf improvement

This way all things should be closer to a real-world use case and the network time won’t affect neither cases … will post results right after hoping there will still be better perf via buffered clone, otherwise I’ll go back to the drawing board to see if there’s any extra room for better perf.

Thanks everyone for chiming in and double checking results: appreciated and happy to get this story right 🙏

[jorroll]

Ok, I think I know what’s the issue … basically the first round trip adds the Worker initialization time + network.

The test I shared on stackblitz factors this in. Before each test run, a single round-trip request is made and awaited before beginning benching.

async function test(type: 'query' | 'encoded-query') {
  // First execute a query to warm up the javascript interpreter
  await get(type);

  const allStart = performance.now();
  let iterations = 0;

  for (let i = 1; i < 5; i++) {
    iterations++;
    const start = performance.now();
    const data = await get(type);
    const total = performance.now() - start;
    // console.log(`get ${type} took ${total}ms`, data);
  }

  const allTotal = performance.now() - allStart;

  console.log(
    `${type} tests took ${allTotal}ms; avg of ${allTotal / iterations}ms`
  );
}

[WebReflection]

@joroll I didn’t check ‘cause I was on my phone but that was indeed an obvious mistake of mine in the bench … I’ve rushed conclusions, it was my fault: I’ll create a way better benchmark that will cover all use cases I’m interested in/after, thanks 🙏

[jorroll]

@joroll I didn’t check ‘cause I was on my phone but that was indeed an obvious mistake of mine in the bench … I’ve rushed conclusions, it was my fault: I’ll create a way better benchmark that will cover all use cases I’m interested in/after, thanks 🙏

Ah gotcha. I thought you were referring to the benchmark I shared when you said "the first round trip adds the Worker initialization time + network".

[WebReflection]

@jorroll and others, a new benchmark is on its way but I can confirm the previous one was just plain wrong.

Current results suggest that while buffered-clone is in fact not faster than structured-clone for a single roundtrip (this is without decoding, it's just encoding once per each benchmark and transfer it to the worker and transfer it back) you can see that in a triple worker roundtrip scenario, which covers a shared worker that distributes from a leading main worker use case, where the worker in the leading main thread would encode once, transfer results to main, which transfers results to worker, which transfers results to the other tab that asked for results, the buffered-clone approach is indeed ideal because it doesn't sum up every single intermediate postMessage dance.

Right now, for that specific use case I'd call it a win as that was the initial discussion but missing from the benchmark are all other use cases I am interested in:

• JSON stringify over big results VS encode (the flatted case)
• StructuredClone Polyfill Extras stringify operation (the coincident use case)

I will update this issue once I have concrete results.

---

edit conclusions