Binary size gets bloated by mapping tables

[d3lm]

I have noticed that, when compiling Ada to a Wasm module (via the wasi-sdk), there's a lot of bloat coming from data sections likely the mapping tables. Here's the output from twiggy when profiling the code size of the Wasm module:

As you can see, more than 60% is just data.

I was wondering if there's a way to minimze the bloat from these tables or if there's other ways to reduce the binary size.

CC @lemire @anonrig

[lemire]

Pull request invited !

[lemire]

The binary should be about 280KB. For context, ICU (a standard dependency which we avoid) is over 40 MB or nearly 150 times larger.

[d3lm]

Yes that's true. ICU is a lot bigger. I don't know from the top of my head how we could avoid that "bloat", except for relying on host functionality if compiled to Wasm / WASI. So that we don't have to include the tables, and toUnicode and toAscii is something that could be provided by the host?

[lemire]

After standard compression, the entire ada library (including idna) is well under 200 KB. The compressed WASM is about 100 KB. This is much less than the JavaScript aggregation of most websites. A typical website these days require megabytes of downloads (with caching, of course). It is much, much smaller than ICU which is already almost everywhere.

Except for relying on host functionality if compiled to Wasm / WASI. So that we don't have to include the tables, and toUnicode and toAscii is something that could be provided by the host?

What is contained in your host?

Ada (in its entirety) is distributed with every version of Node.js. Why don't you just distribute ada as binary?

If you are concerned with the 100KB WASM download and you cannot install ada locally, why don't you cache it?

[d3lm]

Yea, the reason I am mostly concerned is not the compressed size. It's small as you said, but for us, we have a multi-process architecture, and userland can spawn any number of processes or threads. Each process instantiates a WebAssembly module that includes ada, which means every process has to parse it, and every process then basically includes its own set of mapping tables. But there is also no standard Web API that we could use to provide toUnicode or toAscii other than implementing this in JS and that would also mean we'd have to have some mapping tables in every process as well.

[lemire]

Each process instantiates a WebAssembly module that includes ada

Even processes that do not need ada? How is ada different, as a library, from any other library like, say, libicu, libzstd or libssh? Or the rust-url library for example?

As far as I know, ada/idna is the most compact library available currently with this functionality.

I am going to close this. You are invited to provide code or specific actionable advice on how to make this library smaller. A pull request would be appreciated.