Skip to content

Conversation

carlopi
Copy link
Collaborator

@carlopi carlopi commented May 13, 2025

For the statically linked extension build path, this should be basically irrelevant, moving from inlined list of functions in lib/CMakeLists.txt to a file hosting the list, now at lib/base_exported_list.txt.

For the loadable extension path, the more interesting, this move to a two step linking process:

  1. first, duckdb-wasm is built regularly (as now)
  2. then, duckdb-wasm is linked, also here as currently done, passing -s MAIN_MODULE=1, this means there is limited pruning of functions, but on the other side this means the built WebAssembly module will offer the duckdb-functionality, skipping the needs to duplicate that in each extension
  3. a list of exported functions is obtained. Here solution is an hack, but I will review a (duck-based?) tool to do this more properly with less bash glue.
  4. duckdb-wasm is linked a second time, now with -s MAIN_MODULE=2 and -s EXPORTED_FUNCTIONS=@exported_list.txt. MAIN_MODULE=2 means more aggressive dead code elimination is now perfomed, but keeping intact the list exported at the WebAssembly level (so that say the v-table for all relevant class is not actually pruned)
  5. usual cleanup of JavaScript-level export is performed

This works around some limitation in current tooling, and allows to bring in specific informations that are otherwise not propagated.

Result should be moving the on-the-wire (=compressed) size of duckdb-wasm from 7.2 MB of Wasm and 384 KB of JS to X MB of Wasm and Y of JS.

There is a follow up planned to further reduce via splitting the main duckdb-wasm module in two parts, but that's for the follow-up.

@carlopi carlopi merged commit 82c134e into duckdb:main May 13, 2025
15 checks passed
@Tishj
Copy link
Contributor

Tishj commented May 18, 2025

warning: /opt/homebrew/Cellar/emscripten/4.0.6/libexec/src/postamble.js: invalid item in EXPORTED_RUNTIME_METHODS: createDyncallWrapper
em++: warning: warnings in JS library compilation [-Wjs-compiler]
[100%] Built target duckdb_wasm
+ '[' yes == yes ']'
+ make TARGET=eh update_exported_list
cd build/relsize/eh && wasm2wat duckdb_wasm.wasm --enable-all -o duckdb-wasm.wat
/bin/sh: wasm2wat: command not found
make[1]: *** [update_exported_list] Error 127
make: *** [build_loadable] Error 2

Fixed by:

brew install wabt

But this isn't mentioned anywhere outside of the github action files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants