WASM Support, NPM package#510
Conversation
These bugs all exist in stock Magic but were tolerated by the K&R-loose
native build. The strict WASM call_indirect type checks turned them up.
* cif/CIFhier.c: ASSERT in cifFlatMaskHints accessed
oldproprec->prop_value.prop_type, but prop_type is a top-level member
of PropertyRecord. Changed to oldproprec->prop_type.
* extflat/EFargs.c: efLoadSearchPath was assigning a pointer to a
string literal ("." in RO data), which callers later try to free or
StrDup. Replaced with StrDup(path, ".") so the pointer always lives
on the heap.
* router/rtrVia.c: rtrFollowName called RtrMilestonePrint("#"), but
the function takes no arguments.
* sim/SimSelect.c: SimAddLabels called DBWLabelChanged with five
arguments, but its real signature is (CellDef *, Label *, int).
Replaced with the equivalent DBWAreaChanged call.
* windows/windView.c: extern declaration of DBMovePoint had return
type void, but the function actually returns bool.
WASM call_indirect enforces an exact type match between the caller and the callee. Many Magic callbacks had K&R-style () forward declarations and a single-argument definition, but were passed to iterators that always push a trailing ClientData argument. Native builds tolerated the mismatch via loose prototypes; WASM traps with "indirect call signature mismatch". Added the missing ClientData (or, where the concrete type is known, FindRegion *) parameter to: * calma/CalmaRead.c, calma/CalmaWrite.c, calma/CalmaWriteZ.c — calmaWriteInitFunc * cif/CIFwrite.c — cifWriteInitFunc * commands/CmdSubrs.c — cmdWindSet * database/DBtimestmp.c — dbStampFunc * dbwind/DBWelement.c — dbwElementAlways1 * dbwind/DBWfdback.c — dbwfbWindFunc * dbwind/DBWhlights.c — DBWHLRedrawWind * ext2spice/ext2hier.c — spcnodeHierVisit * extract/ExtBasic.c — extSDTileFunc, extTransPerimFunc, extAnnularTileFunc, extResistorTileFunc * extract/ExtMain.c — extDefInitFunc * extract/ExtTimes.c — extTimesInitFunc Also adjusted commands/CmdE.c and commands/CmdTZ.c: SelectExpand was being called with four arguments (the legacy surroundFlag), but its real signature has been three arguments for years (the surround mode is encoded in the expandType bit). The fourth argument was redundant (DB_EXPAND_SURROUND in arg 2 is the source of truth) and rejected by WASM. Native behavior is unchanged. The added parameters are unused in the function bodies; they exist only to satisfy the indirect-call signature.
CIFGenSubcells() and extSubtree() set GrDisplayStatus = DISPLAY_IN_PROGRESS while they run a 5-second progress timer, then unconditionally restore DISPLAY_IDLE on exit. In native builds the initial state is always IDLE, so this is harmless. In WASM/headless builds the null display driver sets DISPLAY_SUSPEND at startup, and forcing IDLE at the end of these long operations destroys the SUSPEND guard that protects WindUpdate() from running display callbacks against a non-existent screen. Save the previous status before overwriting and restore it on exit. This is also reentrant-safe: nested DISPLAY_IN_PROGRESS scopes (e.g. extract followed by gds write) now keep the outer state intact. * cif/CIFhier.c — CIFGenSubcells * extract/ExtSubtree.c — extSubtree
Magic's graphics layer routes every drawing primitive through function pointers (GrXxxPtr / grXxxPtr) that are bound to a driver at startup. The original null driver assigned a single 0-arg nullDoNothing() to every pointer, which works in native builds because of K&R loose prototype rules but fails in WASM where call_indirect requires an exact type match between caller and callee. This commit: * Adds typed no-op stubs nullDoNothingI/II/IIII/IIIIIII for void-returning callbacks of various arities. * Adds nullReturnFalseI/II/III for bool-returning callbacks and nullReturnZeroI for int-returning callbacks. * Casts each pointer assignment in nullSetDisplay() to the K&R pointer type the public header still uses, while the underlying function carries the correct WASM signature. * Fills in window-management and backing-store pointers that the original null driver left at NULL — many of these are called unconditionally by WindUpdate paths, and need at least a no-op to avoid traps. * Guards the stdin watch in nullSetDisplay() with #ifndef __EMSCRIPTEN__: WASM has no real stdin file descriptor and TxAdd1InputDevice() / SigWatchFile() are POSIX-specific. Native builds are unaffected: the K&R-loose prototype machinery still accepts the previous and the new code identically.
Glue between the null display driver and the rest of Magic so that running with -d null does not require any process-level resources (signals, timers, stdin, an X display, or a Tcl interpreter). * utils/signals.c — gate setitimer, fcntl-based file watches, kill and the legacy sigsetmask/sigaction setup behind #ifdef __EMSCRIPTEN__. Every signals path becomes a no-op in WASM. Also fixes DBWriteBackup() being called with one argument when its real prototype takes three. * windows/windDisp.c — WindUpdate() returns immediately when GrDisplayStatus == DISPLAY_SUSPEND. This is the runtime counterpart to the null driver's DISPLAY_SUSPEND state. * extflat/EFargs.c — EFArgs() with a missing input name no longer jumps to "usage:" in headless WASM (which would call MainExit and kill the process); it sets *err_result and returns NULL so the caller can decide what to do. Native MAGIC_WRAPPER and native non-MAGIC_WRAPPER builds keep their original behavior. * dbwind/DBWcommands.c — registers exttosim / ext2sim / exttospice / ext2spice in non-MAGIC_WRAPPER builds. Without this, WASM users could not invoke these commands at all (they were previously inside an #ifdef MAGIC_WRAPPER block). The C implementations (CmdExtToSim / CmdExtToSpice) are linked unconditionally outside modular builds. * textio/txCommands.c, textio/textio.h — TxDispatchString(), a new library-style command entry point that parses a single string, dispatches it through WindSendCommand and returns a status code. This is what magic_wasm_run_command() calls from JavaScript.
The pieces that make Magic actually buildable as a WASM library.
* magic/magicWasm.c — new headless entry point exporting four
functions used by the JS wrapper:
- magic_wasm_init() idempotent initialisation
- magic_wasm_run_command(s) dispatch one Magic command
- magic_wasm_source_file(p) execute a script from the VFS
- magic_wasm_update() drive a display-update cycle
Sets CAD_ROOT=/ if unset, so embedded technology files under
/magic/sys/ resolve correctly. Centers the command point inside
GrScreenRect so commands route to the layout window client
rather than the border/window-management client.
* utils/main.c, utils/main.h — split magicMain() into magicMainInit()
+ the dispatch loop. magicMainInit is idempotent (a static flag
guards against re-initialisation) so JS callers can call any of
the four wasm entry points first without sequencing.
* magic/Makefile — adds the WASM link target, gated by MAKE_WASM=1
set from toolchains/emscripten/defs.mak. Conditionally compiles
magicWasm.c into the main binary, links to magic.js and runs
post-build.sh on the result.
* toolchains/emscripten/defs.mak — Emscripten linker flags (WASM=1,
MODULARIZE, EXPORT_ES6, ALLOW_MEMORY_GROWTH, INITIAL_MEMORY=32M,
STACK_SIZE=5M), the four EXPORTED_FUNCTIONS, and the embed-file
bindings for the technology files under /magic/sys/.
* toolchains/emscripten/post-build.sh — patches Emscripten's ESM
output so it works in pure Node.js ESM: aliases require()
through createRequire, injects __filename / __dirname shims,
and resyncs the ___emscripten_embedded_file_data constant from
the wasm global section if Emscripten emitted a stale value.
Idempotent and pinned to emsdk 3.1.56 (see WARNING in the
header).
* toolchains/emscripten/README.md — full build documentation:
quick-start via npm/build.sh, manual build, list of embedded
files, exported C API, JavaScript usage example, and notes on
CAD_ROOT, DISPLAY_SUSPEND, and the signal-API stubs.
* .gitignore — adds the WASM artefacts (magic.js, magic.wasm,
magic.symbols), tightens the editor/OS cruft list, and keeps
toolchains/emscripten/defs.mak tracked despite the `defs.mak`
ignore rule.
The user-facing layer of the WASM port: a publishable npm package
plus the GitHub Actions that build and ship it.
* npm/package.json — publishes as `magic-vlsi-wasm`, ESM-only, HPND
licensed, version tracks Magic's own VERSION file (8.3.637).
Whitelists the published files and exposes index.js + index.d.ts.
* npm/index.js, npm/index.d.ts — thin JS/TS wrapper around the four
WASM exports. createMagic(opts) returns { init, runCommand,
sourceFile, update, FS } so consumers can write into the
Emscripten virtual filesystem and dispatch Magic commands from
Node.js, browsers or Web Workers.
* npm/build.sh — end-to-end build: locates emsdk (via PATH or
EMSDK_DIR), runs distclean+configure+make in the right order
(techs before mains so embed-files are present), copies
magic.js / magic.wasm into npm/. Optional --release, --test,
--pack flags. Preserves configure's exec bits across invocations.
* npm/pack.sh — produces a reproducible npm tarball by touching
every file to the build time and exporting SOURCE_DATE_EPOCH so
pacote does not rewrite mtimes to its 1985 fallback.
* npm/examples/ — runnable smoke tests for the four common
workflows (extract, gds, drc, cif), driven by examples/all.js.
Each example is self-contained and uses the bundled siliwiz
technology. helpers.js encapsulates the boilerplate.
* npm/LICENSE, npm/README.md — license text and consumer-facing
docs (install, quick-start, API, examples, build-from-source,
license, third-party content notice).
* .github/workflows/main.yml — adds a `simple_build_wasm` job that
installs a pinned emsdk (3.1.56), builds the WASM module, runs
the example test suite and uploads the npm tarball as an
artifact. Pinned for reproducibility against the post-build.sh
patches; switchable to "latest" by commenting two lines.
* .github/workflows/main-aarch64.yml — drops the now-redundant
WASM ARM job. WASM is architecture-independent.
* .github/workflows/npm-publish.yml — new workflow. Publishes to
npm on `v*` tag pushes (manual `workflow_dispatch` supported as
a dry-run). Uses the same pinned emsdk and pack.sh.
Also sets FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 in both workflows to
silence the Node.js 20 deprecation warnings until
actions/upload-artifact@v6 ships a Node-24 release.
… job - npm-publish.yml: target GitHub Packages instead of npm registry; no NPM_TOKEN needed, uses GITHUB_TOKEN - Both workflows: emsdk defaults to 'latest' on every automated run so CI tracks emsdk HEAD and catches breakage early; version is overridable via workflow_dispatch input - npm-publish.yml: add parallel ARM (ubuntu-24.04-arm) WASM build job with Emscripten diagnostics step - main.yml, npm-publish.yml: upgrade runners from ubuntu-22.04 to ubuntu-latest
Mirrors the AppImage versioning form (8.3.637~20260414~d157eea) as closely as semver allows: 8.3.637-20260414.d157eea. - package.json: placeholder 0.0.0-dev; CI always overwrites before pack - npm-publish.yml: version step now runs unconditionally (not just on tags), reading VERSION + git date + git hash — dry-run artifacts are stamped too, making them unambiguously traceable to their source commit
All functions that received a ClientData (or FindRegion *) parameter in commit fc21472 solely to satisfy WASM call_indirect signature matching are now annotated so static analysers and casual readers can see the intent has been verified: /*ARGSUSED*/ before each such function definition /* UNUSED */ on each unused parameter in the definition /* UNUSED */ on matching forward declarations Affected: calmaWriteInitFunc, cifWriteInitFunc, cmdWindSet, dbStampFunc, dbwElementAlways1, dbwfbWindFunc, DBWHLRedrawWind, spcnodeHierVisit, extSDTileFunc, extTransPerimFunc, extAnnularTileFunc, extResistorTileFunc, extDefInitFunc, extTimesInitFunc.
RTimothyEdwards
left a comment
RTimothyEdwards
left a comment
There was a problem hiding this comment.
I have a few issues with the scripts used for the CI, but nothing major (a few methods are outdated with respect to recent code changes, and I feel like it would make more sense to use the existing scmos.tech technology file than to import the siliwiz.tech technology file).
This appears to compile the wasm version without Tcl support? I'm concerned that that removes a lot of capability from the tool. But if I've done my job maintaining backwards compatibility with the non-Tcl build, then it should work okay.
I will approve this but would like to hear from Darryl Miles.
There was a problem hiding this comment.
I think my previous instruction was misunderstood slightly.
The main-aarach64.yml simple_build_wasm_arm can be removed, if there is a new replacement, that does the same and optionally publishes a package. It is only the "Emscripten Diagnostic" step that needs to be copied into the new WASM CI build task. Which was the matter in my previous instruction.
It is OK to remove the simple_build_wasm_arm step, as shown in the previous PR/change-set iteration.
I assume it should not matter if Intel or Arm was used to build WASM the output binary blob should the same ?
The same is true of removing the simple_build_wasm from main.yml (so the WASM build is in a new workflow)
|
Ive changed the example to a scmos example. Also Ive updated the readme in /npm, since it was outdated. Yes, it doenst use tcl at the moment, but there is a parser that can take a script and run it. No complex logic as of yet. @dlmiles I was confused, since the comment was in the aarch file. Yeah WASM / Emcripten does not care if it is run and/or compiled on different ISAs. (Could even run on a more modern phone probably). Give me a minute and I make a change. |
|
Maybe the |
|
@dlmiles Ive updated as u requested. I hope its satifiying. tcl would be the next todo item for me, but what im working on can now continue forwards. Btw. absolute thanks to all of the work that has been done by both of you (and all of the other maintainers) |
Please cite the methods of concern, otherwise we're guessing, so at least people can take a look.
There has never been Tcl support in the WASM build. This requires building Tcl for WASM object target, then using the output object to link magic against. The problem here is the Tcl engine assumes a Unix process environment at runtime and is based on 1980 programming methods and conventions, threading an after thought, large scale IO after thought (solved ~15y later). I believe it is possible to have a special cut-down Tcl core that might help towards this goal. The current WASM support presented in this change-set is no different to how it has always been. So this change-set do not change the current situation and is not a backward step in this regard. Now on this subject (of Tcl language support for byte-code runtimes). I have as a separate and independent project a working Tcl engine that is specifically designed to run on byte-code runtimes. It still has matters to improve that I am working on over time, over 1000 unit tests that run Tcl script snippets and compare directly against Tcl8.6 and Tcl9.0. At this point it is just parser/ast-generation/interpreted-execution-engine/runtime-library, I am just starting to look at native Tcl bytecode support (yes Tcl has an internal bytecode). Then from getting this to work I have a template for targeting other byte-codes directly such as WASM. I expect many months before announcing something viable. This approach is to bring Tcl up to par with other language support in the byte-code space (as it is somewhat forgotten/left-out by the wider community), but once up to par first-class Python support arrives for free (which is the language the EDA community wants). So now there can be a REPL/shell-interface that understands both Python and Tcl at the same time. My goal here is the best cross-language interop on unconstrained desktop environment, you could say targeting utility, productivity and performance. WASM in the browser-constrained environment running headless magic is useful, but my interest is more in the interop interfaces that allow that to happen naturally as by-product of separation at the right internal boundaries. Luckly most of those boundaries already exist just a few things need unpicking and putting in the correct place. WASM at this time is still receiving major features every 3 months (as the details are worked out) with browser 3 months behind that, in some cases these features might be considered minimum requirements in 2026 but the details take a while to work out, so their process seems to be steady stable and well thought out.
Still looking over things, needs to clone / merge run GitHub CI and try somethings locally, so will take least this week and weekend. So far have I have only commented in the things I can see from GH. |
|
@dlmiles : Nice! So, based on your comments, I think that I have no particular objections to this pull request once you've run your tests and approve it yourself. As for the scripts, now that I've looked over them more carefully, three of them (DRC, GDS, and CIF) are trivially simple (three-line scripts). These three scripts look fine. In the "extract" script (extract.tcl): line 20: Should be "select top cell", which will always select the full layout. It's not clear to me what design is being passed to the extract script, but only a layout that is not hierarchical will produce a valid output. |
|
under the /npm/examples is min.mag (a BJT Transistor in the scmos technology). Thats the basis for all the example scripts |
|
@dlmiles : I have no additional comments here. If you agree, I'll go ahead and merge this pull request. |
|
Confirm no further changes from another walk through. Was hoping to have had time to put up on github and test build releases and confirm release mechanism and how the final binary looks but fixups can be done after merge. |
|
Pulled and merged on opencircuitdesign.com. The github mirror will update overnight. |
3 participants
Ive had to force push something into my repo, so I couldnt reopen that PR, so heres a new one.
Ive implemented what you asked for: