Radical idea: Should we rewrite the VM in...

[lukego]

Should we rewrite the VM in GNU assembler?

Elevator pitch: Frees us from the bootstrap dependency on Lua/LuaJIT/minilua, costs moderate porting effort, and has very little impact from a maintenance perspective.

Situation now:

• RaptorJIT VM is written in DynASM (C variant.)
• RaptorJIT build depends on Lua/LuaJIT/minilua for DynASM.

Issues:

• DynASM is just being used as a generic assembler. Chosen for historical reasons.
• DynASM C variant is being used. Lua would be nicer. (Maybe bad idea with minilua.)
• Build dependency on Lua/LuaJIT/minilua/nix is inconvenient for RaptorJIT end-users (but reasonable for upstream RaptorJIT hackers.)

So we need a solution so that:

• End-users can build RaptorJIT easily (conservative build dependencies.)
• VM hackers can work efficiently (liberal build dependencies if helpful.)
• RaptorJIT code footprint is kept small (don't overwhelm maintainers.)

Solutions:

• Status quo: Currently end-users need to install LuaJIT as a build dependency for DynASM. This is inconvenient, but not the end of the world.
• Short term: We can break the LuaJIT dependency for end-users by checking the generated VM code into the repo. However there is a good chance that this will become unwieldy and kludgy over time if the code is churning often e.g. due to frequent merge conflicts, busy-work of regenerating and checking in, risk of accidentally missing an update and building stale code, etc.
• Long term: Maybe it does make sense to reformulate the VM in GNU assembler? The code should look and feel basically the same as the current DynASM version but be buildable with standard GNU tools. (It will also be more work to integrate VM changes from other LuaJIT branches, but that ship has sailed: we are not going to live in LuaJIT's shadow.) See jonesforth.S for an example of a virtual machine written in GNU assembler.

Related:

• DynASM in Lua mode is an extremely valuable piece of software. Likely it should be bundled with RaptorJIT for convenient use by application programmers. Snabb hackers are increasingly using DynASM for key algorithms and data structures, see e.g. siphash.dasl. This doesn't necessarily mean we need to use DynASM for the RaptorJIT VM though.
• We already decided that rewriting the VM in C is a dubious idea.

See also: #58 "Rewrite the VM bytecode interpreter in C?"