Live updating scripts

[petrihakkinen]

Hi! I'm trying to find a (nice) way to implement live updating of scripts. By live updating I mean patching (global) functions as the program is running. With Lua/LuaJIT this has worked nicely because everything is dynamic by nature. Luau implements caching for globals using imports, which is a very nice performance optimization but at the same time it rubs against live workflows.

So far my best attempt is the following:

1. When a script is live updated, compile and execute it. Any global functions are replaced with new ones in the global env.
2. Find all function protos by scanning the entire Lua universe
3. Scan the bytecode of every proto looking for GETIMPORT opcodes
4. For every GETIMPORT instruction, look up the import name from proto's constant table
5. Resolve the import and patch the new function to the constant table

I have a rough prototype of this and it seems to be working, but of course there are downsides: it requires access to Luau internal headers and efficiently implementing the bytecode scan and especially step 2 (finding all protos) can be tricky.

Some questions:

1. Is there an easier/faster way to accomplish this? I'm worried steps 2 and 3 can be expensive in a big codebase.
2. For step 2, should I be using luaM_visitgco? Is that the best way to traverse every GC object (including protos)?
3. The bytecode scan could be potentially skipped if there was a way to detect imports by just looking at the constant table. But afaik this is not possible? (I can't determine which string and function/nil values in the constant table belongs to imports...)
4. Would it make sense to have some feature or building blocks for live workflows as part of Luau? In my experience, live updating can raise productivity to the next level. It's also very trendy (Live++).

Of course easy way to work around the issue is to disable imports by setting optimization level to 0 or disabling the safe-env feature, but the performance penalty would be severe.

[zeux]

I'm trying to understand why imports are problematic, because at least the way Repl sets up modules, it shouldn't be necessary to reload anything but the current module - since the global environment should only contain builtins and be frozen as part of luaL_sandbox, each individual module gets their own global table that other modules can't reference. As such GETIMPORT would only reference the global table. Maybe you can expand on the larger example?

If you do need to patch imports then the sequence you outlined seems reasonable. There's no dedicated list of protos, and for each proto there's currently no way to identify imports in a special way either, so you need a full heap scan effectively.

[zeux]

Ok gotcha - thanks. I should note that we currently don't have plans to increase the amount of caching we do, so I wouldn't be worried about the fundamentals of your approach, even if it breaks in small parts (eg some instruction encoding changes), it would probably work longer term. That said, a few thoughts on this in general.

I agree in principle that investing in live editing workflows is a good idea. We've had some surface-level internal discussions about this, mostly just identifying this as a valuable need. That said, I'm not yet sure on what the balance is here between "reload magically works" vs "you must write code in a very specific way". Some problems might not be solveable at all, eg how do you reload a function if there's an active stack frame in a yielded coroutine that's running it; some problems might be difficult to solve, eg what happens to an object that has a function in its metatable and the upvalues for that function changed which may cause a cascading effect on some internals; some problems are easier to solve.

A lot of live reload solutions I've seen in Lua tend to simply rerun the module code and deal with consequences. This wouldn't handle a lot of complex scenarios, and lead to a mix of old & new code in the process sometimes; that said, if we can do something simple that makes things easier, it's always beneficial. You're right that with the shared global namespace and simple code, the only issue atm is likely the import handling.

Ideally the imports would be designed in such a way that they are more separate from constants which would make it easier to reload them. We had some plans to improve the flow in the bytecode loader that would move closer in this direction, but fundamentally you'd need a separate import table allocated to be able to easily re-import. This might be a good idea if the overhead of an extra table is small -- we'd need to look into this. If this works well we'd be able to provide some stable utility function to "reimport" a proto, which would reduce the fragility of your approach.

Having said all this, Luau is definitely expecting segregated globals for different modules with a shared read-only global namespace that's populated with host builtins ahead of time. This has several benefits beyond sandboxing in the "trusted" code sense -- this was the original motivation ~15 years ago when Roblox started, but we've since settled on this as the preferred mode for other reasons; one is performance, where imports and some other optimizations are trivially safe (meaning, don't change the meaning of existing code regardless of what that existing code does); another one is type safety, where our type checker reasons about a module as a unit and relies on explicit dependencies between modules via require instead of expecting that global writes from one module are automatically visible in another module. It's also difficult to scale a very large codebase in a shared global environment since something as simple as a global function named the same way in two modules will cause difficult to trace bugs that depend on module load order.

I'm not saying this to suggest that all Luau embedders must use this - we support another mode as well - but to suggest that in general this is the path that gets much more attention from us, and future language and runtime evolution is going to be best for that path, and sometimes suboptimal for the shared global namespace.

Side note: an interesting example is that the tictactoe bench we discussed had a smaller penalty in unsandboxed code a few months ago, but we switched some code in VM which made some code a tiny tiny bit faster in sandboxed mode but resulted in a slowdown for pairs-based loops in unsandboxed code - precisely because we don't really consider unsandboxed mode as a priority from the performance perspective. That specific issue we might go back on for unrelated reasons, but the rule of thumb applies.

Finally, for the world where globals are actually sandboxed per-module, and require is used for cross-module dependencies, it occurred to me that in the style of "live reload works but your module must be ready for it", it's actually not very difficult to set things up so that simply reexecuting a module will do a live reload without issues with dependents. The only thing that's necessary here is to persist the module table across reloads. For example, the basic "idiomatic" module in Luau looks like this:

local module = {}

function module.foo()
end

return module

Simply changing the first line to:

module = module or {}

Will make the module hot-reloadable in that if you reexecute it, the identity of the module table persists (so all dependent modules - ones that called require - will retain the correct reference), it will update the module functions with the new values, and the imports will not need to be patched because the compiler already knows to only generate imports for "unknown" globals. There's obvious tweaks to this technique where you keep the module local but use some other global that's synthesized by the embedding code, eg local module = CACHED_MODULE or {} where the live reload code sets up CACHED_MODULE global correctly.

Which is to say that it could be that one of the attractions of separate module tables is that you can implement live reload with this technique without depending on VM internals :)

[petrihakkinen]

Thanks! I think storing imports in a separate table would be an overall win: smaller bytecode for GETIMPORT (no aux), one entry in import table vs. two in the constant table for every import. Of course, size of Proto would increase slightly but I think the number of imports is greater than the number of Protos in general.

Regarding fully automatic live-updating: in my experience the programmer needs to be aware of live-updating. The most difficult aspect of live updating is keeping data compatible with the updated code. For example, when the patched function expects a new field in a table but the existing table does not have that. There are at least ways to handle this:

• Use the "or" idiom to provide a default value (maybe writing the updated value back to the table)
• Manually run a piece of patching code that goes through the necessary objects and adds the missing field to every instance
• Restart the executable

The last option is least desirable but sometimes it's faster than patching the data manually if the change is too complex. With this kind of workflow coding becomes a Forth like experience where the program is interactively built piece by piece from bottom up.

btw. I just ran some benchmarks and thanks to imports + native vecs my current game project runs faster with Luau than with LuaJIT (JIT off). I haven't fully finalized import patching yet but I think it will work out fine. Only worry at the moment is speed of live updates (I need to tweak numeric values, e.g. colors quickly so anything above 100ms gets painful).

Re: your side note about tictactoe running faster in sandboxed environment. I'm not sure I understand why this is the case. I enable safe-env feature so fastcalls, imports etc. should work equally as with separate envs. From Luau's perspective the global env should still look like it's "sandboxed" right? The only difference is that there's only one environment. Maybe I'm missing something...

Re: persisting the module table, I think that could work in principle as long as no code is caching any module functions. But if the module functions are not imports like you said, wouldn't it be slower to call a module function this way? With imports, the function can be looked up from the constant table very cheapy, vs. dynamically getting the function from module table.

[zeux]

I think storing imports in a separate table would be an overall win: smaller bytecode for GETIMPORT (no aux), one entry in import table vs. two in the constant table for every import

I suspect we will need to keep imports in the constant table, and simply have the import table reference the constant index. This is because right now imports can simply get the import value from the constant table a pointer to which is stored in hardware registers, whereas reaching for the import through the proto would be either slower (require an indirection through closure -> proto), or require using another register which are always in short supply.

But if the module functions are not imports like you said, wouldn't it be slower to call a module function this way?

Maybe but only marginally. The canonical style of imports is

local mod = require(another module)

mod.test()

This compiles into a single GETTABLEKS instruction (plus CALL obv), which, with inline caching we use, doesn't require a full hash table lookup in most cases. This is going to be a little bit slower than an import, but I suspect compared to the cost of CALL instruction alone, plus accounting for the other machinery required here, the net effect will be insignificant.

[petrihakkinen]

right now imports can simply get the import value from the constant table a pointer to which is stored in hardware registers

I missed that, you're right.

Would it make sense to relocate import names from the constant table to the import table though? This way the constant table would be smaller and therefore more cache coherent. This would increase complexity and cause some duplication in those cases where the same string is used in constant table and import table...

[petrihakkinen]

(The efficiency of this optimization highly depends how many imports the code uses. In my case, with everything in global env I suspect the effect could be significant. But I'm not sure what the effect would be for typical Roblox Lua code.)

[questor]

Hi, any proposal how to implement live-reloading after you shipped your product? or an already implemented solution with source code and hints how to use it?

[petrihakkinen]

That depends how you structure your Lua code. If you go with the official recommended way of splitting the code in separate modules, you basically only need to reload the code into the module environment. However, if you inject code into global table, like I do in my personal projects, you'd have to either disable imports (with performance penalty) or patch the imports after reloading. I'm not aware of any readily available source code that does this unfortunately.