General Script API + Lua Implementation

[makspll]

Features every script should have access to, i.e.:

• AddComponent
• RemoveComponent
• SpawnEntity

Lua API has been completed with a general framework extendable by other languages. Rhai will be supported in another release

[coderedart]

from what i understand , bevy's normal app world (where script hosts exist) is not the same as render app world. does that mean we cannot use lua (or rhai) to do custom rendering in bevy?

I was wondering on how to expose egui paint callbacks to lua and i cannot find a way yet.

not a feature request, just wanted to confirm if its a hard limitation (in that case scripts can still spawn / despawn the Text or Gltf render functionality with commands from bevy after #10 merges ) or whether there is a workaround.

[makspll]

Hmm, I am not familiar with the render world, but from a quick read, it appears the worlds are synchronized each frame via ExtractResource and the like.

Are you using bevy_egui ? Could you point me to the callback functionality you mention in their code/docs?

But yeah #10 will add a whole lot of the bevy interface in, namely everything that implements reflect will be supported (at least with minimum functionality, i.e. a metatable), including resources and components under ReflectComponent and ReflectResource which includes Text.

[coderedart]

bevy_egui doesn't implement it yet. although it should be possible as rust callbacks can be static and sent across threads. bevy_egui also clones the egui context into the render app during extract stage to use it inside the render stage of normal egui stuff (without callbacks). egui context is Arc<Mutex<ContextImpl>>, which might cause performance issues if both threads try to lock / use it at the same time as pipelined rendering happens in parallel to the main bevy app systems.

so, if Script hosts are also inside Arc<Mutex<Lua>> they can be cloned / shared into render app.

1. although i don't think if we can send the lua callbacks as they still have 'lua lifetime, i can just be happy with a normal LuaRender event / hook and the script can handle the rest by itself.
2. we also need to use their unique ID to make sure that the render event is sent to the exact same script in render sub app as the entity ids might be different between main and render apps for the same script***.
3. even if bevy_mod_scripting doesn't support the full extent of bevy_render functionality (which is probably huge and idk if its worth the effort), we can just make lua bindings for the wgpu + naga (shaders) and let the scripts do raw draw calls and rendering themselves.

*** one quirk to note from https://bevy-cheatbook.github.io/gpu/intro.html#pipelined-rendering :

Every frame, all entities in the render World are erased, but resources are kept. If you need to persist data from frame to frame, store it in resources. Dynamic data that could change every frame should be copied into the render world in the Extract stage, and typically stored using entities and components.

[makspll]

Hmm so if I understand correctly:

In order to make use of the paint callback functionality, you need to be able to send a callback to the render world of this type?

Arc<dyn Fn(&PaintCallbackInfo, &mut (dyn Any + 'static)) + Send + Sync + 'static>

Yeah in that case, as you say, there is no way you'll send any lua-level callbacks from mlua since all functions are of type

Function<'lua>

which is non static.

The only leftover solutions are going to be trying to pass callbacks which invoke the script host to check for lua callbacks and running them I think. This would probably mean adding some Arc's into the code.

I am happy to accept PR's for this if you come up with a solution, since I am currently focusing on other parts. You're more familiar with both egui and the bevy rendering pipeline to implement this if you want to.

[coderedart]

if ScriptContext was Arc<Mutex<Lua>>, and just like lua gets a bevy world pointer, if lua gets a weak_ptr to its own context Arc<Mutex<Lua>>, then we can impl callbacks like this:
lets say our lua script has a hook function called "on_render".

/// this lua function takes 2 arguments in addition to egui context.
/// Arguments:
/// render_callback_id: String. this will be provided to the "on_render" hook and the script can use this as a key to decide what it needs to render.
/// lua_vm_context: this is the script context's (Arc<Mutex<Lua>>) weak pointer. 
fn egui_callback(lua: &Lua, egui_ctx: egui::Context, render_callback_id: String, lua_vm_context:  Weak<Mutex<Lua>>) {
	let lua_vm_context = lua_vm_context.upgrade().unwrap(); // now we have the script context (Arc<Mutex<Lua>>)
	// create the actual callback
	let lua_paint_callback = Arc::new( move |info: &PaintCallbackInfo, renderer: &Renderer| {
		// get lua vm and call the "on_render" function hook with "render_callback_id" as first argument and info / renderer as second and third arguments
		lua_vm_context.lock().globals().get("on_render").call(...)
	}
	// now this callback is static and we don't need to make any changes to bevy_mod_scripting except wrapping scriptcontext in Arc and providing a WeakPtr as a global in lua vm.
}

having a Weak<Mutex<Lua>> is better as we avoid Arc getting stuck in cyclic reference problem.

there's no urgency though. because bevy_egui doesn't support the callbacks functionality anyway. and lua would need wgpu / naga bindings first before this is actually useful. I am just happy to know that this is possible.

[makspll]

Nice, that looks solid! Wrapping the contexts in an arc is probably the better move anyway since they are inserted/removed often!