Consider Using GLTF as the interchange format rather than bevy's scene format.

[sdfgeoff]

Bevy supports importing via GLTF - which includes animations, textures, materials and a zillion other thing. To implement support for these through the RON scene format would be a significant amount of work - and because the RON scene format is tightly coupled to Bevy's internal representations (eg of things like transforms), breakage between bevy versions is extremely likely.

Summary:

Pros:

• Someone else defines GLTF, so the format is stable
• Someone else has already implemented blender's exporter for GLTF, and it is good.
• Someone else has already implemented bevy's importer for GLTF, and it is good.

Cons:

• May be less suitable for defining large scenes with lots of asset re-use because meshes (etc) are not shared between GLTF files. TBH the way I was doing this currently is rather sketchy, and I'm not that sure that I'm a fan anyway.

Implementation Notes

The main advantage of this repo is the ability to define custom properties and use blender's UI to configure them. This is extremely useful, and the current GLTF pipeline doesn't support costom ECS-like properties in either application. However, support for reading the extra properties from GLTF was implemented in May last year, and by enabling this checkbox in blender's GLTF exporter, you can export custom properties from blender:

Because we store the exporter data in custom properties, enabling this checkbox .... just works!

So, what would we need to do:

1. Implement a function on the bevy side that can hook into GLTF import and convert JSON into components. https://bevy-cheatbook.github.io/3d/gltf.html#gltf-master-asset Shows how you can 'tag' a GLTF on import so it may be possible to create a similar function that is generic and mutates the scene_bundle that comes from the GLTF import. Chances are we can "just use" serde to handle the deserialization. We may have to put all the components into a single "bevy_ecs" property and do some type wrangling/macro stuff.

2. Delete all the RON stuff and encode/decode logic from the exporter (yay! less code)

3. ??? Profit?

[sdfgeoff]

So I had a bit of a fiddle this evening: trying to use the gltf-master-asset approach linked above. However, this block of code:

pub fn spawn_gltfs(
    mut commands: Commands,
    mut gltf_loader: ResMut<BlenderGltfLoader>,
    assets_gltf: Res<Assets<Gltf>>,
    assets_scene: Res<Assets<Scene>>,
) {
    // if the GLTF has loaded, we can navigate its contents

    gltf_loader.gltfs.retain(|gltf| {
        if let Some(mut raw_asset) = assets_gltf.get_mut(gltf) {
            // Loaded already
            for raw_scene in raw_asset.scenes.iter_mut() {
                if let Some(mut scene) = assets_scene.get_mut(raw_scene) {
                    let world = &mut scene.world;
                    let mut nodes_with_extras = world.query::<(Entity, &GltfExtras)>();
                    for (entity, extras) in nodes_with_extras.iter(&world) {
                        println!("{:?}. {:?}", entity, extras);
                    }
                } else {
                    println!("FATAL: GLTF loaded but scenes not loaded?!")
                }
            }

Fails because you can't DerefMut something in the asset server (ie once an asset is loaded, it is immutable as far as I can tell - and I was hoping to mutate the asset).

One solution is to add the components at spawn time. There is a small crate here: https://github.com/nicopap/bevy-scene-hook that demo's how to do this. This will have a slight spawn-time cost, but we'll see if this is an issue - I doubt it will be.

[sdfgeoff]

I fiddled with the bevy_scene_hook this evening with great success: I can attach components after using serde to parse JSON in the GltfExtras field, and it works fairly well.

A couple challenges:

• Generics. I can't find a good way of getting serde to deserialize a map of:

{
    "enumVariantName": {"details":"thing"...}
    "enumVariantName2": {"other":"thing2"...}
    "enumVariantName3": {"new":"thing3"...}
}

I think this is what serde_with::EnumMap is meant to do, but combining that with the fact that blender exports non-existent components as {} this makes things a bit challenging.
This is totally solvable by writing a custom serde deserializer, but I'll finish derisking first.

At this point I can successfully import/export data from blender. Yay!

In the current plugin, physics data (eg convex hull shape) is encoded into the scene at export time, but if we are using GLTF as the interchange, then we need to grab it from the objects Mesh it at load time. For whatever reason, the GLTF Loader creates extra entities. All meshes are added to the object as children of blank transform-entities. So the blank-transform-entities get the GltfExtra component and they have a child with the Handle<Mesh> component. This makes finding what mesh to use for physics shapes rather awkward.
I'll have a bit of a think about this, but I wonder if I should just have a "build physics shape" button in blender and send it via the extra properties..... There's probably some design work to do here.

But anyway, the proof of concept suggests this can be made to work. Now I need to decide if I want to make it work!

---

There are really two parts to this project, and using GLTF as the interchange really clearly distinguishes them:

1. A blender UI that encodes extra data into a GLTF file
2. A bevy plugin that loads components from data in a GLTF file.

The cool thing is that these are completely separable so long as the format of the extra components is well defined. I haven't yet found if anyone has defined a GLTF extension for this, but if someone has, then we should totally use it. It would make the both parts useful even in absense of each other. I can imagine a future where multiple content tools (eg blender, wings3d, max, maya) all can export components, and multiple game engines (eg bevy, godot, unity) can all import the components. This would greatly broaden the ability for scenes containing logic to be described in a cross-tool way

[sdfgeoff]

If I redefine how components are stored in the blender file, then I can avoid a whole stack of complexity in parsing (ie no more serde-mashing required. So that's what I'm planning to do.

I've decided to split this project into two (or three):

1. The specification of how to represent ECS components in GLTF. (Complete)
2. The blender addon to make this easy. (In Progress)
3. The bevy plugin to make loading them easy.

This repository will turn into #3
I imagine that #2 will require relatively little development after initially being created (hopefully not much until blender 4.0....) and because we will be using bevy's GLTF importer, hopefully there won't be much to change in this repo between bevy versions either.

[kulkalkul]

Hey, I was exploring the options about defining components using GLTF and saw this repository and issue. Any updates on this?

Quick update, I saw this now:
sdfgeoff/gltf_ecs_components#9