Serialization API

[kvark]

No description provided.

[White-Oak]

Woah, this is gonna be interesting!

[Aceeri]

Would it be alright for specs to primarily use serde for serialiation purposes? Or maybe have some sort of Serializer trait so you can abstract over different serialization libraries?

I'm assuming this will mainly be used for things like building entities all together from files similar to the amethyst issue that was referenced amethyst/amethyst#112 since serializing singular components is fairly simple. Along with maybe serializing an entire world at an instance.

[kvark]

Yes, behind a feature gate.

On Nov 15, 2016, at 21:21, Aceeri notifications@github.com wrote:

Would it be alright for specs to primarily use serde for serialiation purposes?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[Aceeri]

So I figured out a way to have it abstracted a bit:

pub trait Deserializer<T> {                               
    /// Deserialize data into an entity.                  
    fn deserialize(&mut World, T) -> Result<(), Error>;   

    /// Called when the world has a component registered. 
    fn register_component<C>(&mut self) { }               

    /// Called when the world has a resource registered.  
    fn register_resource<R>(&mut self) { }       

    /// Maybe some other methods related to the world's state being changed?         
}                                                         

pub trait Serializer<T> {                                 
    /// Serialize a part of the world and return it.      
    fn serialize(&mut World) -> Result<T, Error>;         

    /// Called when the world has a component registered. 
    fn register_component<C>(&mut self) { }               

    /// Called when the world has a resource registered.  
    fn register_resource<R>(&mut self) { }                
}                                                         

However, I've run into a bit of a wall. In the files you need some way to identify the component as itself, which would probably be some sort of built in static id for each Component type.
e.g.
.yaml

---
Component1:
    field1: 5
    field2: false

Component2:
    field3: [1, 1, 1]

.json

{
    "Component1": {
        "field1": 5,
        "field2": false
    },
    "Component2": {
        "field3": [1, 1, 1]
    }
}

So I'm thinking of maybe having the Component trait include another function: fn id() -> String or something similar. This would probably be nice to have return the fully qualified type of the Component.

[ghost]

So I'm thinking of maybe having the Component trait include another function: fn id() -> String or something similar.

This seems reasonable to me. If we put this in a separate trait, we could have a id_derive crate to provide something like #[derive(Id)] which would be generated from the component struct name.

[ghost]

Maybe something like this?

[ghost]

I think deserializing a world with serde would be a challenge. Because the world needs components to be registered for deserialization to work, you need to create the world before deserializing it, but the deserialize trait needs to produce an owned value out of nothing. A trait such as the one suggested by @Aceeri would be ideal, but I'm not sure how to make this compatible with serde.

EDIT: This is wrong. You can use DeserializeSeed

[ghost]

For DeserializeSeed convenience functions for serde backends, see serde-rs/serde#644, although DeserializeSeed still works today if you instantiate the Deserializer yourself.

[nchashch]

TL;DR: We need a way to serialize and deserialize components, which depend on an outside resource (like AssetManager/gfx::Factory) for creation.

For a component, serialized and deserialized forms of which correspond one to one, like:

Velocity:
  x: 1.0
  y: 2.0

Serialization or deserialization can be done in one step:

serialized component | serde -> deserialized component
deserialized component | serde -> serialized component

But there are important kinds of components for which, serialized and deserialized forms don't correspond. For example Renderable component in amethyst contains gfx-rs handlers, which can't be serialized or deserialized directly.

A serialized version of Renderable would contain filenames of mesh and texture files, like this:

Renderable:
  mesh: "sphere.obj"
  ambient: "ambient.dds"
  diffuse: "diffuse.dds"
  specular: "specular.dds"

So additional steps would be required in order to get from serialized Renderable to deserialized Renderable:

serialized Renderable |- serde -> Asset filenames |- AssetManager -> Assets in RAM |- gfx::Factory -> gfx-rs handlers |- -> deserialized Renderable

Maybe we could store Options of gfx-rs handlers in Renderable and all the filenames for the assets. Then a System would:

1. walk all Renderables
2. give jobs to AssetManager to load all the required assets
3. check every frame whether or not assets were loaded
4. if they were loaded, fill Renderables with Some(gfx-rs handler)s.

Since filenames would probably be stored in deserialized version serialization would remain one step:

deserialized Renderable | serde -> serialized Renderable

[Aceeri]

@nchashch I think that could just be done with custom implementations of Serialize/Deserialize for those types of components. You might need a secondary "helper" component though I suppose.

Something like:

#[derive(Serialize, Deserialize)]
struct RenderableSerialize {
    mesh: String, // Could also maybe be Path? Not sure if serde implements that.
    ambient: String,
    diffuse: String,
    specular: String,
}
impl Component for RenderableSerialize { ... }

struct RenderableSerializeSystem {
    added: HashSet<Entity>, // possible memory leak-ish stuff if we dont remove entities that die though
}
impl System<()> for RenderableSerializeSystem {
    fn run(&mut self, arg: RunArg, _: ()) {
        let (entities, renderables_s, renderables) = arg.fetch(|w| {
            (w.entities(), w.read::<RenderableSerialize>(), w.write::<Renderable>())
        };
        for (entity, renderable_s) in (&entities, &renderables_s).join() {
            if !self.added.contains(entity) {
                let renderable = { ... }; // construct renderable from pathes in asset manager
                renderables.insert(entity, renderable);
                self.added.insert(entity);
            }
        }
    }
}

[torkleyy]

I guess we can close this now. @kvark

[kvark]

Hooray!