Serialization/Deserialization of a world

[redstonedash]

Describe the problem you are trying to solve.
It is difficult to serialize/deserialize simulation state.

Describe the solution you'd like
a function which outputs a char array that can be written to a file, and a function to read that char array to recreate the world, ideally also a way to exclude some components from serialization (for instance components with pointers)

[SanderMertens]

@redstonedash nice timing- I just implemented a snapshot feature (#113) which takes a snapshot of the world which can be restored at a later point in time. The snapshot feature accepts a filter argument, which both lets you include and exclude components from the snapshot.

Being able to store such a snapshot in a file would be useful. It would probably not be as a char array (flecs does not know the contents of a component, so it can't intelligently turn it into a string, unless you want to base64 everything), but I could do something like a ecs_snapshot_to_blob and ecs_snapshot_from_blob, so that you could do something like this to save:

ecs_world_t *world = ecs_init();

// do stuff ...

ecs_snapshot_t *s = ecs_snapshot_take(world);
size_t size;
void *blob = ecs_snapshot_to_blob(world, s, &size);

FILE *f = fopen("savefile", "w");
fwrite(blob, size, 1, f);
fclose(f);

ecs_snapshot_free(world, s);
ecs_fini(world);

And to restore:

ecs_world_t *world = ecs_init();

size_t size;
void *blob;

// boiler plate to load file into blob & size

ecs_snapshot_t *s = ecs_snapshot_from_blob(world, blob, size);

ecs_snapshot_restore(world, s);

ecs_fini(world);

[redstonedash]

yeah that would be great! would be incredibly useful

[redstonedash]

just curious, what would happen if this blob was loaded into another program that didn't have the same functions? that would be undefined behavior correct?

[redstonedash]

I see that the snapshot feature has been pushed. about how hard would implementing ecs_snapshot_to_blob() be? is it contiguous in memory and lack pointers or no? i was trying to read through it's implementation but couldn't get a handle on it

[redstonedash]

@SanderMertens ping for visibility.

[SanderMertens]

@redstonedash You will be able to load a blob into a different world that has a different set of systems. Since data and logic are separated in ECS, there will be no issues when you try to do this.

I'm still fleshing out the API, as ideally I would like it to be possible to "stream" a blob in chunks of a fixed size. This would let you save blobs to a file / send them over the network without having to allocate the entire blob in RAM (which could be quite large).