Enable World Swapping

[UkoeHB]

What problem does this solve or what need does it fill?

There are two common patterns in many Bevy applications.

1. Game menu: Game menus are separated from game worlds by States. Typically it's a global enum with WorldState::Menu and WorldState::Game variants. When going WorldState::Game -> WorldState::Menu, a lot of cleanup is necessary. See issue Bevy internal entities need public visible marker. #12852 for a great example.
2. Asset to resource: When an app starts up, some resources are loaded from assets and inserted into the world. The world often uses a global States enum to separate the loading phase from runtime systems. This pattern is especially distinct for headless apps where AssetServer is only used to load config resources on startup. After loading is done, the global States only exist to filter out the initial loading systems, which stay in the app.

Fundamentally, Bevy app design is currently monolithic. One world per application, one world to control all logic and data. Architecturally, monolithic worlds are restrictive and require various hacks around States and comprehensive cleanup systems to artificially construct distinct 'sub-worlds' within a single World. Even when you hack together internal 'sub-worlds', there is risk of data leakage between partitions, and it becomes difficult to clean up dead 'sub-worlds' entirely and reliably.

This problem is unique to an ECS engine because the ECS is a kind of data-control singleton with mostly manual allocation management (entity spawn/despawn, resource insert/remove).

What solution would you like?

Instead of one world per application, I'd like to use multiple worlds per application. One world at a time can control the event loop and windows, but worlds can do hand-offs when the user swaps between partitions of the application.

With a multi-world-swapping design, the menu/game separation becomes much more straightforward since game worlds can be cleaned up completely by just dropping them (and only recovering exactly those resources you want to recover).

Similarly, the load -> headless app sequence becomes much cleaner and more concise.

It turns out world-swapping is quite straightforward to implement, and only a few minor changes are required to the Bevy codebase to achieve the necessary level of abstraction. I implemented world-swapping in this experimental crate using a branch of Bevy that has the changes in the following small PRs:

• Add Clone to WinitSettings to facilitate moving windows between foreground worlds. Add Clone to WinitSettings #12787
• Export CachedWindow so swapped-in worlds can be synced with windows it doesn't know about yet. Export CachedWindow #12785
• Split out a section of WinitPlugin into WinitCorePlugin so secondary worlds can hook up to winit without creating an event loop. Refactor non-eventloop setup logic from WinitPlugin into WinitCorePlugin #12786
• Refresh SystemStates used by the winit event loop when the primary World running in the loop changes. Synchronize system states to world changes in the winit loop #12788
• Tie access kit root nodes to winit window ids instead of entity ids, so the root nodes can be transferred between worlds. Tie accesskit root nodes to winit's WindowId instead of window entity #12799
• (optional) Don't make AssetServer in AssetPlugin if an AssetServer already exists. This makes it easier to reuse AssetServer between worlds. It is optional and I might just close this because it sounds like in the future AssetServer will be more deeply embedded in the ECS. Don't make an AssetServer in AssetPlugin if one already exists #12790

Note that the world-swapping approach I implemented uses separate render worlds for each 'swappable' world. The surface area between swappable worlds and an App is quite small. Swapping is mostly transferring window information and hooking up foreground worlds to the event loop.

What alternative(s) have you considered?

A few solutions have been proposed:

• Dynamic plugins
• Entity namespacing
• Just keep hacking things together

While it's certainly useful to have more ECS tools, unfortunately dynamic plugins and entity namespacing are both far away from Bevy today, and are far more complex to implement than world swapping. They are also probably more complex to actually use (and use correctly) in practice. World-swapping is easy to use, and low-cost to support in Bevy.

Additional context

• Bevy internal entities need public visible marker. #12852

[mockersf]

World swapping seems a very counter intuitive and a bad idea to me if you apply it to the whole world. it would make much more sense to me to limit it to some kind of subworld.

As we move to more things "as entity", like windows, gamepads, input devices, ... this feels less and less a good idea

[UkoeHB]

World swapping seems a very counter intuitive and a bad idea to me if you apply it to the whole world.

Can you elaborate? I realize that 'one world' is the common sense right now, but honestly it's quite simple and straightforward. Putting sub-worlds inside a world seems more of an ECS anti-pattern than having actually separate worlds (for the use-cases where it makes sense).

it would make much more sense to me to limit it to some kind of subworld.

Implementing subworlds of any kind is, as far as I know, not even on anyone's roadmap. World-swapping works today with basically trivial code changes.

As we move to more things "as entity", like windows, gamepads, input devices, ... this feels less and less a good idea

The interface between external devices and Bevy worlds is very small, especially compared to the typical volume of code in an application. I don't see these as real blockers, just inconveniences that can be handled with modest effort.

[cart]

First, I see why this is appealing, and it would be cool. Know that I'm on board for allowing world swapping (the simple "change active world that we are currently running app logic on") . In effect, we already do.

But I'm very opposed to "supporting" (as in, changing how we do things or constraining ourselves in order to make it a first class experience) the "swapping worlds as scenes" pattern.

A World (as we all know) is a "container of all of the things required to make an app function". The proposal (as I interpret it) is to allow someone to create a new World, fill it with entities, and then transfer everything required by all Bevy Engine plugins and 3rd party plugins to function without breaking anything. That "set of things" is undefined. We'd essentially be creating a new "thing" people need to remember to support: ensuring the data is registered in a way that it is transferred. And then ensuring that it is built in a way that it can be transferred (ex: entity references don't break).

Also notably, porting any piece of architecture to being "entity backed" (ex: Assets as Entities, Schedules as Entities, etc) introduces a new set of "porting problems".

I guarantee the TODO list in this issue is not exhaustive. We would forever be identifying new issues upstream (and in 3rd party crates). And we would forever feel the need to tie our hands behind our backs to avoid nasty porting issues (for example, avoiding the use of Entity ids in core systems, which is the opposite direction that Bevy is currently headed in).

This is a path we could take. But we would be sacrificing a lot to make it happen. Instead I think we should be investing in things like allocation reuse (ex: allocate tables of entities on separate threads that can be directly and cheaply merged into the main World storage), world partitions / subworlds, etc.

[UkoeHB]

Know that I'm on board for allowing world swapping (the simple "change active world that we are currently running app logic on")

As I understand it, this is what my worldswapping implementation does. I'm not actually sure what you mean by the "swapping worlds as scenes" pattern.. More comments below...

A World (as we all know) is a "container of all of the things required to make an app function".

This is not quite the case, and it's why I thought world swapping could be achieved in the first place. A world does not A) contain the state of the event loop, B) include a render world inside it (this is not super important for world swapping, but it's true anyway).

The proposal (as I interpret it) is to allow someone to create a new World, fill it with entities, and then transfer everything required by all Bevy Engine plugins and 3rd party plugins to function without breaking anything. That "set of things" is undefined. We'd essentially be creating a new "thing" people need to remember to support: ensuring the data is registered in a way that it is transferred. And then ensuring that it is built in a way that it can be transferred (ex: entity references don't break).

World-swapping as implemented is actually creating an entirely new App and extracting its main world and render subapp. When swapping worlds, only transfer those resources/entities that are device-locked (windows, input devices). 99% of normal plugins don't need to care about this, and I explicitly don't intend to support more than the bare minimum of resource/entity synchronization.

I guarantee the TODO list in this issue is not exhaustive. We would forever be identifying new issues upstream (and in 3rd party crates). And we would forever feel the need to tie our hands behind our backs to avoid nasty porting issues (for example, avoiding the use of Entity ids in core systems, which is the opposite direction that Bevy is currently headed in).

I feel like this exaggerates the cost to support this in Bevy. Can you give an example of a feature that would cause nasty porting issues? Bevy is old enough, and the landscape of what goes into a game engine's design is established enough, that it should be possible to come up with an example if one exists (not trying to be rude, it's just I consider this a major UX improvement and don't want to back down without a clear reason).

Instead I think we should be investing in things like allocation reuse (ex: allocate tables of entities on separate threads that can be directly and cheaply merged into the main World storage), world partitions / subworlds, etc.

From my point of view as a user, these items aren't even on anyone's dev roadmap (as far as I know), which means the state of things is unlikely to improve any time soon (if ever). And to be clear, I personally don't have the time or motivation to invest 1+ months in a big ECS project.

[Veritius]

Would sub-worlds be fully self contained? Could an entity exist in multiple worlds simultaneously, or only one at a time? The former would be good for replication and render worlds, but data leakage could be a problem as well.

[cart]

As I understand it, this is what my worldswapping implementation does. I'm not actually sure what you mean by the "swapping worlds as scenes" pattern.. More comments below...

To be clear: I am in favor of continuing to allow people to swap out full World references. Just that functionality, not the accompanying "data transfer / compatibility fixups" being proposed here. When I talk about "swapping Worlds as scenes", I'm talking about things like your "Game Menu World vs Game World" swapping scenario. The "game menu world" is serving essentially the same role as a "game menu scene" would, which is why I'm making that comparison.

This is not quite the case, and it's why I thought world swapping could be achieved in the first place. A world does not A) contain the state of the event loop, B) include a render world inside it (this is not super important for world swapping, but it's true anyway).

Sure. I will concede that point (although it wasn't really the one I was trying to make). That doesn't really change my mind about anything.

World-swapping as implemented is actually creating an entirely new App and extracting its main world and render subapp. When swapping worlds, only transfer those resources/entities that are device-locked (windows, input devices). 99% of normal plugins don't need to care about this, and I explicitly don't intend to support more than the bare minimum of resource/entity synchronization.
I feel like this exaggerates the cost to support this in Bevy. Can you give an example of a feature that would cause nasty porting issues? Bevy is old enough, and the landscape of what goes into a game engine's design is established enough, that it should be possible to come up with an example if one exists (not trying to be rude, it's just I consider this a major UX improvement and don't want to back down without a clear reason).

Your porting list alone is already concerning. The fact that this cross-cuts into Winit configuration, Bevy Window abstractions, Asset Server (with additional implications for efforts such as assets as entities), AccessKit, etc signals to me this is not a "one off" issue to be solved once, but rather a new set of constraints that applies to a wide variety of things across contexts.

Bevy is built in such a way that its built in features aren't "special" and are generally replaceable. The problems manifesting themselves in the areas above inherently aren't just "internal" issues to be abstracted over. If it requires significant changes to the structure of internal systems, that is a "userspace / 3rd party plugin" issue as well.

Given what I've seen I'm highly confident the implications extend into 3rd party plugins. I don't have the time right now to undergo the research project of surveying the ecosystem for incompatibilities. But I would be very surprised if they aren't there. I think places like bevy_sdl2 and bevy_kira_audio are probably a good place to start.

But I want to stress that the changes being suggested above are already enough for me to be concerned. The "manual fixup" pattern being embraced here feels like the fixes would be constantly changing and highly subject to breakage as Bevy features evolve over time. This is a new set of constraints that we would be enforcing and fixing for the rest of Bevy's lifetime. That is a big ask.

Additionally, the amount (and scope) of metadata being stored in World is planned to increase over time (relations, scenes, dynamic/scripted components, schedules as entities, assets as entities, etc). Having an unbounded number of duplicates of metadata across worlds and an unbounded amount of data to transfer back and forth between them feels like we're going in the wrong direction.

From my point of view as a user, these items aren't even on anyone's dev roadmap (as far as I know), which means the state of things is unlikely to improve any time soon (if ever). And to be clear, I personally don't have the time or motivation to invest 1+ months in a big ECS project.

Yup I agree that this is unlikely to happen soon, and may never happen.

[UkoeHB]

Closing this as a pretty clear won't-fix. I will be maintaining a thin fork of Bevy indefinitely to support bevy_worldswap and my personal projects.