Rooms

[Shatur]

Sometimes you replicate only part of the world. Commonly used for things like fog of war or card games.
To solve this @koe from Bevy discord server suggested to implement rooms.
Assign each entity a room ID (inspired by naia's rooms). Each client is a member of an arbitrary number of rooms (e.g. using a hashset of ids). By default entities are in room ‘0’ which means global (or maybe have no room IDs). All other room ids are user-defined. Replicon just needs a resource to track client room membership, and some updates to the replication logic to only replicate an entity for a given client if they are in the same room (and also to ‘despawn’ an entity if it stops being visible to a client).

[UkoeHB]

naia uses a Room resource which allows entities to be members of multiple rooms. At this time I think it would be best to just duplicate naia's design (with some adjustments for Room-based events).

[Shatur]

Yeah, this is exactly what @koe suggested.

[UkoeHB]

Here is a sketch of a solution for rooms (incomplete: it doesn't specify how to handle 'gained/lost visibility').

- entities
    - Replication{ current_room: Room, previous_room: Room }
        - set_room(Room): sets current room (public API)
        - set_prev(): sets previous = current (crate API)
- events
    - field: Room
- clients
    - RoomCache
        - prev_rooms: [ room id : HashSet<[client id]> ]
        - joined_rooms: [ client id : HashSet<room id> ]
        - left_rooms: [ client id : HashSet<room id> ]

- client changes rooms
    - join: add to add to RoomCache::joined_rooms
    - leave: add to RoomCache::left_rooms, remove from RoomCache::joined_rooms

- entity changes rooms
    - call Replication::set_room([new room])
        - will over-write any previous room changes in this tick
    - note: entities must always be in a room (can have a 'null room' for invisible entities)

- replication helpers
    - enum ClientVisibility
        - Gained
        - Lost
        - Maintained
        - None

    - client/entity previously in same room
        - entity prev room == contained client previously
    - client in entity current room
        - short circuit checks
            - did entity change rooms?
            - are joined rooms and left rooms empty?
            - are joined rooms and left rooms empty for a specific client?
        - entity current room
            - in client's joined rooms
            - in client's prev rooms and NOT in client's left rooms

    - fn entity_visibility(entity current room, entity prev room, room cache, client list) -> impl Iterator<Item = (client id, ClientVisibility)> + 'static
        - iterate over client list
            - ClientVisibility::Gained
                - client in entity current room
                - client was not previously in entity prev room
            - ClientVisibility::Lost
                - client not in entity current room
                - client was previously in entity prev room
            - ClientVisibility::Maintained
                - client in entity current room
                - entity was previously in entity prev room
            - ClientVisibility::None
                - client not in entity current room 
                - client not previously in entity prev room

- replication
    - for each entity
        - get client visibility
            - ClientVisibility::Gained
                - set ack tick to zero ??
                - normal replication
            - ClientVisibility::Lost
                - add entity to despawn tracker ??
            - ClientVisibility::Maintained
                - normal replication
            - ClientVisibility::None
                - skip client

- cleanup
    - after replicating an entity, set_prev() on its replication component
    - after replicating everything
        - move RoomCache::joined_rooms to RoomCache::prev_rooms
        - remove RoomCache::left_rooms from RoomCache::prev_rooms

- events (do this after room cleanup)
    - send event to all clients in the event's room (in RoomCache::prev_rooms[event room id])

- bonus idea
    - visibility alias: clients register to one visibility alias, and all clients behind that alias view the same rooms (whatever rooms the alias is in)
        - pro: iterate over aliases instead of clients for identifying visibility changes
        - con: can't replicate private user information/events (unless clients can have multiple aliases? but that defeats the purpose)

[UkoeHB]

In my proposal above, 'gained visibility' and 'lost visibility' are only detected in the tick where they occur. Since the replication diff from that tick may fail to reach the client, we need to re-replicate those spawns/despawns until they have been acked. To do that I think we need a spawn tracker that caches diffs for entities that each client gained visibility on. The logic for that cache will be a little hairy to account for entity and visibility changes that occur after entities were cached. [edit: caching diffs for clients instead of recreating diffs should be a good way to handle this (we need to cache diffs for sync events)]

We also need to be careful about visibility-related spawns and despawns that occur between client acks. Those events cannot be 'cross-merged' (i.e. cancel out a despawn with a subsequent spawn) since a visibility despawn sent in one tick may be acked by the client and so the following visibility spawn needs to also be sent to resurrect the entity (multiple duplicates can be merged though, we only need the latest spawn/despawn). Additionally, spawns and despawns must be strictly ordered on the client to ensure any sequence of visibility spawns/despawns will have the correct final result.

[UkoeHB]

Another problem to consider: a child entity should only be visible to a client if it and all its parents are also visible to that client.

[ActuallyHappening]

I would prefer a more flexible, sole server approach, where a function taking mutable world access, the player id and a specific entity returns a bool whether to replicate to client or not.

This way I can encode "chunking" (like minecraft does it, I presume) however I want to.

From my perspective, not having contributed to bevy_replicon much, this seems the easiest way from the consumers of this library's POV to not replicate the entire world.

This is a slow implementation, but I think configuring a function (signature) to decide which entities are replicated is the most powerful, flexible and easiest to integrate approach

[Shatur]

Rooms have the same amount of flexibility and more ECS-friendly.
You just create a system that modifies this component to decide what to replicate. This system can have access to anything you want in the world depending on your use case. And what is more important is that it can run in parallel with other logic.

[ActuallyHappening]

Yeah you're right, its more ECS friendly.
My use case would involve a room for each player, and every FixedUpdate frame updating each entity to be in all the rooms of players suitably nearby. Thus, you would need a data structure flexible enough for one room per player, and able to handle every frame many entities (on the server side) changing rooms.

Can't wait until this is finalised!

[UkoeHB]

My use case would involve a room for each player, and every FixedUpdate frame updating each entity to be in all the rooms of players suitably nearby. Thus, you would need a data structure flexible enough for one room per player, and able to handle every frame many entities (on the server side) changing rooms.

The design we are considering assigns one room per entity. For this use-case you'd chunk the entities into rooms (as small as one or zero entities per room), then adjust which rooms each player is a member of.

[Shatur]

Working on rooms right now.

In the proposed solution, we essentially assign an ID (room) to each object, and then specify which IDs each client sees.
But what if we use an existing entity ID (Entity) and just store it for each client? This approach have the same amount of flexibility, but I would expect it to be much faster due to less amount of lookups. Also much easier implement (users only modify entities that each client sees, entity IDs remain the same unlike with entity room). To make it more convenient to use, we should also have a policy switch (all entities, blacklist and whitelist). I will try to prototype it first and see how it goes.

[UkoeHB]

But what if we use an existing entity ID (Entity) and just store it for each client?

This would be one room per entity. I think it is an API question. With distinct rooms you can separately assign entities to rooms and assign clients to rooms, with per-entity visibility you have to explicitly assign/remove entities from clients (which I imagine would be way more bug prone and laborious).

[Shatur]

I wouldn't say "way more", but I agree that for some games it would be less convenient.
However the approach I mentioned is faster. It's even zero-cost when you don't use visibility! If it's too barebones for some games, users can easily extend it.
And it's also easy to implement. I played with it this morning and really how the API turned out. Maybe I will finish it quickly, show it and we will discuss?

[UkoeHB]

And it's also easy to implement. I played with it this morning and really how the API turned out. Maybe I will finish it quickly, show it and we will discuss?

Ok let's see how it looks.

If it's too barebones for some games, users can easily extend it.

We should make sure the 'how to extend it' story is figured out before releasing this. And add examples/tests for the various scenarios:

• One entity, one client.
• Few entities, many clients.
• Many entities, few clients. Few (logical) rooms vs many (logical) rooms.
• Many entities, many clients. Few (logical) rooms vs many (logical) rooms.
• Etc?

[Shatur]

We should make sure the 'how to extend it' story is figured out before releasing this.

Agree, consider this as a quick prototype.

And add examples/tests for the various scenarios:

I want to showcase the working design first.
I will sure add test for any possible case when we agree on the design. I like tests, we have quite a high test coverage for a reason :)

[Shatur]

Let's move discussion about possible rooms API on top from the PR here.
You said:

 think it can be done with one custom SystemParam (instead of an entity component) called RepliconRooms with this core API:

    rooms.join(client id, room id)
    rooms.leave(client id, room id)
    rooms.add_entity(entity, room id)
    rooms.remove_entity(entity, room id)

I'm not sure how to integrate it with events. Maybe a bespoke solution is required (a distinct events API that wraps bevy_replicon, like RoomWriter or some such).

I think that a SystemParam is a way to go.
Alternatively it could be a Room component and a ClientRooms resource.

About events. How about to provide an extension trait for App that registers a room-based event (like App::add_room_server_event) and under the hood registers an event like ToClients<RoomEvent<T>> (with an alias to something like ToRooms<T>) with custom sending and receiving systems?

[UkoeHB]

How about to provide an extension trait for App that registers a room-based event (like App::add_room_server_event) and under the hood registers an event like ToClients<RoomEvent> (with an alias to something like ToRooms) with custom sending and receiving systems?

I like this idea :)