Module or synapse option : [Room Visibility] Publish the federated public rooms on the local room directory

[jdauphant]

Problem

Clients need to retrieve all rooms from all HS room directory

Solution

Each HS could have the room directories of all HS

Setting could be

- sync_room_directory_of_hs:
  - matrix.dinum.agent.tchap.gouv.fr

Notes

• When room is removed from room directory, we should remove it too

[giomfo]

@babolivier, we would like to retrieve from the user's HS all the Tchap public rooms (created on the multiple federated instances)

We succeeded to add in the agent.dinum HS room directory a public room created on agent.interieur, by changing the room visibility settings from a room member hosted on agent.dinum (this user was a room member (admin))

• the public room is now visible from both room directories (agent.interieur and agent.dinum)
• no local alias /address was required to succeed

Can we plan to set up a optional module in Synapse to group all the existing public rooms in a closed federation by using this approach ?

[babolivier]

After some exploration and discussion we're going to implement this in the form of server-side aggregation of public rooms. The idea is that homeservers would be able to use a list of homeservers to query the room directory of (provided either by the client, the server's config, or both), and when receiving a request on the /publicRooms endpoint a homeserver would query those homeservers as well as its local database and send an aggregation of the results it's received. We're deciding to do this in Synapse's mainline rather than a module as this change would be useful to the wider ecosystem. I'm happy to discuss this further if more information is needed.

[giomfo]

Thank you @babolivier for your feedback. This sounds like a good plan.
@jdauphant can you please confirm we can go for it?

[jdauphant]

Looks good to me.

@giomfo We only be able to retrieve this feature when we will be in full mainline?

[giomfo]

We only be able to retrieve this feature when we will be in full mainline?

yes

[anoadragon453]

@giomfo is Tchap planning to allow clients to select from a list the remote homeservers to aggregate results from? And if so, will that work be merged into upstream mainline Element clients?

Or will you be relying on a static homeserver config to choose the homeservers to query public rooms from?

[giomfo]

@giomfo is Tchap planning to allow clients to select from a list the remote homeservers to aggregate results from? And if so, will that work be merged into upstream mainline Element clients?

Or will you be relying on a static homeserver config to choose the homeservers to query public rooms from?

Hi @anoadragon453, the plan is the second option: "a static homeserver config".
@jdauphant can you please confirm the homeservers selection is not expected on the client side?

[anoadragon453]

FYI, I'm going to be operating under the assumption that homeserver selection is not expected on the client side for now.

[giomfo]

@jdauphant can you confirm the homeserver selection is not expected on the client side for this feature?

[jdauphant]

We will remove the switch on the web client.
If the switch is showe and selected, results from all HS will be served anyway on any selection.

[giomfo]

@jdauphant FYI here is the current status from @anoadragon453 :
"- A draft PR is up that will allow Synapse modules to make federation requests: Add a method for Synapse modules to make federation requests
I’m keeping this as a draft until everything is ready and I have tested it all works.

• A PR for allowing Synapse modules to take over the response for /publicRooms is in progress.

• A new Synapse module that implements both of the above new functionality is in progress.

I’m hoping to have a working implementation of the latter two done by the end of today, then it will just be waiting on review. It’s expected that the module API changes for this issue will also land in Synapse v1.79."