Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

room directory is very very slow over federation as that API doesn't let you paginate #3327

Closed
richvdh opened this issue Feb 27, 2017 · 14 comments
Closed

room directory is very very slow over federation as that API doesn't let you paginate #3327

richvdh opened this issue Feb 27, 2017 · 14 comments
Labels
A-Performance A-Room-Directory P1 S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Z-Backend Z-Synapse
Projects
First Impressions

Comments

@richvdh
Copy link
Member

richvdh commented Feb 27, 2017

It takes at least 90 seconds to get a room list
@richvdh richvdh mentioned this issue Feb 27, 2017
Closed
@ara4n ara4n added T-Defect S-Major Severely degrades major functionality or product features, with no satisfactory workaround P1 A-Room-Directory labels Feb 28, 2017
@ara4n
Copy link
Member

ara4n commented Feb 28, 2017 •
edited

@erikjohnston are you seeing this performance prob?

@ara4n ara4n added S-Critical Prevents work, causes data loss and/or has no workaround and removed S-Major Severely degrades major functionality or product features, with no satisfactory workaround labels Feb 28, 2017
@richvdh
Copy link
Member Author

richvdh commented Feb 28, 2017

(ftr, it has never been quick. I assume it pulls the entire room list from matrix.org before responding to the client)

@erikjohnston
Copy link
Member

yes, it is quite slow

@ara4n
Copy link
Member

ara4n commented Mar 9, 2017

...and it gets stuck issuing 50x's half the time :(

@ara4n ara4n mentioned this issue Mar 9, 2017
Closed
@ara4n ara4n changed the title room directory is unusable over federation room directory is unusable (especially over federation) Mar 9, 2017
This was referenced Mar 9, 2017
Closed
Closed
Closed
@ara4n
Copy link
Member

ara4n commented Mar 10, 2017

I've split this into two synapse bugs: matrix-org/synapse#1980 (general perf problems) and matrix-org/synapse#1981 (client-reader gets entirely wedged).

@lampholder lampholder modified the milestone: Riot web next release Mar 20, 2017
@ara4n
Copy link
Member

ara4n commented Mar 20, 2017

so @erikjohnston seems to have fixed the 'general perf problems' part of this, but the problem over federation still exists. Apparently this is because the publicRooms API simply doesn't paginate at all over federation.

@ara4n ara4n changed the title room directory is unusable (especially over federation) room directory is very very slow over federation as that API doesn't let you paginate Mar 20, 2017
@ara4n
Copy link
Member

ara4n commented Mar 20, 2017

apparently it should be usable but slow over federation (i.e. it's as slow as if you weren't paginating).

@ara4n ara4n added S-Major Severely degrades major functionality or product features, with no satisfactory workaround A-Performance and removed S-Critical Prevents work, causes data loss and/or has no workaround X-Release-Blocker labels Mar 20, 2017
@ara4n
Copy link
Member

ara4n commented Mar 20, 2017

reprioritising appropriately

@lampholder lampholder removed this from the Riot web next release milestone Mar 27, 2017
@ara4n ara4n mentioned this issue May 15, 2017
Closed
@martindale
Copy link

Current status?

@ara4n
Copy link
Member

ara4n commented Aug 1, 2017

still waiting for someone to make the directory api paginate over federation, sadly :(

@erikjohnston erikjohnston removed their assignment Oct 11, 2017
@ara4n
Copy link
Member

ara4n commented Sep 12, 2018

This is going to get really bad with the advent of hosted homeservers.

@ara4n
Copy link
Member

ara4n commented Sep 12, 2018

I'm also not sure whether the problem is really a question of pagination or caching problems. This might also improve if we ever land the matthew/stats branch, so that the public room directory can just be pulled straight out of a room_stats table rather than having to do more sophisticated shenanigans?

@martindale
Copy link

martindale commented Sep 13, 2018 via email •
edited

@lampholder lampholder added this to To Do in First Impressions Sep 26, 2018
@turt2live turt2live moved this from To Do to Can't Fix (see labels on issues) in First Impressions Dec 6, 2018
@turt2live
Copy link
Member

Searching matrix.org now that the directory is paginated is pretty quick.

First Impressions automation moved this from Can't Fix (see labels on issues) to Done Dec 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-Performance A-Room-Directory P1 S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Z-Backend Z-Synapse
Projects
No open projects
First Impressions
  
Done
Milestone
No milestone
Development

No branches or pull requests
6 participants
@martindale @turt2live @ara4n @richvdh @lampholder @erikjohnston