Skip to content

538-srv178-web115

@jnu jnu tagged this 13 Jan 16:04
- Script to generate local threads to simulate load
 - Optimized all threads query path for admins

The "all threads" query is known to be slow (#189). We bumped the
timeout on prod again today to 30s and are seeing latency of ~20s when
admins load the page.

Currently, the endpoint requests all visible threads from OpenFGA and
uses that to filter the database to pull info for those threads. This
works fine for most users, who have access to maybe dozens of threads, a
very low percentage of all threads. For admins, this type of query is
not only slow (searching ~10,000s of threads currently) but also
inefficient: since admins can see the large majority of threads in the
database, filtering by visible IDs doesn't ultimately remove that many
from the resultset.

With that in mind, I added a fast query path for admins where we flip
the procedure: first pull a page of threads from the DB, then filter it
for visibility, and repeat until we have N threads (or run out of
pages).

In my local testing, this is much faster. At least an order of magnitude
faster, brings response times to the hundreds of milliseconds range.
However the performance on prod will ultimately depend on the
distribution of thread visibility. For example, in a deployment with
entirely private groups, this approach will be much slower than the
original query. Locally, it took ~1000 sequential threads in a private
group for this approach to be as slow as the status quo. So it's very
likely that this will be an improvement.

It would be nice in the future to add more heuristics to determine when
to use this new query path rather than simply admin vs non-admin.
Assets 2
Loading