Skip to content
This repository has been archived by the owner on May 20, 2024. It is now read-only.

Backend performance improvements #2667

Closed
nicksellen opened this issue Jul 25, 2023 · 2 comments
Closed

Backend performance improvements #2667

nicksellen opened this issue Jul 25, 2023 · 2 comments
Labels

Comments

@nicksellen
Copy link
Member

Due to #2666 we have silk profiling available in prod again, I started noticing a few of the slower requests.

/api/bootstrap/ is slowed down as we have not followed the versatileimagefield performance steps on https://django-versatileimagefield.readthedocs.io/en/latest/improving_performance.html to create thumbnails post_save. At the moment it has to do a lot of checking to see if the thumbnails exist yet (that info is stored in redis cache). See profile.

The bootstrap sql queries could be improved too. Fetching less data would make a lot of sense, but that might need changes to the frontend data fetching to fetch some things on demand. At the moment we load all group infos, even if groups are inactive.

@nicksellen
Copy link
Member Author

Sentry also seems to log slow SQL queries now, see https://karrot-dev.sentry.io/issues/?project=147126

@nicksellen
Copy link
Member Author

I think I can call this resolved now, from various adjustments over the last few weeks:

  • turning off on demand image creation for versatileimagefield
  • switching to orjson json serializer
  • unix socket connections for postgres and redis
  • connection pooling for postgres
  • psycopg upgrade claims some performance improvement

It would always be nice to be faster, /api/bootstrap/ is still too slow if you are part of many larger groups, so future improvements could still be:

  • send less data for bootstrap endpoint
  • perhaps more on-demand data fetching, less relying on having all the data in memory (groups, places, and users come to mind)
  • also, can switch to SSDs on the server soon perhaps, which may or may not make a lot of different (as most stuff is in memory anyway)
  • some query/index optimizing could make sense
  • statistics endpoint is very slow, although probably OK for that use case
  • available activities is a bit slow, as it does a more complex participant type query check

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant