-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enhance Country Leaderboard Availability During Refresh #1099
Conversation
The initial approach to refreshing the country leaderboard, which involved deleting all entries at once, led to data unavailability and HTTP 502 errors for users during updates. We've revised the process to sequentially delete and rebuild entries for each country and time duration. This adjustment ensures that leaderboard data remains accessible to users while updates for other countries are in progress.
|
The updates per <country,timeDuration> tuple were slow, and I was unable to make measurable improvements to the query. The cumulative slowness per tuple took far too long for the existing scheduler, and we had to disable the country leaderboard. For the improvements I had tested but lacked improved results: These are relevant updates:
To clean up the above changes: ALTER TABLE status_actions DROP CONSTRAINT status_actions_users_osm_id_fkey;
drop index idx_status_actions_status;
drop index idx_status_actions_old_status;
drop index idx_status_actions_status_specific;
drop index idx_users_leaderboard_opt_out_false;
drop index idx_users_leaderboard_opt_out;
drop index idx_status_actions_osm_user_id;
delete from users where id = -1; And a sample query (generated by the API):
|
The initial approach to refreshing the country leaderboard, which involved deleting all entries at once, led to data unavailability and HTTP 502 errors (due to hitting the 75s http request timeout) for users during updates. We've revised the process to sequentially delete and rebuild entries for each country and time duration. This adjustment ensures that leaderboard data remains accessible to users while updates for other countries are in progress.