Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve global stats calculation #9662

Closed
nijel opened this issue Aug 1, 2023 · 1 comment · Fixed by #10090
Closed

Improve global stats calculation #9662

nijel opened this issue Aug 1, 2023 · 1 comment · Fixed by #10090
Assignees
@nijel
Labels
enhancement Adding or requesting a new feature.
Milestone
5.1

Comments

@nijel
Copy link
Member

nijel commented Aug 1, 2023

Describe the problem

The global stats update is time-consuming and happens in situations where exact counts are not really necessary.

Describe the solution you'd like

Updating should be pushed by changes instead of always purging all caches.

  • invalidate at translation and component level would only invalidate their caches (with childs)
  • invalidation would schedule cache recalculation in the background
  • the background recalculation would update higher level stats as well

This will make high-level stats up-to-date without recalculating them from the request.

Describe alternatives you've considered

No response

Screenshots

No response

Additional context

This makes rendering views depending on global stats slow on busy sites as it has to update the stats in nearly every request, see https://sentry.weblate.org/organizations/weblate/performance/summary/?project=2&query=&referrer=performance-transaction-summary&statsPeriod=7d&transaction=%2Fexports%2Fog.png&unselectedSeries=p100%28%29&unselectedSeries=avg%28%29
@nijel nijel added the enhancement Adding or requesting a new feature. label Aug 1, 2023
@nijel nijel added this to the 5.1 milestone Sep 12, 2023
@nijel nijel self-assigned this Oct 3, 2023
nijel added a commit to nijel/weblate that referenced this issue Oct 3, 2023
@nijel
stats: keep stats up-to-date instead of invalidating
19fc5c0 
Always calculate up-to-date stats for a translation on a change, parent
stats are updated automatically in the background.

There is no explicit cache invalidation anymore to make sure some stats
are always present (and thus nearly no request on higher level will
force calculating them).

This might cause that parent stats are showing outdated number for very
short time, but this should cause no harm.

Fixes WeblateOrg#9662
@nijel nijel mentioned this issue Oct 3, 2023
Merged
5 tasks
nijel added a commit to nijel/weblate that referenced this issue Oct 3, 2023
@nijel
stats: keep stats up-to-date instead of invalidating
54815fc 
Always calculate up-to-date stats for a translation on a change, parent
stats are updated automatically in the background.

There is no explicit cache invalidation anymore to make sure some stats
are always present (and thus nearly no request on higher level will
force calculating them).

This might cause that parent stats are showing outdated number for very
short time, but this should cause no harm.

Fixes WeblateOrg#9662
nijel added a commit to nijel/weblate that referenced this issue Oct 3, 2023
@nijel
stats: keep stats up-to-date instead of invalidating
69377cc 
Always calculate up-to-date stats for a translation on a change, parent
stats are updated automatically in the background.

There is no explicit cache invalidation anymore to make sure some stats
are always present (and thus nearly no request on higher level will
force calculating them).

This might cause that parent stats are showing outdated number for very
short time, but this should cause no harm.

Fixes WeblateOrg#9662
nijel added a commit to nijel/weblate that referenced this issue Oct 4, 2023
@nijel
stats: keep stats up-to-date instead of invalidating
4208485 
Always calculate up-to-date stats for a translation on a change, parent
stats are updated automatically in the background.

There is no explicit cache invalidation anymore to make sure some stats
are always present (and thus nearly no request on higher level will
force calculating them).

This might cause that parent stats are showing outdated number for very
short time, but this should cause no harm.

Fixes WeblateOrg#9662
nijel added a commit to nijel/weblate that referenced this issue Oct 4, 2023
@nijel
stats: keep stats up-to-date instead of invalidating
efe44c0 
Always calculate up-to-date stats for a translation on a change, parent
stats are updated automatically in the background.

There is no explicit cache invalidation anymore to make sure some stats
are always present (and thus nearly no request on higher level will
force calculating them).

This might cause that parent stats are showing outdated number for very
short time, but this should cause no harm.

Fixes WeblateOrg#9662
nijel added a commit that referenced this issue Oct 4, 2023
@nijel
stats: keep stats up-to-date instead of invalidating
627c4fb 
Always calculate up-to-date stats for a translation on a change, parent
stats are updated automatically in the background.

There is no explicit cache invalidation anymore to make sure some stats
are always present (and thus nearly no request on higher level will
force calculating them).

This might cause that parent stats are showing outdated number for very
short time, but this should cause no harm.

Fixes #9662
@github-actions
Copy link

github-actions bot commented Oct 4, 2023

Thank you for your report; the issue you have reported has just been fixed.

  • In case you see a problem with the fix, please comment on this issue.
  • In case you see a similar problem, please open a separate issue.
  • If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nijel nijel

Labels
enhancement Adding or requesting a new feature.
Projects
None yet
Milestone
5.1
Development

Successfully merging a pull request may close this issue.

stats: keep stats up-to-date instead of invalidating nijel/weblate
1 participant
@nijel