Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Bug 1578057: update locale's latest_translation for visible projects only #1366

Merged
merged 8 commits into from Sep 4, 2019
Merged

Conversation

MikkCZ
Copy link
Contributor

@MikkCZ MikkCZ commented Sep 1, 2019

As of now, translation changes to invisible projects (e.g. system project like tutorial) do update the latest activity column shown on all teams dashboard. That's kind of confusing, since you cannot click through to that system project from any dashboards. As the name suggests, the project is invisible there.

This change omits updating latest_translation for a locale if the translation is being saved for a system project. Note, that latest_translation is still updated in all other tables, including project_locale, because the dashboard of an invisible project is still accessible via direct link or from its translation view and the latest activity should be displayed there.

I have written a small unit test for the behaviour. Also I have tested the patch manually in local setup. Exact steps to reproduce can be found in the bug, though in the local setup you need to add at least one visible project, so the teams dashboard can show something interesting.

@MikkCZ
Copy link
Contributor Author

MikkCZ commented Sep 1, 2019

Hi @mathjazz, can you suggest me a suitable reviewer, please?

instances = [project, locale, translatedresource]
instances = [translatedresource, project]

if Project.objects.visible().filter(pk=project.pk).first():
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a better way to do this in Django? I am not that familiar with the DB layer, but it seems to me there are three SELECTs to the database first (TranslatedResource.objects., Project.objects. and utils.get_object_or_none) to decide whether to update the three tables or not.

Maybe it could be rewritten to something that issues just three UPDATE statements instead, with the proper WHERE clause to ensure the right rows are (not) updated.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's enough to check if the project isn't a system project here like this:

if not project.system_project:

That's also what we do in adjust_all_stats() to make sure we don't update stats on /teams when system projects change.

The Project.visible() method would also check if the project is available (i.e. has been synced for the first time), but we don't need that since in this case the Translation.save() method (which calls update_latest_translation()) would not be callable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, that's one SELECT down.

Do you think I should also get rid of the get on TranslatedResource and ProjectLocale? There wouldn't be the for-loop, just X.objects.filter(...).update(...).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to save another DB request by getting rid of TranslatedResource.objects.get() if you pass translatedresource to self.update_latest_translation() in save().

I'm not sure about the ProjectLocale.

You can use this patch to debug SQL performance and compare the number of DB calls:
https://gist.github.com/mathjazz/92d2f6a36c968e8c1f6496f5e5aa0318

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to do it via QuerySet update() API. update_latest_translation is sometimes called from a context, where the translated resource is not present, or at least not directly.

@mathjazz mathjazz self-requested a review September 2, 2019 09:59
@mathjazz
Copy link
Collaborator

mathjazz commented Sep 2, 2019

Hi @mathjazz, can you suggest me a suitable reviewer, please?

Thanks for filing a bug and creating a PR! I'll be happy to review!

Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works very well! See how you can simplify the condition and save a DB request.

instances = [project, locale, translatedresource]
instances = [translatedresource, project]

if Project.objects.visible().filter(pk=project.pk).first():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's enough to check if the project isn't a system project here like this:

if not project.system_project:

That's also what we do in adjust_all_stats() to make sure we don't update stats on /teams when system projects change.

The Project.visible() method would also check if the project is available (i.e. has been synced for the first time), but we don't need that since in this case the Translation.save() method (which calls update_latest_translation()) would not be callable.

pontoon/base/tests/models/test_translation.py Show resolved Hide resolved
Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done!

I'll keep the PR open in case you want to reduce the number of DB requests.

@MikkCZ
Copy link
Contributor Author

MikkCZ commented Sep 3, 2019

Thanks for the review. Please also check the last two commits. If you like them and CI is green, I intend no more changes.

EDIT: Interesting, the same tests failed as without the last commit, but I cannot reproduced it locally. Even the build on the branch in my fork passed.

Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent job! Seems like this shaves off 5 DB requests when the 1st translation is added, and a whooping 13 requests when another translation already exists!

AFAICT everything works OK, but I need fresh brain to debug what's the problem with the tests.

pontoon/base/models.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@mathjazz mathjazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thanks so much!

The tests are passing now, so it could have been a caching issue which occurs from time to time.

I took a snapshot of update_translation performance numbers on New Relic and I'll check again after deployment.

pontoon/base/models.py Show resolved Hide resolved
@mathjazz mathjazz merged commit dd8c40b into mozilla:master Sep 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants