Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache etags #7107

Conversation

nucleogenesis
Copy link
Member

@nucleogenesis nucleogenesis commented Jun 19, 2020

Summary

After further discussion with @rtibbles this PR has changed to be removing the etag decorator we were looking at altogether.

-- below this is the original comment --

Updated the etag generation decorator function so that it also uses the rendered HTML which includes everything in plugin_data.

Is there a better or easier way to get plugin_data only?

I hate that I call the view_func twice and feel like that could be a problem - I'll come back to it.

Reviewer guidance

  • Load a page and check the root request URL in your devtools network tab.
  • Check its etag
  • Go to the kolibri_plugin.py file for the plugin associated with the URL you're testing. Add a key to the plugin_data
  • Reload the page and see that the same resource's etag has changed
  • Reload again and see that it has not changed

References

#6785


Contributor Checklist

PR process:

  • PR has the correct target branch and milestone
  • PR has 'needs review' or 'work-in-progress' label
  • If PR is ready for review, a reviewer has been added. (Don't use 'Assignees')
  • If this is an important user-facing change, PR or related issue has a 'changelog' label
  • If this includes an internal dependency change, a link to the diff is provided

Testing:

  • Contributor has fully tested the PR manually
  • If there are any front-end changes, before/after screenshots are included
  • Critical user journeys are covered by Gherkin stories
  • Critical and brittle code paths are covered by unit tests

Reviewer Checklist

  • Automated test coverage is satisfactory
  • PR is fully functional
  • PR has been tested for accessibility regressions
  • External dependency files were updated if necessary (yarn and pip)
  • Documentation is updated
  • Contributor is in AUTHORS.md

@nucleogenesis nucleogenesis added the TODO: needs review Waiting for review label Jun 19, 2020
@nucleogenesis nucleogenesis added this to the 0.14.0 milestone Jun 19, 2020
@codecov
Copy link

codecov bot commented Jun 20, 2020

Codecov Report

Merging #7107 into release-v0.14.x will increase coverage by 0.02%.
The diff coverage is 88.00%.

Impacted Files Coverage Δ
kolibri/core/decorators.py 62.50% <88.00%> (+1.89%) ⬆️
kolibri/plugins/default_theme/kolibri_plugin.py 92.30% <0.00%> (-7.70%) ⬇️
kolibri/core/tasks/utils.py 87.03% <0.00%> (-6.07%) ⬇️
...ibri/plugins/learn/assets/src/views/LearnIndex.vue 55.88% <0.00%> (-1.97%) ⬇️
kolibri/core/theme_hook.py 48.30% <0.00%> (-1.70%) ⬇️
kolibri/core/assets/src/core-app/client.js 10.90% <0.00%> (-1.10%) ⬇️
...libri/plugins/user/assets/src/views/SignInPage.vue 34.19% <0.00%> (-0.23%) ⬇️
...ugins/device/assets/src/modules/wizard/handlers.js 8.73% <0.00%> (-0.15%) ⬇️
kolibri/core/auth/management/commands/sync.py 0.00% <0.00%> (ø)
kolibri/plugins/device/assets/src/constants.js 100.00% <0.00%> (ø)
... and 44 more

@nucleogenesis
Copy link
Member Author

I'll add that I've gone through a bit and not seen that calling view_func twice on the request is causing any issues. I was concerned that it was going to duplicate SQL queries by calling whatever method/function is handling the view rendering.

@nucleogenesis
Copy link
Member Author

As @rtibbles pointed out - the original solution renders the view anyway. Ultimately Richard's suggestion is to remove the etag and to address optimization along these lines separately.

So this PR has changed to be removing the etag altogether.

@nucleogenesis nucleogenesis changed the title etag generated with plugin_data Remove etag from plugins Jun 23, 2020
@indirectlylit
Copy link
Contributor

What is the effect of CACHE_TIMEOUT?

  1. throttle to prevent regenerating etags more often than CACHE_TIMEOUT, even if DB has changed
  2. etag is regenerated on every request, but it gets thrown out after CACHE_TIMEOUT

Just noting that even if etags are regenerated on every request and render the full page, they still save network bandwidth by returning a 304 Not Modified with no body

setattr(_response, "response", None)

request = args[0]
etag = cache.get(request.path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought of one more thing in review:

I don't think it is huge likely, but might be good to prefix the request path with something to prevent accidental cache key collisions if something else is using the path:

So we can add this in the local scope:
CACHE_KEY_TEMPLATE = "SPA_ETAG_CACHE_{}"

Then to generate the cache key:
cache_key = CACHE_KEY_TEMPLATE.format(request.path)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed using the cache key template here :)

@nucleogenesis nucleogenesis changed the title Remove etag from plugins Cache etags Jun 30, 2020
Copy link
Member

@rtibbles rtibbles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good - have manually tested and confirmed that I get a 304, even if I wait > 15 secs, so the regeneration of the etag is working properly.

@rtibbles rtibbles merged commit 2842e43 into learningequality:release-v0.14.x Jun 30, 2020
@jonboiser jonboiser removed the TODO: needs review Waiting for review label Aug 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants