Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache survey results #3201

Merged
merged 10 commits into from
Feb 12, 2021
Merged

Cache survey results #3201

merged 10 commits into from
Feb 12, 2021

Conversation

willgearty
Copy link
Member

@willgearty willgearty commented Jan 29, 2021

This caches the survey review pages for admins (html and pdf versions!), making them load a lot faster once they've been run once. The rendering for each survey is cached independently, and we only ever update the caches when new responses are saved (just like how each class on the catalog is cached independently).

Examples of speed up (on my dev server, using "CPU Time"):

  • manage/one/two/surveys/review: ~20500 ms -> 500 ms (woohoo!)
  • manage/one/two/surveys/review (only one of two surveys updated): ~20500 ms -> 9160 ms
  • manage/one/two/surveys/review_pdf: hard to time since the debug toolbar is gone, but likely about the same speed up, if not faster because we aren't rendering the rest of the site

Fixes #3181.

@willgearty

This comment has been minimized.

@willgearty
Copy link
Member Author

willgearty commented Jan 29, 2021

OK, I've fixed the problem by mimicking the caching we do for the catalog (caching at the template level instead of the python level) to avoid any user-specific or form-specific parts. I'm still thinking about how to cache the page for teachers or for when we look up a teacher, but really those have always loaded pretty fast, so I'm not really sure we need to (~1500-2000 ms of CPU Time on my dev server).

@willgearty willgearty changed the title Cache survey review functions Cache survey review page for admins Jan 29, 2021
@lgtm-com

This comment has been minimized.

@willgearty
Copy link
Member Author

LGTM alerts are fine, that's how that file works (the unused imports make the caching work).

@willgearty
Copy link
Member Author

OK, I decided that we could do better. Now we cache each part of the survey results individually so we only refresh those parts when they have new relevant answers (the entire "individual questions" part is one part, and then each section is a separate part). So, for example, if a student fills out a survey for a particular section, only the cache for the part of the page that shows the results for that particular section gets refreshed.

Since this means we have a little more rendering to do each time the page is loaded, the minimum time to open the page is now a little longer, but the average time to open the page should be a lot shorter, especially during times when students are occasionally filling out surveys. This also means that we can cache the parts for teachers (and when we look up specific teachers), and the really cool part is that if a teacher has recently opened their page, the parts of the survey review for that teacher's sections will already be cached if an admin opens the full review (and vice versa).

Example times now:

  • manage/one/two/surveys/review: ~25000 ms -> ~1500 ms
  • manage/one/two/surveys/review (after submitting a section survey): ~1500 ms -> ~2300 ms
  • manage/one/two/surveys/review_pdf: hard to time since the debug toolbar is gone, but likely about the same speed up, if not faster because we aren't rendering the rest of the site
  • teach/one/two/surveys/review: ~1800 ms -> ~800 ms
  • teach/one/two/surveys/review_pdf: again, hard to estimate, but seems about as fast as /review
  • teach/one/two/onsitesurvey/review: ~3500 ms -> ~500 ms

Note: I believe the biggest component of loading the survey review page is creating and loading the histograms for numeric questions. The more histograms in a survey component, the longer that survey component will take to load if the cache is refreshed.

@willgearty willgearty changed the title Cache survey review page for admins Cache survey results Jan 30, 2021
@learning-unlimited learning-unlimited deleted a comment from lgtm-com bot Jan 30, 2021
@lgtm-com

This comment has been minimized.

@willgearty willgearty merged commit b26ef0d into main Feb 12, 2021
@willgearty willgearty deleted the cache-surveys branch February 12, 2021 02:58
willgearty added a commit that referenced this pull request Feb 18, 2021
willgearty added a commit that referenced this pull request May 27, 2021
* Initial docs for stable release 13

* Docs for #3116, #3117, and #3118

* Added docs about django upgrade

* Docs for #3128

* Docs for #3129, #3133, #3134, and #3137

* Docs for #3156 and #3153

* Docs for #3174, #3163, and #3184

* Docs for #3139, #3140, and #3141

* Docs for #3143, #3150, #3154, #3160, #3162, and #3168

* Docs for #3171, #3185, #3186, and #3188

* Docs for #3131 and #3189

* Docs for #3149 and #3190

* Docs for #3193, #3194, #3195, #196, and #3197

* Clarification

* Docs for #3192, #3201, #3209, and #2248

* Docs for #3204, #3212, #3214, #3205, 9fd073c, and #3226

* Docs for #3232, de5861c, #3231, #3233, #3234, #3237, #3238, and #3239

* Fix indent

* Docs for #3227 and #3235

* Add missing word

* spelling

* Docs for e57581f, #3255, #3253, #3257, and #3249

* Docs for #3254, #3260, and #3262

* Docs for #3263, #3272, #3240, #3264, #3266, and #3270

* clarifications

* Docs for #3283 and #3252

* Docs for #3288 and misc commits

* Docs for #3292, #3311, #3286, #3289, and #3279

* Docs for a377f0d; move note

* Docs for #3315, #3290, and #3322

* Docs for #3273 and #3317

* Final edits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Survey Results should be really be cached
2 participants