bug 1177264 - waffle switch for dumb doc urls (27.7% of app server time) #3331

Merged
merged 1 commit into from Jul 9, 2015

Projects

None yet

3 participants

@groovecoder
Member

Per the latest New Relic thread profile, kuma.core.urlresolvers.py.reverse takes up as much as:

  • 11% (in calls from wiki_url in base.html)
  • 6.7% (during calls from helpers.py.url in document_macros.html)
  • 4.7% (during calls from Document.get_absolute_url from document.html extra_head block)
  • 4% (during calls from Document.get_absolute_url from document.html lang_switcher block)
  • 1.3% (during calls from wiki_url in base.html footer_copyright_redesign block)
  • A couple other areas of 0-1%

kuma-reverse-profile

For a total of 27.7%+ of all back end execution time. The majority of the time spent in reverse is spent waiting on the memcache.py._get in the DocumentZoneURLRemapsJob. This job remaps any /docs/{slug} urls into /{zone-url-root} urls, if there's a zone mapping for the document. But, spot-checking some requests to /docs/{slug} show they re-direct to /{zone-url-root} anyway.

Effectively, our document.html templates spend 27.7% looking up possible url remaps for every doc link in the template, even though a plain /docs/{slug} link will work when the user visits them. I.e., we're incurring needless url remapping logic for every document view although there are only ~100 total zones, and most users will never even click any of the remapped links!

This change introduces a waffle switch that will allow us to short-circuit that logic in production to test:

  • The effect on back end execution time
  • The effect of the new redirects on user experience

If the effect on back end execution time is worth more than the effect on user experience, we can make this change permanent.

@jezdez jezdez was assigned by groovecoder Jul 9, 2015
@jezdez
Member
jezdez commented Jul 9, 2015

Nice analysis, I concur. r+

@jezdez jezdez merged commit 1e65aad into master Jul 9, 2015

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jezdez jezdez deleted the optimize-document-html-1177264 branch Jul 9, 2015
@jwhitlock
Contributor

My recollection is that Google sees internal redirects as damaged links and dings you as a poorly maintained site. Is there a way to mark these links so that there is no redirect, without repeating the lookup for all links on the site?

@groovecoder
Member

@jwhitlock good point and something to follow up on before we make the change permanent. Do you know a good way to measure if/how-much search engines may ding us? Or how quickly can we expect to see the effects on our search-based traffic?

@groovecoder
Member

Initial effect on back end execution time looks good ... I enabled the waffle switch before I pushed the code so the deployment reports from NR would be accurate ...

The document view apdex jumped from 0.95 to 0.99 immediately:
dumb-doc-url-doc-apdex

Most of the Memcache get time during document view was eliminated immediately, along with a noticeable amount of Render/wiki/document.html:
dumb-doc-url-doc-resp-time

All this dropped the average site-wide response time from ~200-300ms to ~100ms:
dumb-doc-urls-overall-resp-time

@groovecoder groovecoder changed the title from bug 1177264 - waffle switch for dumb doc urls to bug 1177264 - waffle switch for dumb doc urls (27.7% of app server time) Aug 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment