Skip to content
This repository

allow the optional use of a path-to-class dict instead of regex-matching all paths, workaround Issue #214 and general perf improvement #215

Open
wants to merge 2 commits into from

2 participants

Aaron Daubman Anand Chitipothu
Aaron Daubman

If you have a constant (no regex matching necessary) set of paths to map, you can now add use_path_dict=True to your:
app = web.application(urls, globals(), use_path_dict=True)
call to use a dict of something like:
{'web.ctx.path-1': class_to_handle_path-1}

rather than avoiding the regex matching against all paths for every call.

While this will have a tiny perf boost, its main benefit is avoiding the non-thread-safe memoization of re_compile discussed in Issue #214

This should be completely transparent and backwards compatible for any users requiring regex matching of paths...

Anand Chitipothu
Collaborator

This is not related to re.compile issue at all. The re.compile issue needs to be fixed, but I don't think this work-around for handling just a rare special case is worth it.

Aaron Daubman

@anandology Some followup questions / comments:

1) IMHO, this is directly related to the re.compile issue - it does not FIX that issue, but provides a workaround for the most highly-threaded use of re.compile that is currently causing us to throw ~200 500 Errors an hour in production.

2) Can you qualify "rare special case"? I would think this functionality would be highly desirable for any site that uses params rather than paths, and so would never ever need the cost of regex matching a potentially long list of paths (memoization or not) - if you ever look at a CProfile of web.py calls for a site with many paths, it gets pretty crazy as all those regexes are tested... Is there data that suggests that almost every site using web.py actually uses/needs the regex matching?

3) Can you qualify "worth it"? IMHO, the cost of this patch is zero to anybody not interested in using it... (and in total involves <15 LOC changes)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.

Showing 1 changed file with 15 additions and 4 deletions. Show diff stats Hide diff stats

  1. +15 4 web/application.py
19 web/application.py
@@ -41,9 +41,10 @@ class application:
41 41 >>> app.request("/hello").data
42 42 'hello'
43 43 """
44   - def __init__(self, mapping=(), fvars={}, autoreload=None):
  44 + def __init__(self, mapping=(), fvars={}, autoreload=None, use_path_dict=False):
45 45 if autoreload is None:
46 46 autoreload = web.config.get('debug', False)
  47 + self.use_path_dict = use_path_dict
47 48 self.init_mapping(mapping)
48 49 self.fvars = fvars
49 50 self.processors = []
@@ -109,13 +110,19 @@ def _unload(self):
109 110 def _cleanup(self):
110 111 # Threads can be recycled by WSGI servers.
111 112 # Clearing up all thread-local state to avoid interefereing with subsequent requests.
112   - utils.ThreadedDict.clear_all()
  113 + if hasattr(utils.ThreadedDict, 'clear_all'):
  114 + utils.ThreadedDict.clear_all()
113 115
114 116 def init_mapping(self, mapping):
115 117 self.mapping = list(utils.group(mapping, 2))
  118 + if self.use_path_dict:
  119 + self.mapping = dict(x for x in reversed(self.mapping)) # reversed so that if there are duplicate paths the first entry "wins"
116 120
117 121 def add_mapping(self, pattern, classname):
118   - self.mapping.append((pattern, classname))
  122 + if self.use_path_dict:
  123 + self.mapping[pattern] = classname
  124 + else:
  125 + self.mapping.append((pattern, classname))
119 126
120 127 def add_processor(self, processor):
121 128 """
@@ -226,7 +233,11 @@ def browser(self):
226 233 return browser.AppBrowser(self)
227 234
228 235 def handle(self):
229   - fn, args = self._match(self.mapping, web.ctx.path)
  236 + if self.use_path_dict:
  237 + fn = self.mapping.get(web.ctx.path)
  238 + args = []
  239 + else:
  240 + fn, args = self._match(self.mapping, web.ctx.path)
230 241 return self._delegate(fn, self.fvars, args)
231 242
232 243 def handle_with_processors(self):

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.