Skip to content
Permalink
Browse files

[2.0.x] Fixed CVE-2018-7536 -- Fixed catastrophic backtracking in url…

…ize and urlizetrunc template filters.

Thanks Florian Apolloner for assisting with the patch.
  • Loading branch information
timgraham committed Feb 24, 2018
1 parent 2da0064 commit e157315da3ae7005fa0683ffc9751dbeca7306c8
Showing with 64 additions and 13 deletions.
  1. +21 −12 django/utils/html.py
  2. +11 −0 docs/releases/1.11.11.txt
  3. +11 −0 docs/releases/1.8.19.txt
  4. +11 −0 docs/releases/2.0.3.txt
  5. +10 −1 tests/utils_tests/test_html.py
@@ -14,12 +14,7 @@
from .html_parser import HTMLParseError, HTMLParser

# Configuration for urlize() function.
TRAILING_PUNCTUATION_RE = re.compile(
'^' # Beginning of word
'(.*?)' # The URL in word
'([.,:;!]+)' # Allowed non-wrapping, trailing punctuation
'$' # End of word
)
TRAILING_PUNCTUATION_CHARS = '.,:;!'
WRAPPING_PUNCTUATION = [('(', ')'), ('<', '>'), ('[', ']'), ('&lt;', '&gt;'), ('"', '"'), ('\'', '\'')]

# List of possible strings used for bullets in bulleted lists.
@@ -29,7 +24,6 @@
word_split_re = re.compile(r'''([\s<>"']+)''')
simple_url_re = re.compile(r'^https?://\[?\w', re.IGNORECASE)
simple_url_2_re = re.compile(r'^www\.|^(?!http)\w[^@]+\.(com|edu|gov|int|mil|net|org)($|/.*)$', re.IGNORECASE)
simple_email_re = re.compile(r'^\S+@\S+\.\S+$')


@keep_lazy(str, SafeText)
@@ -276,10 +270,10 @@ def trim_punctuation(lead, middle, trail):
trimmed_something = False

# Trim trailing punctuation.
match = TRAILING_PUNCTUATION_RE.match(middle)
if match:
middle = match.group(1)
trail = match.group(2) + trail
stripped = middle.rstrip(TRAILING_PUNCTUATION_CHARS)
if middle != stripped:
trail = middle[len(stripped):] + trail
middle = stripped
trimmed_something = True

# Trim wrapping punctuation.
@@ -296,6 +290,21 @@ def trim_punctuation(lead, middle, trail):
trimmed_something = True
return lead, middle, trail

def is_email_simple(value):
"""Return True if value looks like an email address."""
# An @ must be in the middle of the value.
if '@' not in value or value.startswith('@') or value.endswith('@'):
return False
try:
p1, p2 = value.split('@')
except ValueError:
# value contains more than one @.
return False
# Dot must be in p2 (e.g. example.com)
if '.' not in p2 or p2.startswith('.'):
return False
return True

words = word_split_re.split(force_text(text))
for i, word in enumerate(words):
if '.' in word or '@' in word or ':' in word:
@@ -315,7 +324,7 @@ def trim_punctuation(lead, middle, trail):
elif simple_url_2_re.match(middle):
middle, middle_unescaped, trail = unescape(middle, trail)
url = smart_urlquote('http://%s' % middle_unescaped)
elif ':' not in middle and simple_email_re.match(middle):
elif ':' not in middle and is_email_simple(middle):
local, domain = middle.rsplit('@', 1)
try:
domain = domain.encode('idna').decode('ascii')
@@ -5,3 +5,14 @@ Django 1.11.11 release notes
*March 6, 2018*

Django 1.11.11 fixes two security issues in 1.11.10.

CVE-2018-7536: Denial-of-service possibility in ``urlize`` and ``urlizetrunc`` template filters
===============================================================================================

The ``django.utils.html.urlize()`` function was extremely slow to evaluate
certain inputs due to catastrophic backtracking vulnerabilities in two regular
expressions. The ``urlize()`` function is used to implement the ``urlize`` and
``urlizetrunc`` template filters, which were thus vulnerable.

The problematic regular expressions are replaced with parsing logic that
behaves similarly.
@@ -5,3 +5,14 @@ Django 1.8.19 release notes
*March 6, 2018*

Django 1.8.19 fixes two security issues in 1.18.18.

CVE-2018-7536: Denial-of-service possibility in ``urlize`` and ``urlizetrunc`` template filters
===============================================================================================

The ``django.utils.html.urlize()`` function was extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``urlize()`` function is used to implement the ``urlize`` and
``urlizetrunc`` template filters, which were thus vulnerable.

The problematic regular expression is replaced with parsing logic that behaves
similarly.
@@ -7,6 +7,17 @@ Django 2.0.3 release notes
Django 2.0.3 fixes two security issues and several bugs in 2.0.2. Also, the
latest string translations from Transifex are incorporated.

CVE-2018-7536: Denial-of-service possibility in ``urlize`` and ``urlizetrunc`` template filters
===============================================================================================

The ``django.utils.html.urlize()`` function was extremely slow to evaluate
certain inputs due to catastrophic backtracking vulnerabilities in two regular
expressions. The ``urlize()`` function is used to implement the ``urlize`` and
``urlizetrunc`` template filters, which were thus vulnerable.

The problematic regular expressions are replaced with parsing logic that
behaves similarly.

Bugfixes
========

@@ -5,7 +5,7 @@
from django.utils.functional import lazystr
from django.utils.html import (
conditional_escape, escape, escapejs, format_html, html_safe, linebreaks,
smart_urlquote, strip_spaces_between_tags, strip_tags,
smart_urlquote, strip_spaces_between_tags, strip_tags, urlize,
)
from django.utils.safestring import mark_safe

@@ -216,3 +216,12 @@ def test_html_safe_doesnt_define_str(self):
@html_safe
class HtmlClass:
pass

def test_urlize_unchanged_inputs(self):
tests = (
('a' + '@a' * 50000) + 'a', # simple_email_re catastrophic test
('a' + '.' * 1000000) + 'a', # trailing_punctuation catastrophic test
)
for value in tests:
with self.subTest(value=value):
self.assertEqual(urlize(value), value)

0 comments on commit e157315

Please sign in to comment.
You can’t perform that action at this time.