Finish C tokenizer refactoring (fixes #37)

earwig · Jul 28, 2015 · 6503b56 · 6503b56
1 parent 8e7a600
commit 6503b56
Show file tree

Hide file tree

Showing 2 changed files with 11 additions and 2 deletions.
diff --git a/CHANGELOG b/CHANGELOG
@@ -11,7 +11,10 @@ v0.4.1 (unreleased):
 - Fixed parser bugs involving:
   - templates with completely blank names;
   - templates with newlines and comments.
-- Heavy refactoring and fixes to the C tokenizer.
+- Heavy refactoring and fixes to the C tokenizer, including:
+  - corrected a design flaw in text handling, allowing for substantial speed
+    improvements when parsing long strings of plain text;
+  - implemented new Python 3.3 PEP 393 Unicode APIs.
 - Fixed some bugs in the release scripts.
 
 v0.4 (released May 23, 2015):

diff --git a/docs/changelog.rst b/docs/changelog.rst
@@ -20,7 +20,13 @@ Unreleased
   - templates with completely blank names;
   - templates with newlines and comments.
 
-- Heavy refactoring and fixes to the C tokenizer.
+- Heavy refactoring and fixes to the C tokenizer, including:
+
+  - corrected a design flaw in text handling, allowing for substantial speed
+    improvements when parsing long strings of plain text;
+  - implemented new Python 3.3
+    `PEP 393 <https://www.python.org/dev/peps/pep-0393/>`_ Unicode APIs.
+
 - Fixed some bugs in the release scripts.
 
 v0.4