Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Fixed #5849 -- Strip whitespace from blocktrans #1773

Closed
wants to merge 1 commit into from

2 participants

@Bouke

See ticket 5849 for background on this PR. The patch was written by mpessas, the PR rebased by me. This PR supersedes PR #803.

django/utils/translation/__init__.py
@@ -194,3 +194,9 @@ def get_language_info(lang_code):
return LANG_INFO[generic_lang_code]
except KeyError:
raise KeyError("Unknown language code %s and %s." % (lang_code, generic_lang_code))
+
+
+trim_whitespace_re = re.compile('\s*\n\s*')
+def trim_whitespace(s):
+ """Trim the whitespace from s."""
@aaugustin Owner

That docstring doesn't add much info. It isn't useful to paraphrase a function's signature!

@Bouke
Bouke added a note

Indeed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
django/utils/translation/__init__.py
@@ -194,3 +194,9 @@ def get_language_info(lang_code):
return LANG_INFO[generic_lang_code]
except KeyError:
raise KeyError("Unknown language code %s and %s." % (lang_code, generic_lang_code))
+
+
+trim_whitespace_re = re.compile('\s*\n\s*')
+def trim_whitespace(s):
+ """Trim the whitespace from s."""
+ return trim_whitespace_re.sub(' ', s.strip('\n'))
@aaugustin Owner

Why not just s.strip()?

@Bouke
Bouke added a note

You're right, makes no difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
django/utils/translation/trans_real.py
@@ -530,19 +531,22 @@ def templatize(src, origin=None):
pluralmatch = plural_re.match(t.contents)
if endbmatch:
if inplural:
+ singular = trim_whitespace(''.join(singular))
@aaugustin Owner

Isn't this going to trim whitespace regardless of whether the trans block we're dealing with has the trimmed option set?

@Bouke
Bouke added a note

Good catch! The whole Parser/Lexer is unknown to me, so correct me if I've fixed it incorrectly. I've also added a unit test which showed that the trimmed option wasn't used in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/i18n/translation.txt
@@ -672,7 +672,32 @@ markers<contextual-markers>` using the ``context`` keyword:
.. code-block:: html+django
- {% blocktrans with name=user.username context "greeting" %}Hi {{ name }}{% endblocktrans %}
+ {% blocktrans with name=user.username context "greeting" %}Hi {{ name }}{%
+ endblocktrans %}
@aaugustin Owner

You cannot have a line break within a Django template tag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/i18n/translation.txt
((12 lines not shown))
+separate them. This is quite useful for indenting the content of a ``{%
+blocktrans %}`` tag without having the indentation characters end up in the
+corresponding entry in the PO file, which makes the translation process easier.
+
+For instance, the following ``{% blocktrans %}`` tag::
+
+ {% blocktrans trimmed %}
+ First sentence.
+ Second paragraph.
+ {% endblocktrans %}
+
+will result in the entry "First sentence. Second paragraph." in the PO file,
+compared to "\\nFirst sentence.\\nSecond sentence.\\n", if the ``trimmed``
+option had not been specified.
+
+.. versionchanged:: 1.7
@aaugustin Owner

Could you add a line in the release notes for 1.7 too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@aaugustin
Owner

Overall this is pretty good; my comments are rather minor.

Once you've addressed them, feel free to mark the corresponding ticket as RFC yourself.

@Bouke Bouke Fixed #5849 -- Strip whitespace from blocktrans
Add the trimmed option to the blocktrans tag to trim any newlines and
whitespace from its content.

This allows the developer to indent the blocktrans tag without adding
new lines and whitespace to the msgid in the PO file.

Thanks to mpessas for the initial patch and Dmitri Fedortchenko for the
report.
e1dd2da
@Bouke

Thanks for the review; I've updated the PR with the appropriate changes.

@Bouke

Merged in 7a7c789

@Bouke Bouke closed this
@Bouke Bouke deleted the branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 2, 2013
  1. @Bouke

    Fixed #5849 -- Strip whitespace from blocktrans

    Bouke authored
    Add the trimmed option to the blocktrans tag to trim any newlines and
    whitespace from its content.
    
    This allows the developer to indent the blocktrans tag without adding
    new lines and whitespace to the msgid in the PO file.
    
    Thanks to mpessas for the initial patch and Dmitri Fedortchenko for the
    report.
This page is out of date. Refresh to see the latest.
View
15 django/templatetags/i18n.py
@@ -97,14 +97,16 @@ def render(self, context):
class BlockTranslateNode(Node):
+
def __init__(self, extra_context, singular, plural=None, countervar=None,
- counter=None, message_context=None):
+ counter=None, message_context=None, trimmed=False):
self.extra_context = extra_context
self.singular = singular
self.plural = plural
self.countervar = countervar
self.counter = counter
self.message_context = message_context
+ self.trimmed = trimmed
def render_token_list(self, tokens):
result = []
@@ -115,7 +117,10 @@ def render_token_list(self, tokens):
elif token.token_type == TOKEN_VAR:
result.append('%%(%s)s' % token.contents)
vars.append(token.contents)
- return ''.join(result), vars
+ msg = ''.join(result)
+ if self.trimmed:
+ msg = translation.trim_whitespace(msg)
+ return msg, vars
def render(self, context, nested=False):
if self.message_context:
@@ -429,6 +434,8 @@ def do_block_translate(parser, token):
'"context" in %r tag expected '
'exactly one argument.') % bits[0]
six.reraise(TemplateSyntaxError, TemplateSyntaxError(msg), sys.exc_info()[2])
+ elif option == "trimmed":
+ value = True
else:
raise TemplateSyntaxError('Unknown argument for %r tag: %r.' %
(bits[0], option))
@@ -444,6 +451,8 @@ def do_block_translate(parser, token):
message_context = None
extra_context = options.get('with', {})
+ trimmed = options.get("trimmed", False)
+
singular = []
plural = []
while parser.tokens:
@@ -465,7 +474,7 @@ def do_block_translate(parser, token):
raise TemplateSyntaxError("'blocktrans' doesn't allow other block tags (seen %r) inside it" % token.contents)
return BlockTranslateNode(extra_context, singular, plural, countervar,
- counter, message_context)
+ counter, message_context, trimmed=trimmed)
@register.tag
def language(parser, token):
View
8 django/utils/translation/__init__.py
@@ -2,7 +2,7 @@
Internationalization support.
"""
from __future__ import unicode_literals
-
+import re
from django.utils.encoding import force_text
from django.utils.functional import lazy
from django.utils import six
@@ -194,3 +194,9 @@ def get_language_info(lang_code):
return LANG_INFO[generic_lang_code]
except KeyError:
raise KeyError("Unknown language code %s and %s." % (lang_code, generic_lang_code))
+
+trim_whitespace_re = re.compile('\s*\n\s*')
+
+
+def trim_whitespace(s):
+ return trim_whitespace_re.sub(' ', s.strip())
View
22 django/utils/translation/trans_real.py
@@ -17,7 +17,7 @@
from django.utils.safestring import mark_safe, SafeData
from django.utils import six
from django.utils.six import StringIO
-from django.utils.translation import TranslatorCommentWarning
+from django.utils.translation import TranslatorCommentWarning, trim_whitespace
# Translations are cached in a dictionary for every language+app tuple.
@@ -477,6 +477,7 @@ def blankout(src, char):
"""
return dot_re.sub(char, src)
+
context_re = re.compile(r"""^\s+.*context\s+((?:"[^"]*?")|(?:'[^']*?'))\s*""")
inline_re = re.compile(r"""^\s*trans\s+((?:"[^"]*?")|(?:'[^']*?'))(\s+.*context\s+((?:"[^"]*?")|(?:'[^']*?')))?\s*""")
block_re = re.compile(r"""^\s*blocktrans(\s+.*context\s+((?:"[^"]*?")|(?:'[^']*?')))?(?:\s+|$)""")
@@ -500,6 +501,7 @@ def templatize(src, origin=None):
message_context = None
intrans = False
inplural = False
+ trimmed = False
singular = []
plural = []
incomment = False
@@ -529,20 +531,29 @@ def templatize(src, origin=None):
endbmatch = endblock_re.match(t.contents)
pluralmatch = plural_re.match(t.contents)
if endbmatch:
+ if trimmed:
+ singular = trim_whitespace(''.join(singular))
+ else:
+ singular = ''.join(singular)
+
if inplural:
+ if trimmed:
+ plural = trim_whitespace(''.join(plural))
+ else:
+ plural = ''.join(plural)
if message_context:
- out.write(' npgettext(%r, %r, %r,count) ' % (message_context, ''.join(singular), ''.join(plural)))
+ out.write(' npgettext(%r, %r, %r,count) ' % (message_context, singular, plural))
else:
- out.write(' ngettext(%r, %r, count) ' % (''.join(singular), ''.join(plural)))
+ out.write(' ngettext(%r, %r, count) ' % (singular, plural))
for part in singular:
out.write(blankout(part, 'S'))
for part in plural:
out.write(blankout(part, 'P'))
else:
if message_context:
- out.write(' pgettext(%r, %r) ' % (message_context, ''.join(singular)))
+ out.write(' pgettext(%r, %r) ' % (message_context, singular))
else:
- out.write(' gettext(%r) ' % ''.join(singular))
+ out.write(' gettext(%r) ' % singular)
for part in singular:
out.write(blankout(part, 'S'))
message_context = None
@@ -625,6 +636,7 @@ def templatize(src, origin=None):
message_context = message_context.strip("'")
intrans = True
inplural = False
+ trimmed = 'trimmed' in t.split_contents()
singular = []
plural = []
elif cmatches:
View
9 docs/releases/1.7.txt
@@ -327,6 +327,15 @@ Internationalization
still read from in 1.7. Sessions will be migrated to the new ``_language``
key as they are written.
+* The :ttag:`blocktrans` now supports a ``trimmed`` option. This
+ option will remove newline characters from the beginning and the end of the
+ content of the ``{% blocktrans %}`` tag, replace any whitespace at the
+ beginning and end of a line and merge all lines into one using a space
+ character to separate them. This is quite useful for indenting the content of
+ a ``{% blocktrans %}`` tag without having the indentation characters end up
+ in the corresponding entry in the PO file, which makes the translation
+ process easier.
+
Management Commands
^^^^^^^^^^^^^^^^^^^
View
24 docs/topics/i18n/translation.txt
@@ -674,6 +674,30 @@ markers<contextual-markers>` using the ``context`` keyword:
{% blocktrans with name=user.username context "greeting" %}Hi {{ name }}{% endblocktrans %}
+Another feature ``{% blocktrans %}`` supports is the ``trimmed`` option. This
+option will remove newline characters from the beginning and the end of the
+content of the ``{% blocktrans %}`` tag, replace any whitespace at the beginning
+and end of a line and merge all lines into one using a space character to
+separate them. This is quite useful for indenting the content of a ``{%
+blocktrans %}`` tag without having the indentation characters end up in the
+corresponding entry in the PO file, which makes the translation process easier.
+
+For instance, the following ``{% blocktrans %}`` tag::
+
+ {% blocktrans trimmed %}
+ First sentence.
+ Second paragraph.
+ {% endblocktrans %}
+
+will result in the entry ``"First sentence. Second paragraph."`` in the PO file,
+compared to ``"\n First sentence.\n Second sentence.\n"``, if the ``trimmed``
+option had not been specified.
+
+.. versionchanged:: 1.7
+
+ The ``trimmed`` option was added.
+
+
String literals passed to tags and filters
------------------------------------------
View
12 tests/i18n/commands/templates/test.html
@@ -82,3 +82,15 @@
{% trans "Translatable literal with context wrapped in double quotes" context "Context wrapped in double quotes" as var %}
{% blocktrans context 'Special blocktrans context wrapped in single quotes' %}Translatable literal with context wrapped in single quotes{% endblocktrans %}
{% blocktrans context "Special blocktrans context wrapped in double quotes" %}Translatable literal with context wrapped in double quotes{% endblocktrans %}
+
+
+{# BasicExtractorTests.test_blocktrans_trimmed #}
+{% blocktrans %}
+ Text with a few
+ line breaks.
+{% endblocktrans %}
+{% blocktrans trimmed %}
+ Again some text with a few
+ line breaks, this time
+ should be trimmed.
+{% endblocktrans %}
View
11 tests/i18n/test_extraction.py
@@ -120,6 +120,17 @@ def test_templatize_blocktrans_tag(self):
self.assertMsgId('I think that 100%% is more that 50%% of %(obj)s.', po_contents)
self.assertMsgId("Blocktrans extraction shouldn't double escape this: %%, a=%(a)s", po_contents)
+ def test_blocktrans_trimmed(self):
+ os.chdir(self.test_dir)
+ management.call_command('makemessages', locale=LOCALE, verbosity=0)
+ self.assertTrue(os.path.exists(self.PO_FILE))
+ with open(self.PO_FILE, 'r') as fp:
+ po_contents = force_text(fp.read())
+ # should not be trimmed
+ self.assertNotMsgId('Text with a few line breaks.', po_contents)
+ # should be trimmed
+ self.assertMsgId("Again some text with a few line breaks, this time should be trimmed.", po_contents)
+
def test_force_en_us_locale(self):
"""Value of locale-munging option used by the command is the right one"""
from django.core.management.commands.makemessages import Command
View
11 tests/i18n/tests.py
@@ -245,6 +245,17 @@ def test_template_tags_pgettext(self):
rendered = t.render(Context())
self.assertEqual(rendered, 'Andere: Es gibt 5 Kommentare')
+ # Using trimmed
+ t = Template('{% load i18n %}{% blocktrans trimmed %}\n\nThere\n\t are 5 \n\n comments\n{% endblocktrans %}')
+ rendered = t.render(Context())
+ self.assertEqual(rendered, 'There are 5 comments')
+ t = Template('{% load i18n %}{% blocktrans with num_comments=5 context "comment count" trimmed %}\n\nThere are \t\n \t {{ num_comments }} comments\n\n{% endblocktrans %}')
+ rendered = t.render(Context())
+ self.assertEqual(rendered, 'Es gibt 5 Kommentare')
+ t = Template('{% load i18n %}{% blocktrans context "other super search" count number=2 trimmed %}\n{{ number }} super \n result{% plural %}{{ number }} super results{% endblocktrans %}')
+ rendered = t.render(Context())
+ self.assertEqual(rendered, '2 andere Super-Ergebnisse')
+
# Mis-uses
self.assertRaises(TemplateSyntaxError, Template, '{% load i18n %}{% blocktrans context with month="May" %}{{ month }}{% endblocktrans %}')
self.assertRaises(TemplateSyntaxError, Template, '{% load i18n %}{% blocktrans context %}{% endblocktrans %}')
Something went wrong with that request. Please try again.