Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Trac #17181 - Allow for specifying multiple locales and domains for makemessages and compilemessages commands #121

Closed
wants to merge 5 commits into from

4 participants

Craig Blaszczyk Claude Paroz Florian Apolloner Ramiro Morales
Craig Blaszczyk

option to pass in multiple locales to the compilemessages command, and multiple locales and multiple domains to the makemessages command

https://code.djangoproject.com/ticket/17181

I've updated the existing patch (from the ticket), so that it applies cleanly. I've added tests and docs.

Let me know if this is up to scratch!

Craig Blaszczyk jakul allow multiple domains and languages for makemessages command; add op…
…tion to pass in multiple language to the compilemessages command
4fbb940
Claude Paroz
Collaborator

There are already tests in tests/regressiontests/i18n/commands/compilation.py and tests/regressiontests/i18n/commands/extraction.py. You shouldn't create new test files.

Craig Blaszczyk

Merged the changes into the existing files; I also changed one of the tests to not use the it locale, which is used by some other tests

Florian Apolloner
Owner

I'd like to be able to use -l de -l en -l es as alternate for -l de,en,es to stay consistent with -n of startproject. The patch is also missing versionadded/changed directives.

Craig Blaszczyk

@apollo13 I've added the versionchanged directives. The patch already supported specifiying multiple locales through "-l de -l pt -l it"

Florian Apolloner
Owner

Could you squash those commits into one? Btw do we really need tests/regressiontests/i18n/commands/locale/hr/LC_MESSAGES/django.po -- the locale folder there has already 3 existing locales, could you reuse those?

Craig Blaszczyk

I need 2 locales with .po files and without .mo files in order to check that the compilation works properly.

Of the existing locales:
'it' has an error in the .po file and doesn't compile to a .mo file (this is intentional)
'fr' already has a commited .mo file

I can remove the hr locale if it is OK to remove the .mo file from the fr locale, however I didn't think that was a good thing to do.

Ramiro Morales
Collaborator

Closing, adapted and committed code from this PR in 6158c79. See https://code.djangoproject.com/ticket/17181#comment:9 for further details.

Thanks!

Ramiro Morales ramiro closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 7, 2012
  1. Craig Blaszczyk

    allow multiple domains and languages for makemessages command; add op…

    jakul authored
    …tion to pass in multiple language to the compilemessages command
  2. Craig Blaszczyk
  3. Craig Blaszczyk

    give tests more sensible names; add hr locale to properly check compi…

    jakul authored
    …lemessages without deleting the django.mo in the it language
Commits on Jun 13, 2012
  1. Craig Blaszczyk

    add versionchanged directive

    jakul authored
Commits on Jun 15, 2012
  1. Craig Blaszczyk

    add myself to AUTHORS

    jakul authored
This page is out of date. Refresh to see the latest.
1  AUTHORS
View
@@ -91,6 +91,7 @@ answer newbie questions, and generally made Django that much better:
Mark Biggers <biggers@utsl.com>
Paul Bissex <http://e-scribe.com/>
Simon Blanchard
+ Craig Blaszczyk <masterjakul@gmail.com>
David Blewett <david@dawninglight.net>
Matt Boersma <matt@sprout.org>
Artem Gnilov <boobsd@gmail.com>
50 django/core/management/commands/compilemessages.py
View
@@ -24,34 +24,38 @@ def compile_messages(stderr, locale=None):
raise CommandError("This script should be run from the Django Git checkout or your project or app tree, or with the settings module specified.")
for basedir in basedirs:
+ dirs = [basedir]
if locale:
- basedir = os.path.join(basedir, locale, 'LC_MESSAGES')
- for dirpath, dirnames, filenames in os.walk(basedir):
- for f in filenames:
- if f.endswith('.po'):
- stderr.write('processing file %s in %s\n' % (f, dirpath))
- fn = os.path.join(dirpath, f)
- if has_bom(fn):
- raise CommandError("The %s file has a BOM (Byte Order Mark). Django only supports .po files encoded in UTF-8 and without any BOM." % fn)
- pf = os.path.splitext(fn)[0]
- # Store the names of the .mo and .po files in an environment
- # variable, rather than doing a string replacement into the
- # command, so that we can take advantage of shell quoting, to
- # quote any malicious characters/escaping.
- # See http://cyberelk.net/tim/articles/cmdline/ar01s02.html
- os.environ['djangocompilemo'] = pf + '.mo'
- os.environ['djangocompilepo'] = pf + '.po'
- if sys.platform == 'win32': # Different shell-variable syntax
- cmd = 'msgfmt --check-format -o "%djangocompilemo%" "%djangocompilepo%"'
- else:
- cmd = 'msgfmt --check-format -o "$djangocompilemo" "$djangocompilepo"'
- os.system(cmd)
+ dirs = [os.path.join(basedir, l, 'LC_MESSAGES') for l in (locale if isinstance(locale, list) else [locale])]
+
+
+ for dir in dirs:
+ for dirpath, dirnames, filenames in os.walk(dir):
+ for f in filenames:
+ if f.endswith('.po'):
+ stderr.write('processing file %s in %s\n' % (f, dirpath))
+ fn = os.path.join(dirpath, f)
+ if has_bom(fn):
+ raise CommandError("The %s file has a BOM (Byte Order Mark). Django only supports .po files encoded in UTF-8 and without any BOM." % fn)
+ pf = os.path.splitext(fn)[0]
+ # Store the names of the .mo and .po files in an environment
+ # variable, rather than doing a string replacement into the
+ # command, so that we can take advantage of shell quoting, to
+ # quote any malicious characters/escaping.
+ # See http://cyberelk.net/tim/articles/cmdline/ar01s02.html
+ os.environ['djangocompilemo'] = pf + '.mo'
+ os.environ['djangocompilepo'] = pf + '.po'
+ if sys.platform == 'win32': # Different shell-variable syntax
+ cmd = 'msgfmt --check-format -o "%djangocompilemo%" "%djangocompilepo%"'
+ else:
+ cmd = 'msgfmt --check-format -o "$djangocompilemo" "$djangocompilepo"'
+ os.system(cmd)
class Command(BaseCommand):
option_list = BaseCommand.option_list + (
- make_option('--locale', '-l', dest='locale',
- help='The locale to process. Default is to process all.'),
+ make_option('--locale', '-l', dest='locale', action="append",
+ help='A locale to process. Default is to process all.'),
)
help = 'Compiles .po files to .mo files for use with builtin gettext support.'
52 django/core/management/commands/makemessages.py
View
@@ -248,7 +248,7 @@ def write_po_file(pofile, potfile, domain, locale, verbosity, stdout,
raise CommandError(
"errors happened while running msgattrib\n%s" % errors)
-def make_messages(locale=None, domain='django', verbosity=1, all=False,
+def make_messages(locale=None, domain=None, verbosity=1, all=False,
extensions=None, symlinks=False, ignore_patterns=None, no_wrap=False,
no_location=False, no_obsolete=False, stdout=sys.stdout):
"""
@@ -283,10 +283,15 @@ def make_messages(locale=None, domain='django', verbosity=1, all=False,
"is not created automatically, you have to create it by hand "
"if you want to enable i18n for your project or application.")
- if domain not in ('django', 'djangojs'):
+ if domain is None:
+ domains = ['django']
+ else:
+ domains = [domain] if not isinstance(domain, list) else domain
+
+ if any(d not in ('django', 'djangojs') for d in domains):
raise CommandError("currently makemessages only supports domains 'django' and 'djangojs'")
- if (locale is None and not all) or domain is None:
+ if (locale is None and not all) or not domains:
message = "Type '%s help %s' for usage information." % (os.path.basename(sys.argv[0]), sys.argv[1])
raise CommandError(message)
@@ -302,7 +307,7 @@ def make_messages(locale=None, domain='django', verbosity=1, all=False,
locales = []
if locale is not None:
- locales.append(locale)
+ locales += locale.split(',') if not isinstance(locale, list) else locale
elif all:
locale_dirs = filter(os.path.isdir, glob.glob('%s/*' % localedir))
locales = [os.path.basename(l) for l in locale_dirs]
@@ -317,28 +322,31 @@ def make_messages(locale=None, domain='django', verbosity=1, all=False,
if not os.path.isdir(basedir):
os.makedirs(basedir)
- pofile = os.path.join(basedir, '%s.po' % domain)
- potfile = os.path.join(basedir, '%s.pot' % domain)
-
- if os.path.exists(potfile):
- os.unlink(potfile)
-
- for dirpath, file in find_files(".", ignore_patterns, verbosity,
- stdout, symlinks=symlinks):
- process_file(file, dirpath, potfile, domain, verbosity, extensions,
- wrap, location, stdout)
-
- if os.path.exists(potfile):
- write_po_file(pofile, potfile, domain, locale, verbosity, stdout,
- not invoked_for_django, wrap, location, no_obsolete)
+ for domain in domains:
+ if verbosity > 1:
+ stdout.write("processing domain %s\n" % domain)
+ pofile = os.path.join(basedir, '%s.po' % domain)
+ potfile = os.path.join(basedir, '%s.pot' % domain)
+
+ if os.path.exists(potfile):
+ os.unlink(potfile)
+
+ for dirpath, file in find_files(".", ignore_patterns, verbosity,
+ stdout, symlinks=symlinks):
+ process_file(file, dirpath, potfile, domain, verbosity, extensions,
+ wrap, location, stdout)
+
+ if os.path.exists(potfile):
+ write_po_file(pofile, potfile, domain, locale, verbosity, stdout,
+ not invoked_for_django, wrap, location, no_obsolete)
class Command(NoArgsCommand):
option_list = NoArgsCommand.option_list + (
- make_option('--locale', '-l', default=None, dest='locale',
- help='Creates or updates the message files for the given locale (e.g. pt_BR).'),
- make_option('--domain', '-d', default='django', dest='domain',
- help='The domain of the message files (default: "django").'),
+ make_option('--locale', '-l', default=None, dest='locale', action='append',
+ help='Creates or updates the message files for the given locale(s) (e.g. pt_BR).'),
+ make_option('--domain', '-d', default=None, dest='domain', action='append',
+ help='The domain(s) of the message files (default: "django").'),
make_option('--all', '-a', action='store_true', dest='all',
default=False, help='Updates the message files for all existing locales.'),
make_option('--extension', '-e', dest='extensions',
26 docs/ref/django-admin.txt
View
@@ -104,12 +104,16 @@ compilemessages
Compiles .po files created with ``makemessages`` to .mo files for use with
the builtin gettext support. See :doc:`/topics/i18n/index`.
-Use the :djadminopt:`--locale` option to specify the locale to process.
+Use the :djadminopt:`--locale` option to specify the locale(s) to process.
If not provided, all locales are processed.
Example usage::
- django-admin.py compilemessages --locale=br_PT
+ django-admin.py compilemessages --locale=pt_BR --locale=fr
+
+.. versionchanged:: 1.5
+
+Added the option to specify multiple locales.
createcachetable
----------------
@@ -422,11 +426,19 @@ Separate multiple extensions with commas or use -e or --extension multiple times
django-admin.py makemessages --locale=de --extension=html,txt --extension xml
-Use the :djadminopt:`--locale` option to specify the locale to process.
+Use the :djadminopt:`--locale` option to specify the locale(s) to process.
Example usage::
- django-admin.py makemessages --locale=br_PT
+ django-admin.py makemessages --locale=pt_BR --locale=fr
+
+You can also use commas to separate multiple locales::
+
+ django-admin.py makemessages --locale=de,fr,pt_BR
+
+.. versionchanged:: 1.5
+
+Added the option to specify multiple locales.
.. django-admin-option:: --domain
@@ -436,6 +448,12 @@ Currently supported:
* ``django`` for all ``*.py``, ``*.html`` and ``*.txt`` files (default)
* ``djangojs`` for ``*.js`` files
+
+Example usage::
+
+ django-admin.py makemessages --domain=django --domain=djangojs
+
+
.. django-admin-option:: --symlinks
Use the ``--symlinks`` or ``-s`` option to follow symlinks to directories when
44 tests/regressiontests/i18n/commands/compilation.py
View
@@ -1,11 +1,17 @@
import os
from io import BytesIO
+from StringIO import StringIO
+from django.conf import settings
from django.core.management import call_command, CommandError
+from django.core.management.commands import compilemessages
from django.test import TestCase
from django.test.utils import override_settings
from django.utils import translation
+
+
+
test_dir = os.path.abspath(os.path.dirname(__file__))
class MessageCompilationTests(TestCase):
@@ -66,3 +72,41 @@ def test_percent_symbol_escaping(self):
t = Template('{% load i18n %}{% trans "Completed 50%% of all the tasks" %}')
rendered = t.render(Context({}))
self.assertEqual(rendered, 'IT translation of Completed 50%% of all the tasks')
+
+
+class CompilationMultipleLocalesTestCase(TestCase):
+ MO_FILE_HR = None
+ MO_FILE_FR = None
+
+ def setUp(self):
+ self._old_locale_paths = settings.LOCALE_PATHS
+ self.stderr = StringIO()
+ self.localedir = os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), 'locale'
+ )
+ settings.LOCALE_PATHS = [self.localedir]
+ self.MO_FILE_HR = os.path.join(self.localedir, 'hr/LC_MESSAGES/django.mo')
+ self.MO_FILE_FR = os.path.join(self.localedir, 'fr/LC_MESSAGES/django.mo')
+
+ def tearDown(self):
+ settings.LOCALE_PATHS = self._old_locale_paths
+ self.stderr.close()
+ self._rmfile(os.path.join(self.localedir, self.MO_FILE_HR))
+ self._rmfile(os.path.join(self.localedir, self.MO_FILE_FR))
+
+ def _rmfile(self, filepath):
+ if os.path.exists(filepath):
+ os.remove(filepath)
+
+ def test_one_locale(self):
+ command = compilemessages.Command()
+ command.execute(locale='hr', stderr=self.stderr)
+
+ self.assertTrue(os.path.exists(self.MO_FILE_HR))
+
+ def test_multiple_locales(self):
+ command = compilemessages.Command()
+ command.execute(locale=['hr', 'fr'], stderr=self.stderr)
+
+ self.assertTrue(os.path.exists(self.MO_FILE_HR))
+ self.assertTrue(os.path.exists(self.MO_FILE_FR))
28 tests/regressiontests/i18n/commands/extraction.py
View
@@ -267,3 +267,31 @@ def test_no_location_disabled(self):
with open(self.PO_FILE, 'r') as fp:
po_contents = fp.read()
self.assertTrue('#: templates/test.html:55' in po_contents)
+
+
+class ExtractionMultipleLocalesTestCase(ExtractorTests):
+ PO_FILE_PT = 'locale/pt/LC_MESSAGES/django.po'
+ PO_FILE_DE = 'locale/de/LC_MESSAGES/django.po'
+ LOCALES = ['pt', 'de', 'ch']
+
+
+ def tearDown(self):
+ os.chdir(self.test_dir)
+ for locale in self.LOCALES:
+ try:
+ self._rmrf('locale/%s' % locale)
+ except OSError:
+ pass
+ os.chdir(self._cwd)
+
+ def test_multiple_locales(self):
+ os.chdir(self.test_dir)
+ management.call_command('makemessages', locale=['pt','de'], verbosity=0)
+ self.assertTrue(os.path.exists(self.PO_FILE_PT))
+ self.assertTrue(os.path.exists(self.PO_FILE_DE))
+
+ def test_comma_separated_locales(self):
+ os.chdir(self.test_dir)
+ management.call_command('makemessages', locale='pt,de,ch', verbosity=0)
+ self.assertTrue(os.path.exists(self.PO_FILE_PT))
+ self.assertTrue(os.path.exists(self.PO_FILE_DE))
71 tests/regressiontests/i18n/commands/locale/hr/LC_MESSAGES/django.po
View
@@ -0,0 +1,71 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
+# This file is distributed under the same license as the PACKAGE package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
+#
+msgid ""
+msgstr ""
+"Project-Id-Version: PACKAGE VERSION\n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2011-12-04 04:59-0600\n"
+"PO-Revision-Date: 2011-12-10 19:12-0300\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language-Team: LANGUAGE <LL@li.org>\n"
+"Language: fr\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Plural-Forms: nplurals=2; plural=(n != 1)\n"
+
+#. Translators: Django template comment for translators
+#: templates/test.html:9
+#, python-format
+msgid "I think that 100%% is more that 50%% of anything."
+msgstr ""
+
+#: templates/test.html:10
+#, python-format
+msgid "I think that 100%% is more that 50%% of %(obj)s."
+msgstr ""
+
+#: templates/test.html:70
+#, python-format
+msgid "Literal with a percent symbol at the end %%"
+msgstr ""
+
+#: templates/test.html:71
+#, python-format
+msgid "Literal with a percent %% symbol in the middle"
+msgstr ""
+
+#: templates/test.html:72
+#, python-format
+msgid "Completed 50%% of all the tasks"
+msgstr ""
+
+#: templates/test.html:73
+#, python-format
+msgctxt "ctx0"
+msgid "Completed 99%% of all the tasks"
+msgstr ""
+
+#: templates/test.html:74
+#, python-format
+msgid "Shouldn't double escape this sequence: %% (two percent signs)"
+msgstr ""
+
+#: templates/test.html:75
+#, python-format
+msgctxt "ctx1"
+msgid "Shouldn't double escape this sequence %% either"
+msgstr ""
+
+#: templates/test.html:76
+#, python-format
+msgid "Looks like a str fmt spec %%s but shouldn't be interpreted as such"
+msgstr "Translation of the above string"
+
+#: templates/test.html:77
+#, python-format
+msgid "Looks like a str fmt spec %% o but shouldn't be interpreted as such"
+msgstr "Translation contains %% for the above string"
4 tests/regressiontests/i18n/tests.py
View
@@ -29,10 +29,10 @@
from .commands.extraction import (ExtractorTests, BasicExtractorTests,
JavascriptExtractorTests, IgnoredExtractorTests, SymlinkExtractorTests,
CopyPluralFormsExtractorTests, NoWrapExtractorTests,
- NoLocationExtractorTests)
+ NoLocationExtractorTests, ExtractionMultipleLocalesTestCase)
if can_run_compilation_tests:
from .commands.compilation import (PoFileTests, PoFileContentsTests,
- PercentRenderingTests)
+ PercentRenderingTests, CompilationMultipleLocalesTestCase)
from .contenttypes.tests import ContentTypeTests
from .forms import I18nForm, SelectDateForm, SelectDateWidget, CompanyForm
from .models import Company, TestModel
Something went wrong with that request. Please try again.