Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Trac #17181 - Allow for specifying multiple locales and domains for makemessages and compilemessages commands #121

Closed
wants to merge 5 commits into from

4 participants

Craig Blaszczyk Claude Paroz Florian Apolloner Ramiro Morales
Craig Blaszczyk
jakul commented June 07, 2012

option to pass in multiple locales to the compilemessages command, and multiple locales and multiple domains to the makemessages command

https://code.djangoproject.com/ticket/17181

I've updated the existing patch (from the ticket), so that it applies cleanly. I've added tests and docs.

Let me know if this is up to scratch!

Craig Blaszczyk allow multiple domains and languages for makemessages command; add op…
…tion to pass in multiple language to the compilemessages command
4fbb940
Claude Paroz
Owner

There are already tests in tests/regressiontests/i18n/commands/compilation.py and tests/regressiontests/i18n/commands/extraction.py. You shouldn't create new test files.

Craig Blaszczyk
jakul commented June 07, 2012

Merged the changes into the existing files; I also changed one of the tests to not use the it locale, which is used by some other tests

Florian Apolloner
Owner

I'd like to be able to use -l de -l en -l es as alternate for -l de,en,es to stay consistent with -n of startproject. The patch is also missing versionadded/changed directives.

Craig Blaszczyk
jakul commented June 13, 2012

@apollo13 I've added the versionchanged directives. The patch already supported specifiying multiple locales through "-l de -l pt -l it"

Florian Apolloner
Owner

Could you squash those commits into one? Btw do we really need tests/regressiontests/i18n/commands/locale/hr/LC_MESSAGES/django.po -- the locale folder there has already 3 existing locales, could you reuse those?

Craig Blaszczyk
jakul commented June 20, 2012

I need 2 locales with .po files and without .mo files in order to check that the compilation works properly.

Of the existing locales:
'it' has an error in the .po file and doesn't compile to a .mo file (this is intentional)
'fr' already has a commited .mo file

I can remove the hr locale if it is OK to remove the .mo file from the fr locale, however I didn't think that was a good thing to do.

Ramiro Morales
Owner

Closing, adapted and committed code from this PR in 6158c79. See https://code.djangoproject.com/ticket/17181#comment:9 for further details.

Thanks!

Ramiro Morales ramiro closed this January 17, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 5 unique commits by 1 author.

Jun 07, 2012
Craig Blaszczyk allow multiple domains and languages for makemessages command; add op…
…tion to pass in multiple language to the compilemessages command
4fbb940
Craig Blaszczyk move tests into existing test files 305867c
Craig Blaszczyk give tests more sensible names; add hr locale to properly check compi…
…lemessages without deleting the django.mo in the it language
c978636
Jun 13, 2012
Craig Blaszczyk add versionchanged directive cc9eb62
Jun 15, 2012
Craig Blaszczyk add myself to AUTHORS c6b274d
This page is out of date. Refresh to see the latest.
1  AUTHORS
@@ -91,6 +91,7 @@ answer newbie questions, and generally made Django that much better:
91 91
     Mark Biggers <biggers@utsl.com>
92 92
     Paul Bissex <http://e-scribe.com/>
93 93
     Simon Blanchard
  94
+    Craig Blaszczyk <masterjakul@gmail.com>    
94 95
     David Blewett <david@dawninglight.net>
95 96
     Matt Boersma <matt@sprout.org>
96 97
     Artem Gnilov <boobsd@gmail.com>
50  django/core/management/commands/compilemessages.py
@@ -24,34 +24,38 @@ def compile_messages(stderr, locale=None):
24 24
         raise CommandError("This script should be run from the Django Git checkout or your project or app tree, or with the settings module specified.")
25 25
 
26 26
     for basedir in basedirs:
  27
+        dirs = [basedir]
27 28
         if locale:
28  
-            basedir = os.path.join(basedir, locale, 'LC_MESSAGES')
29  
-        for dirpath, dirnames, filenames in os.walk(basedir):
30  
-            for f in filenames:
31  
-                if f.endswith('.po'):
32  
-                    stderr.write('processing file %s in %s\n' % (f, dirpath))
33  
-                    fn = os.path.join(dirpath, f)
34  
-                    if has_bom(fn):
35  
-                        raise CommandError("The %s file has a BOM (Byte Order Mark). Django only supports .po files encoded in UTF-8 and without any BOM." % fn)
36  
-                    pf = os.path.splitext(fn)[0]
37  
-                    # Store the names of the .mo and .po files in an environment
38  
-                    # variable, rather than doing a string replacement into the
39  
-                    # command, so that we can take advantage of shell quoting, to
40  
-                    # quote any malicious characters/escaping.
41  
-                    # See http://cyberelk.net/tim/articles/cmdline/ar01s02.html
42  
-                    os.environ['djangocompilemo'] = pf + '.mo'
43  
-                    os.environ['djangocompilepo'] = pf + '.po'
44  
-                    if sys.platform == 'win32': # Different shell-variable syntax
45  
-                        cmd = 'msgfmt --check-format -o "%djangocompilemo%" "%djangocompilepo%"'
46  
-                    else:
47  
-                        cmd = 'msgfmt --check-format -o "$djangocompilemo" "$djangocompilepo"'
48  
-                    os.system(cmd)
  29
+            dirs = [os.path.join(basedir, l, 'LC_MESSAGES') for l in (locale if isinstance(locale, list) else [locale])]
  30
+            
  31
+        
  32
+        for dir in dirs:
  33
+            for dirpath, dirnames, filenames in os.walk(dir):
  34
+                for f in filenames:
  35
+                    if f.endswith('.po'):
  36
+                        stderr.write('processing file %s in %s\n' % (f, dirpath))
  37
+                        fn = os.path.join(dirpath, f)
  38
+                        if has_bom(fn):
  39
+                            raise CommandError("The %s file has a BOM (Byte Order Mark). Django only supports .po files encoded in UTF-8 and without any BOM." % fn)
  40
+                        pf = os.path.splitext(fn)[0]
  41
+                        # Store the names of the .mo and .po files in an environment
  42
+                        # variable, rather than doing a string replacement into the
  43
+                        # command, so that we can take advantage of shell quoting, to
  44
+                        # quote any malicious characters/escaping.
  45
+                        # See http://cyberelk.net/tim/articles/cmdline/ar01s02.html
  46
+                        os.environ['djangocompilemo'] = pf + '.mo'
  47
+                        os.environ['djangocompilepo'] = pf + '.po'
  48
+                        if sys.platform == 'win32': # Different shell-variable syntax
  49
+                            cmd = 'msgfmt --check-format -o "%djangocompilemo%" "%djangocompilepo%"'
  50
+                        else:
  51
+                            cmd = 'msgfmt --check-format -o "$djangocompilemo" "$djangocompilepo"'
  52
+                        os.system(cmd)
49 53
 
50 54
 
51 55
 class Command(BaseCommand):
52 56
     option_list = BaseCommand.option_list + (
53  
-        make_option('--locale', '-l', dest='locale',
54  
-            help='The locale to process. Default is to process all.'),
  57
+        make_option('--locale', '-l', dest='locale', action="append",
  58
+            help='A locale to process. Default is to process all.'),
55 59
     )
56 60
     help = 'Compiles .po files to .mo files for use with builtin gettext support.'
57 61
 
52  django/core/management/commands/makemessages.py
@@ -248,7 +248,7 @@ def write_po_file(pofile, potfile, domain, locale, verbosity, stdout,
248 248
             raise CommandError(
249 249
                 "errors happened while running msgattrib\n%s" % errors)
250 250
 
251  
-def make_messages(locale=None, domain='django', verbosity=1, all=False,
  251
+def make_messages(locale=None, domain=None, verbosity=1, all=False,
252 252
         extensions=None, symlinks=False, ignore_patterns=None, no_wrap=False,
253 253
         no_location=False, no_obsolete=False, stdout=sys.stdout):
254 254
     """
@@ -283,10 +283,15 @@ def make_messages(locale=None, domain='django', verbosity=1, all=False,
283 283
                 "is not created automatically, you have to create it by hand "
284 284
                 "if you want to enable i18n for your project or application.")
285 285
 
286  
-    if domain not in ('django', 'djangojs'):
  286
+    if domain is None:
  287
+        domains = ['django']
  288
+    else:
  289
+        domains = [domain] if not isinstance(domain, list) else domain
  290
+
  291
+    if any(d not in ('django', 'djangojs') for d in domains):
287 292
         raise CommandError("currently makemessages only supports domains 'django' and 'djangojs'")
288 293
 
289  
-    if (locale is None and not all) or domain is None:
  294
+    if (locale is None and not all) or not domains:
290 295
         message = "Type '%s help %s' for usage information." % (os.path.basename(sys.argv[0]), sys.argv[1])
291 296
         raise CommandError(message)
292 297
 
@@ -302,7 +307,7 @@ def make_messages(locale=None, domain='django', verbosity=1, all=False,
302 307
 
303 308
     locales = []
304 309
     if locale is not None:
305  
-        locales.append(locale)
  310
+        locales += locale.split(',') if not isinstance(locale, list) else locale
306 311
     elif all:
307 312
         locale_dirs = filter(os.path.isdir, glob.glob('%s/*' % localedir))
308 313
         locales = [os.path.basename(l) for l in locale_dirs]
@@ -317,28 +322,31 @@ def make_messages(locale=None, domain='django', verbosity=1, all=False,
317 322
         if not os.path.isdir(basedir):
318 323
             os.makedirs(basedir)
319 324
 
320  
-        pofile = os.path.join(basedir, '%s.po' % domain)
321  
-        potfile = os.path.join(basedir, '%s.pot' % domain)
322  
-
323  
-        if os.path.exists(potfile):
324  
-            os.unlink(potfile)
325  
-
326  
-        for dirpath, file in find_files(".", ignore_patterns, verbosity,
327  
-                stdout, symlinks=symlinks):
328  
-            process_file(file, dirpath, potfile, domain, verbosity, extensions,
329  
-                    wrap, location, stdout)
330  
-
331  
-        if os.path.exists(potfile):
332  
-            write_po_file(pofile, potfile, domain, locale, verbosity, stdout,
333  
-                    not invoked_for_django, wrap, location, no_obsolete)
  325
+        for domain in domains:
  326
+            if verbosity > 1:
  327
+                stdout.write("processing domain %s\n" % domain)
  328
+            pofile = os.path.join(basedir, '%s.po' % domain)
  329
+            potfile = os.path.join(basedir, '%s.pot' % domain)
  330
+    
  331
+            if os.path.exists(potfile):
  332
+                os.unlink(potfile)
  333
+    
  334
+            for dirpath, file in find_files(".", ignore_patterns, verbosity,
  335
+                    stdout, symlinks=symlinks):
  336
+                process_file(file, dirpath, potfile, domain, verbosity, extensions,
  337
+                        wrap, location, stdout)
  338
+    
  339
+            if os.path.exists(potfile):
  340
+                write_po_file(pofile, potfile, domain, locale, verbosity, stdout,
  341
+                        not invoked_for_django, wrap, location, no_obsolete)
334 342
 
335 343
 
336 344
 class Command(NoArgsCommand):
337 345
     option_list = NoArgsCommand.option_list + (
338  
-        make_option('--locale', '-l', default=None, dest='locale',
339  
-            help='Creates or updates the message files for the given locale (e.g. pt_BR).'),
340  
-        make_option('--domain', '-d', default='django', dest='domain',
341  
-            help='The domain of the message files (default: "django").'),
  346
+        make_option('--locale', '-l', default=None, dest='locale', action='append',
  347
+            help='Creates or updates the message files for the given locale(s) (e.g. pt_BR).'),
  348
+        make_option('--domain', '-d', default=None, dest='domain', action='append',
  349
+            help='The domain(s) of the message files (default: "django").'),
342 350
         make_option('--all', '-a', action='store_true', dest='all',
343 351
             default=False, help='Updates the message files for all existing locales.'),
344 352
         make_option('--extension', '-e', dest='extensions',
26  docs/ref/django-admin.txt
@@ -104,12 +104,16 @@ compilemessages
104 104
 Compiles .po files created with ``makemessages`` to .mo files for use with
105 105
 the builtin gettext support. See :doc:`/topics/i18n/index`.
106 106
 
107  
-Use the :djadminopt:`--locale` option to specify the locale to process.
  107
+Use the :djadminopt:`--locale` option to specify the locale(s) to process.
108 108
 If not provided, all locales are processed.
109 109
 
110 110
 Example usage::
111 111
 
112  
-    django-admin.py compilemessages --locale=br_PT
  112
+    django-admin.py compilemessages --locale=pt_BR --locale=fr
  113
+    
  114
+.. versionchanged:: 1.5
  115
+
  116
+Added the option to specify multiple locales.
113 117
 
114 118
 createcachetable
115 119
 ----------------
@@ -422,11 +426,19 @@ Separate multiple extensions with commas or use -e or --extension multiple times
422 426
 
423 427
     django-admin.py makemessages --locale=de --extension=html,txt --extension xml
424 428
 
425  
-Use the :djadminopt:`--locale` option to specify the locale to process.
  429
+Use the :djadminopt:`--locale` option to specify the locale(s) to process.
426 430
 
427 431
 Example usage::
428 432
 
429  
-    django-admin.py makemessages --locale=br_PT
  433
+    django-admin.py makemessages --locale=pt_BR --locale=fr
  434
+    
  435
+You can also use commas to separate multiple locales::
  436
+
  437
+    django-admin.py makemessages --locale=de,fr,pt_BR
  438
+    
  439
+.. versionchanged:: 1.5
  440
+
  441
+Added the option to specify multiple locales.
430 442
 
431 443
 .. django-admin-option:: --domain
432 444
 
@@ -436,6 +448,12 @@ Currently supported:
436 448
 * ``django`` for all ``*.py``, ``*.html`` and ``*.txt`` files (default)
437 449
 * ``djangojs`` for ``*.js`` files
438 450
 
  451
+
  452
+Example usage::
  453
+
  454
+    django-admin.py makemessages --domain=django --domain=djangojs
  455
+    
  456
+    
439 457
 .. django-admin-option:: --symlinks
440 458
 
441 459
 Use the ``--symlinks`` or ``-s`` option to follow symlinks to directories when
44  tests/regressiontests/i18n/commands/compilation.py
... ...
@@ -1,11 +1,17 @@
1 1
 import os
2 2
 from io import BytesIO
  3
+from StringIO import StringIO
3 4
 
  5
+from django.conf import settings
4 6
 from django.core.management import call_command, CommandError
  7
+from django.core.management.commands import compilemessages
5 8
 from django.test import TestCase
6 9
 from django.test.utils import override_settings
7 10
 from django.utils import translation
8 11
 
  12
+
  13
+
  14
+
9 15
 test_dir = os.path.abspath(os.path.dirname(__file__))
10 16
 
11 17
 class MessageCompilationTests(TestCase):
@@ -66,3 +72,41 @@ def test_percent_symbol_escaping(self):
66 72
             t = Template('{% load i18n %}{% trans "Completed 50%% of all the tasks" %}')
67 73
             rendered = t.render(Context({}))
68 74
             self.assertEqual(rendered, 'IT translation of Completed 50%% of all the tasks')
  75
+
  76
+
  77
+class CompilationMultipleLocalesTestCase(TestCase):
  78
+    MO_FILE_HR = None
  79
+    MO_FILE_FR = None
  80
+    
  81
+    def setUp(self):
  82
+        self._old_locale_paths = settings.LOCALE_PATHS
  83
+        self.stderr = StringIO()
  84
+        self.localedir = os.path.join(
  85
+            os.path.dirname(os.path.abspath(__file__)), 'locale'
  86
+        )
  87
+        settings.LOCALE_PATHS = [self.localedir]
  88
+        self.MO_FILE_HR = os.path.join(self.localedir, 'hr/LC_MESSAGES/django.mo')
  89
+        self.MO_FILE_FR = os.path.join(self.localedir, 'fr/LC_MESSAGES/django.mo')
  90
+        
  91
+    def tearDown(self):
  92
+        settings.LOCALE_PATHS = self._old_locale_paths
  93
+        self.stderr.close()
  94
+        self._rmfile(os.path.join(self.localedir, self.MO_FILE_HR))
  95
+        self._rmfile(os.path.join(self.localedir, self.MO_FILE_FR))
  96
+        
  97
+    def _rmfile(self, filepath):
  98
+        if os.path.exists(filepath):
  99
+            os.remove(filepath)
  100
+            
  101
+    def test_one_locale(self):
  102
+        command = compilemessages.Command()
  103
+        command.execute(locale='hr', stderr=self.stderr)
  104
+        
  105
+        self.assertTrue(os.path.exists(self.MO_FILE_HR))
  106
+        
  107
+    def test_multiple_locales(self):
  108
+        command = compilemessages.Command()
  109
+        command.execute(locale=['hr', 'fr'], stderr=self.stderr)
  110
+        
  111
+        self.assertTrue(os.path.exists(self.MO_FILE_HR))
  112
+        self.assertTrue(os.path.exists(self.MO_FILE_FR))
28  tests/regressiontests/i18n/commands/extraction.py
@@ -267,3 +267,31 @@ def test_no_location_disabled(self):
267 267
         with open(self.PO_FILE, 'r') as fp:
268 268
             po_contents = fp.read()
269 269
             self.assertTrue('#: templates/test.html:55' in po_contents)
  270
+
  271
+
  272
+class ExtractionMultipleLocalesTestCase(ExtractorTests):
  273
+    PO_FILE_PT = 'locale/pt/LC_MESSAGES/django.po'
  274
+    PO_FILE_DE = 'locale/de/LC_MESSAGES/django.po'
  275
+    LOCALES = ['pt', 'de', 'ch']
  276
+    
  277
+    
  278
+    def tearDown(self):
  279
+        os.chdir(self.test_dir)
  280
+        for locale in self.LOCALES:
  281
+            try:
  282
+                self._rmrf('locale/%s' % locale)
  283
+            except OSError:
  284
+                pass
  285
+        os.chdir(self._cwd)
  286
+    
  287
+    def test_multiple_locales(self):
  288
+        os.chdir(self.test_dir)
  289
+        management.call_command('makemessages', locale=['pt','de'], verbosity=0)
  290
+        self.assertTrue(os.path.exists(self.PO_FILE_PT))
  291
+        self.assertTrue(os.path.exists(self.PO_FILE_DE))
  292
+    
  293
+    def test_comma_separated_locales(self):
  294
+        os.chdir(self.test_dir)
  295
+        management.call_command('makemessages', locale='pt,de,ch', verbosity=0)
  296
+        self.assertTrue(os.path.exists(self.PO_FILE_PT))
  297
+        self.assertTrue(os.path.exists(self.PO_FILE_DE))
71  tests/regressiontests/i18n/commands/locale/hr/LC_MESSAGES/django.po
... ...
@@ -0,0 +1,71 @@
  1
+# SOME DESCRIPTIVE TITLE.
  2
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
  3
+# This file is distributed under the same license as the PACKAGE package.
  4
+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
  5
+#
  6
+msgid ""
  7
+msgstr ""
  8
+"Project-Id-Version: PACKAGE VERSION\n"
  9
+"Report-Msgid-Bugs-To: \n"
  10
+"POT-Creation-Date: 2011-12-04 04:59-0600\n"
  11
+"PO-Revision-Date: 2011-12-10 19:12-0300\n"
  12
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
  13
+"Language-Team: LANGUAGE <LL@li.org>\n"
  14
+"Language: fr\n"
  15
+"MIME-Version: 1.0\n"
  16
+"Content-Type: text/plain; charset=UTF-8\n"
  17
+"Content-Transfer-Encoding: 8bit\n"
  18
+"Plural-Forms: nplurals=2; plural=(n != 1)\n"
  19
+
  20
+#. Translators: Django template comment for translators
  21
+#: templates/test.html:9
  22
+#, python-format
  23
+msgid "I think that 100%% is more that 50%% of anything."
  24
+msgstr ""
  25
+
  26
+#: templates/test.html:10
  27
+#, python-format
  28
+msgid "I think that 100%% is more that 50%% of %(obj)s."
  29
+msgstr ""
  30
+
  31
+#: templates/test.html:70
  32
+#, python-format
  33
+msgid "Literal with a percent symbol at the end %%"
  34
+msgstr ""
  35
+
  36
+#: templates/test.html:71
  37
+#, python-format
  38
+msgid "Literal with a percent %% symbol in the middle"
  39
+msgstr ""
  40
+
  41
+#: templates/test.html:72
  42
+#, python-format
  43
+msgid "Completed 50%% of all the tasks"
  44
+msgstr ""
  45
+
  46
+#: templates/test.html:73
  47
+#, python-format
  48
+msgctxt "ctx0"
  49
+msgid "Completed 99%% of all the tasks"
  50
+msgstr ""
  51
+
  52
+#: templates/test.html:74
  53
+#, python-format
  54
+msgid "Shouldn't double escape this sequence: %% (two percent signs)"
  55
+msgstr ""
  56
+
  57
+#: templates/test.html:75
  58
+#, python-format
  59
+msgctxt "ctx1"
  60
+msgid "Shouldn't double escape this sequence %% either"
  61
+msgstr ""
  62
+
  63
+#: templates/test.html:76
  64
+#, python-format
  65
+msgid "Looks like a str fmt spec %%s but shouldn't be interpreted as such"
  66
+msgstr "Translation of the above string"
  67
+
  68
+#: templates/test.html:77
  69
+#, python-format
  70
+msgid "Looks like a str fmt spec %% o but shouldn't be interpreted as such"
  71
+msgstr "Translation contains %% for the above string"
4  tests/regressiontests/i18n/tests.py
@@ -29,10 +29,10 @@
29 29
     from .commands.extraction import (ExtractorTests, BasicExtractorTests,
30 30
         JavascriptExtractorTests, IgnoredExtractorTests, SymlinkExtractorTests,
31 31
         CopyPluralFormsExtractorTests, NoWrapExtractorTests,
32  
-        NoLocationExtractorTests)
  32
+        NoLocationExtractorTests, ExtractionMultipleLocalesTestCase)
33 33
 if can_run_compilation_tests:
34 34
     from .commands.compilation import (PoFileTests, PoFileContentsTests,
35  
-        PercentRenderingTests)
  35
+        PercentRenderingTests, CompilationMultipleLocalesTestCase)
36 36
 from .contenttypes.tests import ContentTypeTests
37 37
 from .forms import I18nForm, SelectDateForm, SelectDateWidget, CompanyForm
38 38
 from .models import Company, TestModel
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.