Skip to content
This repository

Drupal gettext #4235

Closed
wants to merge 6 commits into from

13 participants

Clemens Tolboom Don't Add Me To Your Organization a.k.a The Travis Bot Christophe Coevoet Joseph Bielawski Tobias Schultze Benjamin Eberlei Fabien Potencier Gábor Hojtsy Drak Lukas Kahwe Smith Crell chx David Persson
Clemens Tolboom

(This patch is mostly intended as a discussion)

This is a contrib back from Drupal to make PO file parsing better and hopefully use the translation component within Drupal into the near future.

It delivers:

Reading PO Headers
Multi-line source / translation

See more @ clemens-tolboom@69b7d25#L1L56

Clemens Tolboom

(This patch is mostly intended as a discussion)

This is a contrib back from Drupal to make PO file parsing better and hopefully use the translation component within Drupal into the near future.

It delivers:

  • Reading PO Headers
  • Multi-line source / translation

It currently has coding style 'problems' as part of the code is copied verbatim from Drupal to make sure those keep in sync for now.

  • this will be fixed within the next commit as that is bogus :p

Problems from a Drupal perspective are:

  • loading large PO files (or slow servers) are handled by a browser oriented batch interface. Can we do that within the Symfony framework?
  • The loaded PO files (or messages) are editable on a Drupal site. I'm not sure how a Symfony workflow could support this right now. This patch has an 'almost verbatim' copy of http://drupal.org/node/1189184 which is a patch in progress.
  • so code style in not ok (but that code needs to be kept in sync with the progress on Drupal patch)

The POLoader produces a parser log of errors for which there is no way to read.

  • Is that only Gettext PO problem or also ie MO and other resources?

It still needs a test for parsing a header successfully.

The header info is not used anywhere within the system.

  • That sounds bad as the header contains credentials about the translation team.
Clemens Tolboom

I'm not sure where to test the multiline parsing.

  • adding a Translator gives Class 'Symfony\Component\Translation\Tests\Loader\Translator' not found.

I want to make sure the texts are joined correctly by the PoFileLoader.

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged 69b7d25 into e54f4e4).

Christophe Coevoet
Collaborator
stof commented

@clemens-tolboom this class indeed does not exist. You probably forgot a use statement

Christophe Coevoet
Collaborator
stof commented

I will let someone else review it. I don't know gettext enough to know if the implementation is right, and my brain's parser gives me too much warnings when reading the code...

Joseph Bielawski
stloyd commented

@clemens-tolboom First of all you should follow Symfony2 CS.

Tobias Schultze
Collaborator
Tobion commented

@stloyd he wrote that the CS is copied from Drupal and will be fixed later. So no need to tell him again... (especially since its intended as RFC)

Clemens Tolboom

@stof : how many symfony people use PO files? Drupal only uses PO so is this worth the effort?!?

@stloyd & @Tobion : I somehow expected travis to kick my CS ass ... Netbeans does not support two PHP CS formats at once afaik :(

Is the problem about batch loading PO file solvable within the symfony framework?

Benjamin Eberlei
Collaborator

I don't want to crush the party, but this code is GPL licensed.

Fabien Potencier
Owner
fabpot commented

As @beberlei mentioned, the biggest problem is the license of the code... Symfony being licensed under the MIT license.

Christophe Coevoet
Collaborator
stof commented

@clemens-tolboom Travis is about running tests, not fixing CS.

Gábor Hojtsy
goba commented

Looks like Symfony already supported multiline string parsing (storing the string value in the buffer when reading the string). So that does not seem to be new with this (mentioned above as "Multi-line source / translation").

Drak
Collaborator
drak commented

I would suspect anything ported from Drupal might be hopelessly entangled with the GPL and be uncovertable to another license.

Gábor Hojtsy
goba commented

drak: interesting observation. The code posted above is not yet in Drupal and never was. It is from a work in progress patch from the Drupal issue queues. It is based on code from Drupal 7 but is mostly rewritten from the ground up (I think Clemens could vouch for a better estimate on how much different it is). The Drupal 7 Gettext parser is hopelessly ugly (even though it does work well :).

Christophe Coevoet
Collaborator
stof commented

@clemens-tolboom how much of this code was already in Drupal previously (and so subject to the GPL) ? For the new code, licensing is not an issue as it can be submitted as MIT

Drak
Collaborator
drak commented
Clemens Tolboom

Wow ... thanks for the quick feedback :)

And darn ... you guys are right about license (why didn't I think of that)

Doing a diff on prepared files (locale.inc from Drupal 7 and the PoFileLoader.php) to only check the readLine code the following stats show we have license problems.

Lines that differ ... mostly by having a $this-> in front of state variables
$ diff --suppress-common-lines -F 100 -y -E -b PoFileLoader.php locale-d7.inc | wc -l
137

where total lines counts are
264 PoFileLoader.php
265 locale-d7.inc

Almost 50% are common. Most of the rest has a $this-> and some lines are new related to Batch/state management.

I think I should retract this pull request right?

Sorry for wasting your time :(

Lukas Kahwe Smith

can you figure out the original contributors and ask them for permission?

Gábor Hojtsy
goba commented

This part of the Drupal codebase has a history of about 9 years. It would likely be impossible to track all the people down who wrote it originally and then fixed bugs and added new features to them.

Lukas Kahwe Smith

I see .. in that case there are 2 options:
1) you simply leave this file inside the Drupal code base, but try to make it easy for users to grab standalone
2) you write the code from scratch, while using the original code as inspiration .. this is perfectly legal as we are only talking about copyright and not patents here.

Crell
Crell commented

We're trying to do option 1 already as part of internal cleanup. Long road, but hopefully. :-) That still keeps it GPL, though.

If clemens is up for option 2, I would have no problem backing us using that in place of the existing code provided it is in fact technically better than what we have now. We'd have to work out how we deal with "Drupal-created 3rd party dependencies", though. That's a discussion better had over on Drupal.org than here.

Clemens Tolboom

@Crell If we go for option 2 we from Drupal only have to add another symfony component 'Translation' to the plate right?

chx
chx commented

Gabor, let me challenge that. 9 years or not, we have git blame and this code , as far as I can remember is not a particularly often changed. The likely culprits are you and Gerhard anyways and both of you are still within easy reach. Someone needs to run a bunch of git blames to figure this out. I think it's doable.

Benjamin Eberlei
Collaborator

git blame is not enough, whitespace changes or coding standard adjustments move the blame away from the copyright holders of the code. You have to walk the history of files back, plus handling potential moves of the file or parts of the code. Its really not that easy :)

Lukas Kahwe Smith

i just clicked through https://github.com/drupal/drupal/commits/7.x/includes/locale.inc?page=8 .. seems like mostly the same names .. though many commits were done on behalf of someone else .. but as @chx points out .. its only a portion of the file ..

chx
chx commented

It really is not that hard. We need someone with more time than me to verify this completely but I think only 57c9a13e, 30c2e89c, 9841e29a, 941139bf are responsible. At least that's when the import one was introduced. This suggests the same names as I suspected originally plus kkaefer/timcn.

Gábor Hojtsy
goba commented

Well, for that, we should assume that the commit message surely includes everybody who contributed to the patches involved. Is that legally possible to assume? Eg. how do you parse "New locale module thanks to Gerhard, Goba, Marco, Kristjan and others." for the complete list of people involved? From http://drupalcode.org/project/drupal.git/commit/1831e1b690f02d7f551d38ef88a0ba200f786497 Then how do you dig up the discussion history from 8 years ago (2004) on that commit to figure out who were the "others"?

chx
chx commented

Do we have code from that commit? If yes, that's unfortunate.

Gábor Hojtsy
goba commented

Well, you might be surprised how little changed in the actual .po file / plural formula parsing code since then. For example, the much dreaded eval() for formula evaluation was introduced with this 2004 patch and is still used in the 2011 released Drupal 7. But this is just the big early commit. There are more than likely intermediate commits with "and others" mentioned from times where we did not have an issue queue based process to track people down retroactively.

Clemens Tolboom

Puzzled by the multiline problem and other I did another #4249

@ #drupal-i18n we discussed our effort and decided to follow two tracks of which another is to try to use as much as possible of the Translation component:

I'm not sure whether we should close this pull for now. I'm willing to write a new parser when a sponsor is available.

David Persson

As one of the original authors of the currently used implementation. I'd also be interested in any developments regarding plural rule parsing and evaluation without using evil eval(). I remember CakePHP wasn't using eval for that task. You may also want to look if anything has changed/improved since you last looked on the implementation.

https://github.com/UnionOfRAD/lithium/blob/master/g11n/catalog/adapter/Gettext.php
https://github.com/UnionOfRAD/lithium/blob/master/tests/cases/g11n/catalog/adapter/GettextTest.php

Full disclosure: I'm part of the Lithium team and CakePHP alumni.

However: I just discovered you're using the implementation and didn't know it before - hope it does a good job.

chx
chx commented

http://drupal.org/node/1273968 rips out the eval from Drupal. This code, however, was written by me, and as such, unless someone convinces me otherwise, is GPL.

David Persson

Than it can't be used in non-GPL projects :(
But nice work!

Clemens Tolboom

I retract this pull request as it is be replaced by #4249

Clemens Tolboom clemens-tolboom closed this
Clemens Tolboom

From a Symfony perspective we have PluralizationRules which can be extended by ones own pluralization rule.

I hope @chx can interface with that for Drupal as my Drupal patch http://drupal.org/node/1570346 (needs lots of work still) is using MessageSelector already.

@davidpersson tnx for the pointers.

Drak
Collaborator
drak commented

@davidpersson - there is nothing evil about eval in this particular circumstance: please see #2630

Drak
Collaborator
drak commented
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
446  src/Symfony/Component/Translation/Loader/PoFileLoader.php
@@ -65,46 +65,420 @@ private function parse($resource)
65 65
         $messages = array();
66 66
         $item = $defaults;
67 67
 
68  
-        while ($line = fgets($stream)) {
69  
-            $line = trim($line);
70  
-
71  
-            if ($line === '') {
72  
-                if (is_array($item['translated'])) {
73  
-                    $messages[$item['ids']['singular']] = stripslashes($item['translated'][0]);
74  
-                    if (isset($item['ids']['plural'])) {
75  
-                        $plurals = array();
76  
-                        foreach ($item['translated'] as $plural => $translated) {
77  
-                            $plurals[] = sprintf('{%d} %s', $plural, $translated);
78  
-                        }
79  
-                        $messages[$item['ids']['plural']] = stripcslashes(implode('|', $plurals));
80  
-                    }
81  
-                } elseif(!empty($item['ids']['singular'])) {
82  
-                    $messages[$item['ids']['singular']] = stripslashes($item['translated']);
83  
-                }
84  
-                $item = $defaults;
85  
-            } elseif (substr($line, 0, 7) === 'msgid "') {
86  
-                $item['ids']['singular'] = substr($line, 7, -1);
87  
-            } elseif (substr($line, 0, 8) === 'msgstr "') {
88  
-                $item['translated'] = substr($line, 8, -1);
89  
-            } elseif ($line[0] === '"') {
90  
-                $continues = isset($item['translated']) ? 'translated' : 'ids';
91  
-
92  
-                if (is_array($item[$continues])) {
93  
-                    end($item[$continues]);
94  
-                    $item[$continues][key($item[$continues])] .= substr($line, 1, -1);
95  
-                } else {
96  
-                    $item[$continues] .= substr($line, 1, -1);
97  
-                }
98  
-            } elseif (substr($line, 0, 14) === 'msgid_plural "') {
99  
-                $item['ids']['plural'] = substr($line, 14, -1);
100  
-            } elseif (substr($line, 0, 7) === 'msgstr[') {
101  
-                $size = strpos($line, ']');
102  
-                $item['translated'][(integer) substr($line, 7, 1)] = substr($line, $size + 3, -1);
103  
-            }
  68
+        // Prepare state
  69
+        $this->lineno = 0;
  70
+        $this->_fd = $stream;
  71
+        $this->finished = FALSE;
  72
+        $this->context = 'COMMENT';
104 73
 
  74
+        while(!$this->finished) {
  75
+          $this->readItem();
  76
+          $translation = $this->translation;
  77
+          $this->translation = null;
  78
+          // now map our parsed data to $messages structure
  79
+          if (!is_null($translation)) {
  80
+            //var_dump($translation);
  81
+            if ($translation->plural) {
  82
+              $messages[$translation->source[0]] = $translation->translation[0];
  83
+              //var_dump($translation);
  84
+              $trans = $translation->translation;
  85
+              $plurals = array();
  86
+              foreach ($trans as $plural => $translated) {
  87
+                $plurals[] = sprintf('{%d} %s', $plural, $translated);
  88
+              }
  89
+              $messages[$translation->source[1]] = stripcslashes(implode('|', $plurals));
  90
+            }
  91
+            else {
  92
+              $messages[$translation->source] = $translation->translation;
  93
+            }
  94
+          }
105 95
         }
  96
+
106 97
         fclose($stream);
107 98
 
108 99
         return array_filter($messages);
109 100
     }
  101
+
  102
+    /*
  103
+     * Code is taken from http://drupal.org/node/1189184 work in progres
  104
+     *
  105
+     * That is it is copied verbatim from the mentioned patch applied to
  106
+     * Drupal Core then pasted into here.
  107
+     *
  108
+     * Edits aka exceptions are:
  109
+     * - "new POItem()" is replaced by "(object) array()"
  110
+     *
  111
+     * TODO:
  112
+     * - add a header to po file to test it is skipped nicely
  113
+     */
  114
+
  115
+  /**
  116
+   * Reads the header from the given input stream.
  117
+   *
  118
+   * We need to read the optional first COMMENT
  119
+   * Next read a MSGID and a MSGSTR
  120
+   *
  121
+   * TODO: is a header required?
  122
+   */
  123
+  private function readHeader() {
  124
+    $translation = $this->readTranslation();
  125
+    $header = new PoHeader;
  126
+    $header->setFromString(trim($translation->translation));
  127
+    $this->_header = $header;
  128
+  }
  129
+
  130
+  /**
  131
+   * Return a translation object (singular or plural)
  132
+   *
  133
+   * @todo Define a translation object for this purpose?
  134
+   *       Or use a standard class for better performance?
  135
+   */
  136
+  public function readItem() {
  137
+    $this->readTranslation();
  138
+    return $this->translation;
  139
+  }
  140
+
  141
+  private function readTranslation() {
  142
+    $this->translation = NULL;
  143
+    while (!$this->finished && is_null($this->translation)) {
  144
+      $this->readLine();
  145
+    }
  146
+    return $this->translation;
  147
+  }
  148
+
  149
+  /**
  150
+   * Reads a line from a PO file.
  151
+   *
  152
+   * While reading a line it's content is processed according to current
  153
+   * context.
  154
+   *
  155
+   * The parser context. Can be:
  156
+   *  - 'COMMENT' (#)
  157
+   *  - 'MSGID' (msgid)
  158
+   *  - 'MSGID_PLURAL' (msgid_plural)
  159
+   *  - 'MSGCTXT' (msgctxt)
  160
+   *  - 'MSGSTR' (msgstr or msgstr[])
  161
+   *  - 'MSGSTR_ARR' (msgstr_arg)
  162
+   *
  163
+   * @return boolean FALSE or NULL
  164
+   */
  165
+  private function readLine() {
  166
+    // a string or boolean FALSE
  167
+    $line = fgets($this->_fd);
  168
+    $this->finished = ($line === FALSE);
  169
+    if (!$this->finished) {
  170
+
  171
+      if ($this->lineno == 0) {
  172
+        // The first line might come with a UTF-8 BOM, which should be removed.
  173
+        $line = str_replace("\xEF\xBB\xBF", '', $line);
  174
+        // Current plurality for 'msgstr[]'.
  175
+        $this->plural = 0;
  176
+      }
  177
+
  178
+      $this->lineno++;
  179
+
  180
+      // Trim away the linefeed.
  181
+      $line = trim(strtr($line, array("\\\n" => "")));
  182
+
  183
+      if (!strncmp('#', $line, 1)) {
  184
+        // Lines starting with '#' are comments.
  185
+
  186
+        if ($this->context == 'COMMENT') {
  187
+          // Already in comment token, insert the comment.
  188
+          $this->current['#'][] = substr($line, 1);
  189
+        }
  190
+        elseif (($this->context == 'MSGSTR') || ($this->context == 'MSGSTR_ARR')) {
  191
+          // We are currently in string token, close it out.
  192
+          $this->saveOneString();
  193
+
  194
+          // Start a new entry for the comment.
  195
+          $this->current = array();
  196
+          $this->current['#'][] = substr($line, 1);
  197
+
  198
+          $this->context = 'COMMENT';
  199
+          return TRUE;
  200
+        }
  201
+        else {
  202
+          // A comment following any other token is a syntax error.
  203
+          $this->log('The translation file %filename contains an error: "msgstr" was expected but not found on line %line.', $this->lineno);
  204
+          return FALSE;
  205
+        }
  206
+        return;
  207
+      }
  208
+      elseif (!strncmp('msgid_plural', $line, 12)) {
  209
+        // A plural form for the current message.
  210
+
  211
+        if ($this->context != 'MSGID') {
  212
+          // A plural form cannot be added to anything else but the id directly.
  213
+          $this->log('The translation file %filename contains an error: "msgid_plural" was expected but not found on line %line.', $this->lineno);
  214
+          return FALSE;
  215
+        }
  216
+
  217
+        // Remove 'msgid_plural' and trim away whitespace.
  218
+        $line = trim(substr($line, 12));
  219
+        // At this point, $line should now contain only the plural form.
  220
+
  221
+        $quoted = $this->parseQuoted($line);
  222
+        if ($quoted === FALSE) {
  223
+          // The plural form must be wrapped in quotes.
  224
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  225
+          return FALSE;
  226
+        }
  227
+
  228
+        // Append the plural form to the current entry.
  229
+        if (is_string($this->current['msgid'])) {
  230
+          // The first value was stored as string. Now we know the context is
  231
+          // plural, it is converted to array.
  232
+          $this->current['msgid'] = array($this->current['msgid']);
  233
+        }
  234
+        $this->current['msgid'][] = $quoted;
  235
+
  236
+        $this->context = 'MSGID_PLURAL';
  237
+        return;
  238
+      }
  239
+      elseif (!strncmp('msgid', $line, 5)) {
  240
+        // Starting a new message.
  241
+
  242
+        if (($this->context == 'MSGSTR') || ($this->context == 'MSGSTR_ARR')) {
  243
+          // We are currently in a message string, close it out.
  244
+          $this->saveOneString();
  245
+
  246
+          // Start a new context for the id.
  247
+          $this->current = array();
  248
+        }
  249
+        elseif ($this->context == 'MSGID') {
  250
+          // We are currently already in the context, meaning we passed an id with no data.
  251
+          $this->log('The translation file %filename contains an error: "msgid" is unexpected on line %line.', $this->lineno);
  252
+          return FALSE;
  253
+        }
  254
+
  255
+        // Remove 'msgid' and trim away whitespace.
  256
+        $line = trim(substr($line, 5));
  257
+        // At this point, $line should now contain only the message id.
  258
+
  259
+        $quoted = $this->parseQuoted($line);
  260
+        if ($quoted === FALSE) {
  261
+          // The message id must be wrapped in quotes.
  262
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  263
+          return FALSE;
  264
+        }
  265
+
  266
+        $this->current['msgid'] = $quoted;
  267
+        $this->context = 'MSGID';
  268
+        return;
  269
+      }
  270
+      elseif (!strncmp('msgctxt', $line, 7)) {
  271
+        // Starting a new context.
  272
+
  273
+        if (($this->context == 'MSGSTR') || ($this->context == 'MSGSTR_ARR')) {
  274
+          // We are currently in a message, start a new one.
  275
+          $this->saveOneString($this->current);
  276
+          $this->current = array();
  277
+        }
  278
+        elseif (!empty($this->current['msgctxt'])) {
  279
+          // A context cannot apply to another context.
  280
+          $this->log('The translation file %filename contains an error: "msgctxt" is unexpected on line %line.', $this->lineno);
  281
+          return FALSE;
  282
+        }
  283
+
  284
+        // Remove 'msgctxt' and trim away whitespaces.
  285
+        $line = trim(substr($line, 7));
  286
+        // At this point, $line should now contain the context.
  287
+
  288
+        $quoted = $this->parseQuoted($line);
  289
+        if ($quoted === FALSE) {
  290
+          // The context string must be quoted.
  291
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  292
+          return FALSE;
  293
+        }
  294
+
  295
+        $this->current['msgctxt'] = $quoted;
  296
+
  297
+        $this->context = 'MSGCTXT';
  298
+        return;
  299
+      }
  300
+      elseif (!strncmp('msgstr[', $line, 7)) {
  301
+        // A message string for a specific plurality.
  302
+
  303
+        if (($this->context != 'MSGID') && ($this->context != 'MSGCTXT') && ($this->context != 'MSGID_PLURAL') && ($this->context != 'MSGSTR_ARR')) {
  304
+          // Message strings must come after msgid, msgxtxt, msgid_plural, or other msgstr[] entries.
  305
+          $this->log('The translation file %filename contains an error: "msgstr[]" is unexpected on line %line.', $this->lineno);
  306
+          return FALSE;
  307
+        }
  308
+
  309
+        // Ensure the plurality is terminated.
  310
+        if (strpos($line, ']') === FALSE) {
  311
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  312
+          return FALSE;
  313
+        }
  314
+
  315
+        // Extract the plurality.
  316
+        $frombracket = strstr($line, '[');
  317
+        $this->plural = substr($frombracket, 1, strpos($frombracket, ']') - 1);
  318
+
  319
+        // Skip to the next whitespace and trim away any further whitespace, bringing $line to the message data.
  320
+        $line = trim(strstr($line, " "));
  321
+
  322
+        $quoted = $this->parseQuoted($line);
  323
+        if ($quoted === FALSE) {
  324
+          // The string must be quoted.
  325
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  326
+          return FALSE;
  327
+        }
  328
+        if (!isset($this->current['msgstr']) || !is_array($this->current['msgstr'])) {
  329
+          $this->current['msgstr'] = array();
  330
+        }
  331
+
  332
+        $this->current['msgstr'][$this->plural] = $quoted;
  333
+
  334
+        $this->context = 'MSGSTR_ARR';
  335
+        return;
  336
+      }
  337
+      elseif (!strncmp("msgstr", $line, 6)) {
  338
+        // A string for the an id or context.
  339
+
  340
+        if (($this->context != 'MSGID') && ($this->context != 'MSGCTXT')) {
  341
+          // Strings are only valid within an id or context scope.
  342
+          $this->log('The translation file %filename contains an error: "msgstr" is unexpected on line %line.', $this->lineno);
  343
+          return FALSE;
  344
+        }
  345
+
  346
+        // Remove 'msgstr' and trim away away whitespaces.
  347
+        $line = trim(substr($line, 6));
  348
+        // At this point, $line should now contain the message.
  349
+
  350
+        $quoted = $this->parseQuoted($line);
  351
+        if ($quoted === FALSE) {
  352
+          // The string must be quoted.
  353
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  354
+          return FALSE;
  355
+        }
  356
+
  357
+        $this->current['msgstr'] = $quoted;
  358
+
  359
+        $this->context = 'MSGSTR';
  360
+        return;
  361
+      }
  362
+      elseif ($line != '') {
  363
+        // Anything that is not a token may be a continuation of a previous token.
  364
+
  365
+        $quoted = $this->parseQuoted($line);
  366
+        if ($quoted === FALSE) {
  367
+          // The string must be quoted.
  368
+          $this->log('The translation file %filename contains a syntax error on line %line.', $this->lineno);
  369
+          return FALSE;
  370
+        }
  371
+
  372
+        // Append the string to the current context.
  373
+        if (($this->context == 'MSGID') || ($this->context == 'MSGID_PLURAL')) {
  374
+          if (is_array($this->current['msgid'])) {
  375
+            // Add string to last array element.
  376
+            $last_index = count($this->current['msgid']) - 1;
  377
+            $this->current['msgid'][$last_index] .= $quoted;
  378
+          }
  379
+          else {
  380
+            $this->current['msgid'] .= $quoted;
  381
+          }
  382
+        }
  383
+        elseif ($this->context == 'MSGCTXT') {
  384
+          $this->current['msgctxt'] .= $quoted;
  385
+        }
  386
+        elseif ($this->context == 'MSGSTR') {
  387
+          $this->current['msgstr'] .= $quoted;
  388
+        }
  389
+        elseif ($this->context == 'MSGSTR_ARR') {
  390
+          $this->current['msgstr'][$this->plural] .= $quoted;
  391
+        }
  392
+        else {
  393
+          // No valid context to append to.
  394
+          $this->log('The translation file %filename contains an error: there is an unexpected string on line %line.', $this->lineno);
  395
+          return FALSE;
  396
+        }
  397
+        return;
  398
+      }
  399
+    }
  400
+
  401
+    // Empty line read or EOF of PO file, closed out the last entry.
  402
+    if (($this->context == 'MSGSTR') || ($this->context == 'MSGSTR_ARR')) {
  403
+      $this->saveOneString($this->current);
  404
+      $this->current = array();
  405
+    }
  406
+    elseif ($this->context != 'COMMENT') {
  407
+      $this->log('The translation file %filename ended unexpectedly at line %line.', $this->lineno);
  408
+      return FALSE;
  409
+    }
  410
+  }
  411
+
  412
+  /**
  413
+   * Sets an error message if an error occurred during locale file parsing.
  414
+   *
  415
+   * @param $message
  416
+   *   The message to be translated.
  417
+   * @param $lineno
  418
+   *   An optional line number argument.
  419
+   */
  420
+  protected function log($message, $lineno = NULL) {
  421
+    if (isset($lineno)) {
  422
+      $vars['%line'] = $lineno;
  423
+    }
  424
+    $t = get_t();
  425
+    $this->errorLog[] = $t($message, $vars);
  426
+  }
  427
+
  428
+  /**
  429
+   * Store the parsed values as translation object.
  430
+   */
  431
+  public function saveOneString() {
  432
+    $value = $this->current;
  433
+    $plural = FALSE;
  434
+
  435
+    $comments = '';
  436
+    if (isset($value['#'])) {
  437
+      $comments = $this->shortenComments($value['#']);
  438
+    }
  439
+
  440
+    if (is_array($value['msgstr'])) {
  441
+      // Sort plural variants by their form index.
  442
+      ksort($value['msgstr']);
  443
+      $plural = TRUE;
  444
+    }
  445
+
  446
+    $translation = (object) array();
  447
+    $translation->context = isset($value['msgctxt']) ? $value['msgctxt'] : '';
  448
+    $translation->source = $value['msgid'];
  449
+    $translation->translation = $value['msgstr'];
  450
+    $translation->plural = $plural;
  451
+    $translation->comment = $comments;
  452
+
  453
+    $this->translation = $translation;
  454
+
  455
+    $this->context = 'COMMENT';
  456
+  }
  457
+
  458
+  /**
  459
+   * Parses a string in quotes.
  460
+   *
  461
+   * @param $string
  462
+   *   A string specified with enclosing quotes.
  463
+   *
  464
+   * @return
  465
+   *   The string parsed from inside the quotes.
  466
+   */
  467
+  function parseQuoted($string) {
  468
+    if (substr($string, 0, 1) != substr($string, -1, 1)) {
  469
+      return FALSE;   // Start and end quotes must be the same
  470
+    }
  471
+    $quote = substr($string, 0, 1);
  472
+    $string = substr($string, 1, -1);
  473
+    if ($quote == '"') {        // Double quotes: strip slashes
  474
+      return stripcslashes($string);
  475
+    }
  476
+    elseif ($quote == "'") {  // Simple quote: return as-is
  477
+      return $string;
  478
+    }
  479
+    else {
  480
+      return FALSE;             // Unrecognized quote
  481
+    }
  482
+  }
  483
+
110 484
 }
28  src/Symfony/Component/Translation/Tests/Loader/PoFileLoaderTest.php
@@ -54,4 +54,32 @@ public function testLoadDoesNothingIfEmpty()
54 54
         $this->assertEquals('en', $catalogue->getLocale());
55 55
         $this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
56 56
     }
  57
+
  58
+    public function testLoadMultiline()
  59
+    {
  60
+        $loader = new PoFileLoader();
  61
+        $resource = __DIR__.'/../fixtures/multiline.po';
  62
+        $catalogue = $loader->load($resource, 'en', 'domain1');
  63
+
  64
+        $this->assertEquals(3, count($catalogue->all('domain1')));
  65
+        //var_dump($catalogue->all('domain1'));
  66
+        /*
  67
+        $translator = new Translator('en', new MessageSelector());
  68
+        $translator->addLoader('PoFileLoader', $loader);
  69
+        $translator->addResource('PoFileLoader', $resource, 'nl');
  70
+
  71
+        // force catalogue loading
  72
+        $translator->trans('Translation has multiple lines.');
  73
+
  74
+        $translator->setFallbackLocale('nl');
  75
+
  76
+        $this->assertEquals('trans single line', $translator->trans('both single line'));
  77
+
  78
+        $this->assertEquals('trans multi line', $translator->trans('source single line'));
  79
+
  80
+        $this->assertEquals('trans single line', $translator->trans('source multi line'));
  81
+         *
  82
+         */
  83
+    }
  84
+
57 85
 }
12  src/Symfony/Component/Translation/Tests/fixtures/multiline.po
... ...
@@ -0,0 +1,12 @@
  1
+msgid "both single line"
  2
+msgstr "trans single line"
  3
+
  4
+msgid "source single line"
  5
+msgstr ""
  6
+"trans "
  7
+"multi line"
  8
+
  9
+msgid ""
  10
+"source multi "
  11
+"line"
  12
+msgstr "trans single line"
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.