Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gulp task to generate POT file and .php clone files for WPML #1465

Closed
luism-s opened this issue Jun 25, 2017 · 10 comments
Closed

Gulp task to generate POT file and .php clone files for WPML #1465

luism-s opened this issue Jun 25, 2017 · 10 comments
Assignees

Comments

@luism-s
Copy link
Contributor

@luism-s luism-s commented Jun 25, 2017

This isn't an issue, but more of a procedure suggestion. I don't know what would the best place for something like this be, so I'll post this here.

I was having an hard time adapting POEdit to scan .twig files, so I figured that I needed a different approach in order to register the strings from .twig files in WPML.

So I coded a gulp script with the following logic:

  • Iterates over all given .twig files
  • Search and Replace for gettext functions in .the twig files and wraps them around php tags
  • Outputs each file as .php
  • Send files to a cache folder
  • Iterate over all .php files (cache included)
  • Generate .pot
  • ta daaaa

Dependencies:

npm install --save-dev del gulp gulp-if run-sequence gulp-rename gulp-replace gulp-wp-pot

Code (as if the script is in the theme's root):


/**
 * Gettext Scanner Script for Twig Projects
 * v1.0
 * 
 * Developed by luism-s
 * https://github.com/luism-s
 */

/**
 * Usage:
 * `gulp pot`
 *
 * Logic:
 *
 * - Iterates over all given .twig files
 * - Search and replace for gettext functions in Twig files and wraps them around PHP tags
 * - Outputs each file as .php into a cache folder
 * - Scan all .php files for gettext functions using 'gulp-wp-pot' (cache included)
 * - Generate .pot file
 */

/**
 * Dependencies:
 *
 * `npm install --save-dev gulp gulp-wp-pot gulp-replace gulp-rename del gulp-if run-sequence`
 */
var gulp    = require('gulp');
var wpPot   = require('gulp-wp-pot');
var replace = require('gulp-replace');
var rename  = require('gulp-rename');
var del     = require('del');
var gulpif  = require('gulp-if');
var runSequence  = require('run-sequence');

/**
 * Configuration Options
 * 
 * All paths are as if this script is 
 * located in the root of the theme and all the Twig
 * files are located under /views
 */
var config = {
  "text_domain" : "theme-test",       // Replace with your domain
  "twig_files"  : "views/**/*.twig",  // Twig Files
  "php_files"   : "**/*.php",         // PHP Files
  "cacheFolder" : "views/cache",      // Cache Folder
  "destFolder"  : "languages",        // Folder where .pot file will be saved
  "keepCache"   : true                // Delete cache files after script finishes
};

/**
 * Main Task
 */
gulp.task('pot', function(callback) {
  runSequence('compile-twigs', 'generate-pot', callback);
});

/**
 * Generate POT file from all .php files in the theme,
 * including cache folder.
 */
gulp.task('generate-pot', function () {
  return gulp.src(config.php_files)
    .pipe(wpPot( {
      domain: config.text_domain
    } ))
    .pipe(gulp.dest(config.dest_folder + '/' + config.text_domain + '.pot'))
    .pipe(gulpif(!config.keep_cache, del.bind(null, [config.cache_folder], {force: true})));
});

/**
 * Fake Twig Gettext Compiler
 * 
 * Searches and replaces all occurences of __('string', 'domain'), _e('string', 'domain') and so on,
 * with <?php __('string', 'domain'); ?> or <?php _e('string', 'domain'); ?> and saves the content
 * in a .php file with the same name in the cache folder.
 * 
 * Functions supported:
 * 
 * Simple: __(), _e(), translate()
 * Plural: _n()
 * Disambiguation: _x(), _ex(), _nx()
 * Noop: _n_loop(), _nx_noop()
 * 
 * TODO:
 * - Support translate_nooped_plural() function
 * - Skip gettext calls insied Twig comments (ex. {# __('string', 'domain') #})
 */
gulp.task('compile-twigs', function(){
  del.bind(null, [config.cache_folder], {force: true})
  
  /**
   * __
   * _e
   * _x
   * _xn
   * _ex
   * _n_noop
   * _nx_noop
   * translate  -> Match __,  _e, _x and so on
   * \(         -> Match (
   * \s*?       -> Match empty space 0 or infinite times, as few times as possible (ungreedy)
   * [\'\"]     -> Match ' or "
   * .+?        -> Match any character, 1 to infinite times, as few times as possible (ungreedy)
   * ,          -> Match ,
   * .+?        -> Match any character, 1 to infinite times, as few times as possible (ungreedy)
   * \)         -> Match )
   */
  var gettext_regex = {
    simple : /(__|_e|translate)\(\s*?[\'\"].+?[\'\"]\s*?,\s*?[\'\"].+?[\'\"]\s*?\)/g,
    plural : /_n\(\s*?[\'\"].*?[\'\"]\s*?,\s*?[\'\"].*?[\'\"]\s*?,\s*?.+?\s*?,\s*?[\'\"].+?[\'\"]\s*?\)/g,
    disambiguation : /(_x|_ex|_nx)\(\s*?[\'\"].+?[\'\"]\s*?,\s*?[\'\"].+?[\'\"]\s*?,\s*?[\'\"].+?[\'\"]\s*?\)/g,
    noop : /(_n_noop|_nx_noop)\((\s*?[\'\"].+?[\'\"]\s*?),(\s*?[\'\"]\w+?[\'\"]\s*?,){0,1}\s*?[\'\"].+?[\'\"]\s*?\)/g
  };

  gulp.src(config.twig_files)
    .pipe(replace(gettext_regex.simple, function( match ){
      return '<?php ' + match + '; ?>';
    }))
    .pipe(replace(gettext_regex.plural, function( match ){
      return '<?php ' + match + '; ?>';
    }))
    .pipe(replace(gettext_regex.disambiguation, function( match ){
      return '<?php ' + match + '; ?>';
    }))
    .pipe(replace(gettext_regex.noop, function( match ){
      return '<?php ' + match + '; ?>';
    }))
    .pipe(rename(function( file_path ) {
      file_path.extname = ".php"
    }))
    .pipe(gulp.dest(config.cache_folder));
});

Hope this helps someone

@nlemoine

This comment has been minimized.

Copy link
Contributor

@nlemoine nlemoine commented Jun 27, 2017

Thanks for sharing this! I struggled with this myself.

Alternatives ways to achieve this:

@luism-s

This comment has been minimized.

Copy link
Contributor Author

@luism-s luism-s commented Jun 27, 2017

@nlemoine
I also knew those ways, but I was having problemas with the second option. As of the first I wasn't feeling like spending money :)

Moreover, by generating cache files, WPML is able to scan those files and register the strings by itself

@gchtr

This comment has been minimized.

Copy link
Member

@gchtr gchtr commented Jul 22, 2017

@luism-s I really like this solution as an alternative to the existing methods, especially because it might be a better fit for people who like to work with WPML.

Would you mind adding your code to a gist or create a repository out of it, so that we can add your solution to the Internationalization guide in the documentation and link to the code?

@luism-s

This comment has been minimized.

Copy link
Contributor Author

@luism-s luism-s commented Jul 23, 2017

@gchtr Yeah sure! I'll do it shortly when I have some time and post it here ;)

@luism-s

This comment has been minimized.

Copy link
Contributor Author

@luism-s luism-s commented Jul 26, 2017

@gchtr gchtr self-assigned this Oct 5, 2017
@gchtr

This comment has been minimized.

Copy link
Member

@gchtr gchtr commented Apr 1, 2018

Hey @luism-s

I tried out your solution with a sample Twig file, yet out of 5 strings that should be considered, only 2 appear in the resulting Twig file.

I know, some time passed since you posted your solution, but I’m still interested in listing this as a possible solution for i18n in the documentation. In order to do so, I think it should work with this simple test file:

translation-test.twig

{# __ default #}
{{ __( 'Test String 1', 'theme-test' ) }}

{# _x default #}
{{ _x('Test String with Context', 'Description of Context', 'theme-test') }}

{# _n default #}
{{ _n('%s star', '%s stars', 9, 'theme-test') }}

{# Inlined #}
<button role="button" aria-label="{{ __('Show/hide navigation', 'theme-test') }}"></button>

{# _nx default #}
{{ _nx('%s star', '%s stars', 9, 'context', 'theme-test') }}

{# String in comments, should be ignored #}
{# {{ __('Test String in comment', 'theme-test') }} #}

{# _n without text domain, should be ignored #}
{{ _n('%s star', '%s stars', 9) }}

{# __ without text domain, should be ignored #}
{{ __('Ignore this because it should be taken from the default text domain') }}

{# _nx without text domain, should be ignored #}
{{ _nx('%s star', '%s stars', 9, 'context' ) }}

package.json

{
  "name": "pot-test",
  "version": "1.0.0",
  "dependencies": {
    "del": "^3.0.0",
    "gulp": "^3.9.1",
    "gulp-if": "^2.0.2",
    "gulp-rename": "^1.2.2",
    "gulp-replace": "^0.6.1",
    "gulp-wp-pot": "^2.2.0",
    "run-sequence": "^2.2.1"
  },
  "scripts": {
    "gulp": "gulp"
  }
}

Maybe I’m doing something wrong and you can help me out? For me, only the _x default and _nx_default tests work.

@luism-s

This comment has been minimized.

Copy link
Contributor Author

@luism-s luism-s commented Apr 1, 2018

Hey @gchtr

I'll test those cases. It has been a while, but I remember to have improved the script, however I haven't updated this post.

The only case I never thought of is the gettext usage in comments.

@luism-s

This comment has been minimized.

Copy link
Contributor Author

@luism-s luism-s commented Apr 1, 2018

I'm back.

Found a problem with some of the regex's, preventing the script from parsing domains with hyphen (‐).

My initial post has been updated, as well as the gist: https://gist.github.com/luism-s/ebca42b8b8d70e81f8917f675a784060

@gchtr

@gchtr

This comment has been minimized.

Copy link
Member

@gchtr gchtr commented May 1, 2018

@luism-s Cool! And thanks for looking into it again. I guess that hyphens in text domains might be quite common. Glad these are now catched as well. I tested it again, and it works fine!

The only minor thing I stumbled over is that the version number in your package.json doesn’t follow semantic versioning and I got an «Invalid version: "1.2"» error. It should probably be "1.2.0".

The only case I never thought of is the gettext usage in comments.

Thinking about this, I think gettext in comments is an edge case that would fall under «having code in comments», which might be considered bad practice if you use version control. So we can probably ignore it.

I just created a pull request for adding a section about your method in the documentation. So I’m closing this issue.

@gchtr gchtr closed this May 1, 2018
@luism-s

This comment has been minimized.

Copy link
Contributor Author

@luism-s luism-s commented May 1, 2018

Happy to help ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.