Skip to content

Splitting language files up for front-end and admin #2051

Closed
GeertDD opened this Issue Dec 25, 2012 · 36 comments

5 participants

@GeertDD
GeertDD commented Dec 25, 2012

To get a rough idea of the memory usage for loading language files, I ran WooCommerce in Dutch (translation about 80% completed at this time) once normally and once with the load_plugin_textdomain() method disabled. In the last case memory usage went down by 3.6 MB.

Isn’t this quite a significant savings? Since most of the translation strings are for use in the admin area, what do you think about splitting the language files up, just as WordPress itself did in version 3.4? I realize this would mean quite a bit of work initially. Just bouncing off the idea, guys.

@mikejolley
WooThemes member

I'd be happy to do that, so long as the translators updated their po/mo's before launch :)

Merry Xmas

@arhipaiva

Memory usage very important in shared environment. I am in - will update the Finnish translation.

An idea:
Keep the majority of translations as is => admin no change.
Much less to update on the front end => only change front-end.

This would give the translators max time to adjust. Start with the [important] front-end and update the new terms in v. 2.0 admin section later with no time pressure.

This may act as a kick in the ass as well: Update the [front end] translations now if want to join the 2.0 bandwagon or be left behind.

Translations usually lag behind. The question is only for how much. Only some will be ready when 2.0 is out. The kick/necessity to update the front-end may actually do some good?

@a1ur3l
a1ur3l commented Dec 27, 2012

hello,
so to understand, that @mikejolley, you will update the files in order to have the translation split for front-end and admin?
when you will update it, me one, i will finish the RO translation.

thanks and Merry Xmas (with some late :D :)

@mikejolley
WooThemes member

To me I feel frontend should keep the 'woocommerce' text domain, with admin becoming admin-woocommerce. I think thats also how WP does it, but I'll check before coding.

@GeertDD
GeertDD commented Dec 27, 2012

Yeah, we should do it that way. I'd go for "woocommerce-admin" though, instead of "admin-woocommerce". Keeps all woocommerce files together when sorted alphabetically.

@mikejolley
WooThemes member

@GeertDD any idea how to implement dual text domains? Can you only load one or the other?

We need 'woocommerce' to always be loaded, but 'woocommerce-admin' in admin. I cannot get codestyling to pick them both up.

@GeertDD
GeertDD commented Dec 27, 2012

That's right, 'woocommerce' has to be loaded always. Texts that are used both on front-end and in admin (e.g. country names) use the 'woocommerce' textdomain. Texts only used in the admin use 'woocommerce-admin'.

@mikejolley
WooThemes member

Sure, I've got that much. But the actual load_plugin_textdomain and creation of po/mo/pot is what I'm having trouble with...

@GeertDD
GeertDD commented Dec 27, 2012

Are you using Poedit? You need to create a new catalog. Is that what you mean? (Alternatively, on the official WP plugin directory you can also generate a pot-file automatically in the admin section of the plugin.)

@mikejolley
WooThemes member

The load_plugin_textdomain() method of the woocommerce class. What to put there. Something like this?

if ( is_admin() ) {
            load_textdomain( 'woocommerce-admin', WP_LANG_DIR . '/woocommerce/woocommerce-admin-' . apply_filters( 'plugin_locale', get_locale(), 'woocommerce-admin' ) . '.mo' );
            load_plugin_textdomain( 'woocommerce-admin', false, dirname( plugin_basename( __FILE__ ) ) . '/languages/' . $variable_lang );
            load_plugin_textdomain( 'woocommerce-admin', false, dirname( plugin_basename( __FILE__ ) ) . '/languages' );
        }

Plus, generating them using codestyling localisation. It ideally needs to work with codestyling as its pretty popular.

@mikejolley
WooThemes member

(FYI codestyling only seems to pick up one text domain)

Also, the plugin header has text domain, and I think you can only define one there too. So all in all not sure how to handle this :)

@GeertDD
GeertDD commented Dec 28, 2012

That code you posted should be fine. I realize now you're talking about the Codestyling plugin which I don't have experience with. I opened a support topic...

@a1ur3l
a1ur3l commented Dec 28, 2012

i have seen in a theme or plugin (but i don`t remeber where) that they load with php files (2 files, one for admin and other for frontend) located in the lang directory.

one way is to define constant for every string in php files and add them to the lang file, but this will increase the memory usage, even if will don`t load the admin translation, and also this is a hard work.

from my point of view the best way is to split the mo files in 2 (separate files for admin and frontend) and load them according with the backend/frontend status, but i didn`t found a way (with poedit) to scan only the files that i want. i will try with others program today, and i will kept you up.

i see that if you defined different text domains, and parse the files (all) with poedit it parse all the text domains, but i don`t know what is happen if we load only a textdomain ( if it kept the entire mo file in the memory or it just parse the file, extract the text domain and kept only it).

thanks

@GeertDD
GeertDD commented Dec 28, 2012

This all appears to be trickier than I thought it to be. Some things I just found out:

Poedit scans the project for all strings used by translation functions like __(), but it does not and cannot(?) take the actual textdomain into account. I have not found a way to generate separate "woocommerce" and "woocommerce-admin" po-files.

WordPress uses multiple translation files: one general, one for the admin, one for multisite and one for continents and cities. However, except for the continents and cities, all those files use the same textdomain called "default". Have a look at the load_default_textdomain() function. This approach does not require you to update any source code with a new textdomain. The trick is to split up one po-file into multiple ones, but how?

For the continents and cities WP did setup a separate textdomain called "continents-cities". They've got a separate file in place for this.

@a1ur3l
a1ur3l commented Dec 28, 2012

yes it is very trickier, but can give us some memory gain :) and we should search around.

for example i have made some test with my theme and it seems it is working.
i have create 2 separate text domains and load them accordin with the view and it seems it is working.
so create 2 separate text domain: for admin and frontend, and i load them as follow:

global $pagenow;

    if (!is_admin() && $pagenow == 'index.php') {
        load_theme_textdomain( 'frontend' );
        load_theme_textdomain( 'frontend', get_template_directory() . '/lang' );
        if ( function_exists( 'load_child_theme_textdomain' ) )
            load_child_theme_textdomain( 'frontend' );
    } else {
        load_theme_textdomain( 'frontend' );
        load_theme_textdomain( 'frontend', get_template_directory() . '/lang' );
        if ( function_exists( 'load_child_theme_textdomain' ) )
            load_child_theme_textdomain( 'frontend' );

        load_theme_textdomain( 'admin' );
        load_theme_textdomain( 'admin', get_template_directory() . '/lang' );
        if ( function_exists( 'load_child_theme_textdomain' ) )
            load_child_theme_textdomain( 'admin' );
    } 

the global $pagenow it apear to be index.php if i`m on frontend... and yes it is a gain of memory (around 1 mega in my case), with all the strings in the same mo files.

so this it is a way. if we found a way to split the strings in different files (admin, frontend and countries) we can modify the load according with our need, and we must see if we gain more from this variant

@GeertDD
GeertDD commented Dec 28, 2012

@a1ur3l Still the question remains, how do you generate your "frontend.pot" and "admin.pot" files? Note that I would really like to keep using a single textdomain now. That would make things easier for the Codestyling plugin too. The key to saving memory is splitting up the po/mo-files regardless of the number of textdomains.

Here's the ticket for the WP language file split-up. Apparently they use a file called "makepot.php" to generate their separate language files. It looks like they base their language file split-up on a directory basis. The admin pot-file for example only scans the "wp-admin/" directory for translatable strings.

@a1ur3l
a1ur3l commented Dec 28, 2012

but that file need to be rewrite and addapted according to our use. i see also that they print all the things without a text domain for example: __('text string')

Later edit:

http://codex.wordpress.org/I18n_for_WordPress_Developers#Generating_a_POT_file

@arhipaiva

Just printing some texts I came across:

When using 'Codestyling Localization', I see this message:
"WooCommerce
Compatibility
Loading Issue: Author is using load_textdomain instead of load_plugin_textdomain function. This may break behavior of WordPress, because some filters and actions won't be executed anymore. Please contact the Author about that."

  • Maybe this could be fixed as well or maybe this is a false alarm?

From 'Codestyling Localization' Help screen:
"The Textdomain default always stands for the WordPress main language file, this could be either intentionally or accidentally!"

  • Maybe WP can drop textdomain __('text string') because being 'default'?
  • Is this an error: There are 17 entries with textdomain "default" in the woocommerce localization file (rescanned 2.0.0-beta-1). Default textdomain is for WP...

On 'Translate Language File' (Codestyling...) screen there is a dropdown "Textdomain":
From Help: "The extended feature for textdomain separation shows at dropdown box Textdomain the pre-selected primary textdomain. All other additional contained textdomains occur at the source but will not be used, if not explicitely supported by this component!"

  • This makes it possible to translate separately different text domains, but not generate separate .mo or .po files (what we need).
@GeertDD
GeertDD commented Dec 28, 2012

Loading Issue: Author is using load_textdomain instead of load_plugin_textdomain function. This may break behavior of WordPress, because some filters and actions won't be executed anymore. Please contact the Author about that."

That's a false alarm indeed. WooCommerce uses both load_textdomain() and load_plugin_textdomain(). Here's some more info about that.

Maybe WP can drop textdomain __('text string') because being 'default'?

Right. If the textdomain is omitted, "default" is assumed.


As far as I see, we've only got one issue it all comes down to: find a way to automatically generate multiple pot-files (front-end and admin) for the "woocommerce" textdomain. The answer is a custom makepot.php.

@a1ur3l
a1ur3l commented Dec 28, 2012

one way (not the corect one):

assuming that in admin folder are all the things used only for admin

step 0: back-up the original files (po and mo of the lang)

step 1: in poedit go to Catalog properties -> Sources paths and change the paths to have only admin ( from . (point - entire dir) to admin) click update and save as woocommerce-admin

step 2: use the back-up, with no change, cut the admin dir from the plugin and select update in poedit. and save it as woocommerce-frontend, and put the admin dir back

so in this situation we obtain 2 files that should contain the strings for the admin and fronent.
still are some strings that are in both files. for future update is neccessary just to take out the admin folder when update the catalog for frontend.

@mikejolley
WooThemes member

@GeertDD you created a makepot before? I assume we'd need to generate the POT files with every release, as codestyling may not be able to re-scan correctly?

@GeertDD
GeertDD commented Dec 29, 2012

@mikejolley Generating a front-end and admin pot-file with every release is a good idea. Translators can just use those files to base their translations on and don't need to scan the project manually. I've not created or worked with a makepot.php file before, though. The one WordPress uses might work for us too if we change some paths, under the assumption that all translated texts in the "woocommerce/admin/" directory are not used on the front-end.

@GeertDD
GeertDD commented Dec 29, 2012

Extending WordPress’s makepot.php file did the trick. The code will need some cleanups but I was able to generate two pot-files for WooCommerce, one for the admin (all files in the admin/ directory) and one for the front-end (all other files).

These are the two generated files: https://gist.github.com/c926d72f4f15802fcda3

@mikejolley
WooThemes member

Cool. There would be a few 'admin' strings in things like gateways, but we cannot really account for those so this is better than nothing.

Ill need to make the loader (load the correct POT). Are you committing the makepot? It could go in the i18n folder :)

@GeertDD
GeertDD commented Dec 29, 2012

Yah, well, these are my changes to makepot.php, but it requires some other files too from the WP i18n tools repository.

@GeertDD
GeertDD commented Dec 29, 2012

Working on it.

@mikejolley
WooThemes member

This is now down to translators to make use of - I've added the code for it 4c7ea6a

Old translations will work, they'll just be larger.

@mikejolley mikejolley closed this Dec 30, 2012
@mikejolley
WooThemes member

Actually, needs more work because of our structure.

@mikejolley mikejolley added a commit that referenced this issue Dec 30, 2012
@mikejolley mikejolley Wrong paths #2051 1d67070
@mikejolley
WooThemes member

@GeertDD does that look correct to you? The woocommerce-admin ones won't be loaded otherwise, because of the text domain.

@GeertDD
GeertDD commented Dec 30, 2012

@mikejolley Right, plugin_text_domain() assumes textdomain and filename match. Looks fine to me the way you did it. Just one thing, are the file_exists() checks necessary? The load_textdomain() function does such a check already:

if ( !is_readable( $mofile ) ) return false;

PHP manual:

is_readable() returns TRUE if the file or directory specified by filename exists and is readable, FALSE otherwise.

@mikejolley
WooThemes member

Maybe just the first one? To see if the user over-rode it via WP_LANGS. Not sure how they stack.

@mikejolley mikejolley added a commit that referenced this issue Dec 30, 2012
@mikejolley mikejolley 2nd exists #2051 d1372bd
@GeertDD
GeertDD commented Dec 30, 2012

Actually I think you'd just need this:

if ( is_admin() ) {
 // File with custom user translations, outside of plugin directory:
 load_textdomain( 'woocommerce', WP_LANG_DIR . "/woocommerce/woocommerce-admin-$locale.mo" );
 // Default translations that ship with the plugin:
 load_textdomain( 'woocommerce', $this->plugin_path() . "/i18n/languages/woocommerce-admin-$locale.mo" );
}

You can't ignore the second if the first is exists. The custom translation file could very well only contain a select few translations. Always load the default translations too, as fallback.

@a1ur3l
a1ur3l commented Dec 30, 2012

so in the final: we need to manually update (regenerate them) the translation file? (reffer to rescan and recreate it)?
because i see on woocommerce-admin.pot and woocommerce.pot?

can please exaplin me short? in order to help with ro_RO translation?

thanks

@GeertDD
GeertDD commented Dec 30, 2012

@a1ur3l Don't worry, the translation process actually should have become easier. 😉

The POT files are template files for your language's PO file. The "T" in POT stands for "template". POT files contain no translations, they only contain the texts that need to be translated. You no longer need to scan WooCommerce yourself. You can base all your translations on the POT files. The POT files will be updated for every new WooCommerce release.

How does this work practically? Here's the Poedit process. Initially you go to File > New Catalog from POT File.... Then you can start translating and save your language's PO and MO file. When the POT file is updated, you go to Catalogue > Update from POT File... to pull in new texts and delete old ones. That's it.

Update: I might add that in order to save your existing translations it's best to first pick your current PO file when you do File > New Catalog from POT file.... After that update from the real POT file.

@eugine-Kim

Hi, I translated the language file into Korean, version 1.6.5.2 last year. I couldn't summit it here since I didn't know how to. Now I want to upgrade it to current version. Do I have to copy and paste all the strings? Please let me know better way and how to summit it here.

@arhipaiva

Hi, a comment for issue #2393 might be of help.
Link: #2393

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.