Multilingual solution based on data language files #45

fdeneux · 2016-03-12T19:18:00Z

Hello, first of all great work on panini!

I'm looking for a specific multilingual solution, where you have a single multilingual structure (page) that is compiled to language specific html files for every language file in data.

Example:

root
│
│───src/
│   │
│   │───data/
│   │   |───en.yml
│   │   |───es.yml
│   │   └───fr.yml
│   │
│   │───layouts/
│   │   └───default.hbs
│   │
│   └───pages/
│       └───index.hbs
│
└───dist/
    │───index-en.html
    │───index-es.html
    └───index-fr.html

This can be achieved with the combination of gulp-swig and gulp-data, but I'd like to know if such a solution is possible with panini as I really like the generator.

An even better solution is to compile to a folder for every language:

root
│
│───src/
│   │
│   │───data/
│   │   |───en.yml
│   │   |───es.yml
│   │   └───fr.yml
│   │
│   │───layouts/
│   │   └───default.hbs
│   │
│   └───pages/
│       └───index.hbs
│
└───dist/
    │
    │───en/
    │   └───index.html
    │
    │───es/
    │   └───index.html
    │
    └───fr/
        └───index.html

Another interesting solution is to have a structured languages data subfolder, that would allow you to split your data files.

root
│
│───src/
│   │
│   │───data/
│   │   │
│   │   └───languages/
│   │       │
│   │       │───en/
│   │       │   │───info.yml
│   │       │   └───item-list.yml
│   │       │
│   │       │───es/
│   │       │   │───info.yml
│   │       │   └───item-list.yml
│   │       │
│   │       └───fr/
│   │           │───info.yml
│   │           └───item-list.yml
│   │
│   │───layouts/
│   │   └───default.hbs
│   │
│   └───pages/
│       └───index.hbs
│
└───dist/
    │
    │───en/
    │   └───index.html
    │
    │───es/
    │   └───index.html
    │
    └───fr/
        └───index.html

The text was updated successfully, but these errors were encountered:

fredericpfisterer · 2016-04-14T07:31:01Z

I would love to read an answer about this. I have a site to make in two languages and I don't know how to do it with panini.

giofilo · 2016-04-15T13:41:47Z

Me too. My company sends newsletter (I use foundation for emails) to 8 different countries. Would be awesome to have a solution!

XmlmXmlmX · 2016-09-02T07:05:35Z

That would be awesome! Actually we do it like that (out of the box):

root
│
│───src/
│   │
│   │───layouts/
│   │   └───default.html
│   │
│   │───pages/
│   │  │───Foo.Bar.en-GB.html
│   │  │───Foo.Bar.de-DE.html
│   │  └───Foo.Bar.html
│   │
│   └───partials/
│       │───Footer.html
│       │───Foo.Bar.html
│       └───Header.html
│
└───dist/
    │───Foo.Bar.en-GB.html
    │───Foo.Bar.de-DE.html
    └───Foo.Bar.html

Each page contains localization inside the front matter block.

Downside: For common partials like footer or header, all translations are duplicated.

After compiling we have a batch script wich generates npm packages for each template namespace to use it in our services.

XmlmXmlmX · 2016-11-03T07:13:59Z

Nothing new here?

fdeneux · 2016-11-03T08:28:53Z

@XmlmXmlmX I ended up mirroring the project on GitLab and tweak it to my needs on a custom branch.

XmlmXmlmX · 2016-11-15T13:02:31Z

@fdeneux can you share your work or make a PR here?

shaneog · 2017-02-22T17:25:58Z

@XmlmXmlmX Looks like @fdeneux's work is here.

I'm in need of a similar solution, but I haven't tried this fork yet.

XmlmXmlmX · 2017-04-12T08:38:10Z

Hey, I have published my solution in a gist here. It's a bit more complex, because the requirements are very high (currently 12 pages * 31 languages = 372 different emails).

Hopefully this enhancement will be integrated sometime.

@gakimball Is it worth to make a pull-request?

gakimball · 2017-04-12T16:37:53Z

@XmlmXmlmX This looks great! Let's talk a little about how we can simplify it. In general I want to give people one way to do things. Here are a few ideas and questions.

Helper

For the #i18n helper, we could use an inline helper instead of a block helper. Makes it a little faster to write.

{{#i18n 'path.to.key'}}

Locale Strings

For file structure, I think we should enforce one method of organizing and one style of writing file names. A basic structure might look like this (taking inspiration from other i18n libraries I've used):

- src/
  - locales/
    - en.json
    - pl.json

That being said, some folks will mostly likely have too many translation strings to reasonably fit in one file, so an alternative way would be:

- src/
  - locales/
    - en/
      - home.json
      - about.json

Now, that is perhaps giving people two ways to do things, but the hierarchy is the same.

If a locale is a single file, accessing it involves writing the object path to the key. If a locale is multiple files, it's still an object path, but the first key is the name of the file.

Notably, this doesn't enforce any particular style of organizing translation strings. If you want to go by page, you can do that, or if you want to keep things in one place, you can do that as well.

This does differ from your approach in that it doesn't automatically load a set of translation strings based on the page. With this approach, you'd have to specify the page each time, like so:

<h1>{{#i18n 'home.title'}}</h1>
<p>{{#i18n 'home.subtitle'}}</p>

This would make mixing global and page-specific strings easier, however. It also matches the way data files are loaded: one file = one object with name of file.

Thoughts on this approach?

gakimball · 2017-04-21T02:28:32Z

Hey! I've implemented the basics of this in 8b2d617. Here's what the input/output looks like:

gakimball · 2017-04-22T17:30:59Z

Closing this as the basics of i18n are in place. If people have more ideas we can definitely talk about them :)

XAMelleOH · 2017-07-05T12:21:55Z

@gakimball Trying to use your solution, by changing "panini": "^1.4.0" to "panini": "git+https://github.com/zurb/panini.git#v2.0-alt" but getting

[15:04:36] Requiring external module babel-register
[15:04:37] Using gulpfile ~/Work/Email.Templates/gulpfile.babel.js
[15:04:37] Starting 'default'...
[15:04:37] Starting 'build'...
[15:04:38] Starting 'clean'...
[15:04:38] Finished 'clean' after 8.18 ms
[15:04:38] Starting 'pages'...
[ DEBUG ] Using DumbTemplates brand.
[15:04:38] 'pages' errored after 5.49 ms
[15:04:38] TypeError: Class constructor  cannot be invoked without 'new'
    at pages (/Users/andriy/Work/Email.Templates/gulpfile.babel.js:86:11)
    at bound (domain.js:280:14)
    at runBound (domain.js:293:12)
    at asyncRunner (/Users/andriy/Work/Email.Templates/node_modules/async-done/index.js:36:18)
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)
[15:04:38] 'build' errored after 1.88 s
[15:04:38] 'default' errored after 1.88 s
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! foundation-emails-template@1.0.0 start: `gulp`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the foundation-emails-template@1.0.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/andriy/.npm/_logs/2017-07-05T12_04_39_003Z-debug.log

gakimball · 2017-07-05T17:09:56Z

@XAMelleOH The API has changed. Try this:

const panini = require('panini/gulp');

gulp.task('pages', () => {
  return panini('src', {/* options */})
    // Put the usual plugins like Inky here
    .pipe(gulp.dest('dist'));
});

XAMelleOH · 2017-07-06T07:34:26Z

@gakimball Thank you, it works as expected.

The only question, when we used gulp.src we could exclude some files like this:

gulp.src(['src/pages/**/*.html', '!src/pages/archive/**/*.html'])

How can we achieve the same when panini handles that by itself?

Also, how could I reset Panini's cache of layouts and partials? (previously it was just panini.refresh())

gakimball · 2017-07-06T16:40:25Z

How can we achieve the same when panini handles that by itself?

The folder structure is a bit more stringent now, which is an intentional simplification of the library. What's your use case for wanting to ignore specific pages?

(As a temporary workaround, you can use gulp-filter to filter out the processed files.)

Also, how could I reset Panini's cache of layouts and partials? (previously it was just panini.refresh())

This is another part of the API that isn't finalized yet. When used standalone or in a CLI, Panini will do this refreshing automatically, but I haven't yet worked out how it should be handled within Gulp.

To get around it for now, you can create a new Panini instance every time your task runs.

const panini = require('panini/gulp').create;

gulp.task('pages', () => {
  panini()('src', {})
    .pipe(gulp.dest('dist'));
});

Thanks for bearing with me :)

XmlmXmlmX · 2017-07-11T14:04:47Z

Hi @gakimball, first thank you for the integration of this function. Now I finally found time to look at it more closely. For me the i18n-helper does not work. Localized folders are created, but with errors.

In one of my pages I use {{#i18n 'paragraph1'}}.

./locales/de.json:

"paragraph1": "Hallo"

./locales/en.json:

"paragraph1": "Hello"

The result is:

Panini Error

Parse error on line 74:
...
  </body>
</html>
---------------------^
Expecting 'OPEN_INVERSE_CHAIN', 'INVERSE', 'OPEN_ENDBLOCK', got 'EOF'

Stack Trace:

Object.parseError (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\parser.js:268:19)
Object.parse (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\parser.js:337:30)
HandlebarsEnvironment.parse (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\base.js:46:43)
compileInput (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\compiler.js:515:19)
ret (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\compiler.js:524:18)
HandlebarsEngine.render (d:\app\node_modules\panini\engines\handlebars.js:93:14)
pages.map.page (d:\app\node_modules\panini\lib\render.js:157:40)
Array.map (native)
Panini.build (d:\app\node_modules\panini\lib\render.js:151:23)
DestroyableTransform._flush (d:\app\node_modules\panini\lib\render.js:46:11)

Is {{#i18n 'paragraph1'}} the correct syntax?

XAMelleOH · 2017-07-11T14:16:35Z

@XmlmXmlmX Here is a new docs (wip): https://gist.github.com/gakimball/e1016f4003b683277d8fd1b3323ad9d9

So, the correct syntax {{ translate 'paragraph1' }}

savonije · 2017-07-11T14:47:45Z

@XAMelleOH What should the options part look like to make it work with Foundation Emails?

const panini = require('panini/gulp');

gulp.task('pages', () => {
  return panini('src', {/* options */})
    // Put the usual plugins like Inky here
    .pipe(gulp.dest('dist'));
});

gakimball · 2017-07-11T16:41:44Z

@mescie Here are all the current options. Most of them are just for the names of folders, so if you use the defaults, you probably won't need to change anything.

XmlmXmlmX · 2017-07-12T06:10:34Z

Ok, this works pretty good now. Next level would be to be able to use helpers in combination, just like this:

<ol class="paragraph-list text-justify">
    {{#each translate 'disclaimer.paragraphs'}}
        <li>
            <h2 class="h5-style paragraph-header" id="{{ this.id }}">{{ this.title }}</h2>
            <ol>
                {{#each this.paragraphs}}
                <li id="{{ this.id }}">{{ this.text }}</li>
                {{/each}}
            </ol>
        </li>
    {{/each}}
</ol>

~~One downside is, that partials aren't supported yet :/~~ Sorry, it does work :)

savonije · 2017-07-12T07:54:27Z

I've added the default options, but it wont run for me yet.

// Compile layouts, pages, and partials into flat HTML files
// Then parse using Inky templates
function pages() {
    const panini = require('panini/gulp');

    gulp.task('pages', () => {
        return panini('src', {
            root: 'src/pages',
            layouts: 'src/layouts',
            partials: 'src/partials',
            helpers: 'src/helpers'
        })
            .pipe(inky())
            .pipe(gulp.dest('dist'));
    });
}

I get the following error.

[09:48:25] The following tasks did not complete: default, build, pages
[09:48:25] Did you forget to signal async completion?

Im kind of new to javascript, any tips on how to fix this?

gakimball · 2017-07-12T16:57:22Z

Next level would be to be able to use helpers in combination, just like this:

@XmlmXmlmX You should be able to do this with Handlebars subexpressions. By wrapping a Handlebars statement in parentheses, you get a self-contained value that you can pass to another statement. So if you wanted to iterate through an array pulled from a locale file, you should be able to do this:

{{#each (translate 'disclaimer.paragraphs') }}
{{/each}}

Panini Error

Missing helper: "ifpage"

Stack Trace:

Object.<anonymous> (d:\app\node_modules\handlebars\dist\cjs\handlebars\helpers\helper-missing.js:19:13)
Object.eval (eval at createFunctionContext (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\javascript-compiler.js:254:23), <anonymous>:5:77)
main (d:\app\node_modules\handlebars\dist\cjs\handlebars\runtime.js:175:32)
ret (d:\app\node_modules\handlebars\dist\cjs\handlebars\runtime.js:178:12)
ret (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\compiler.js:526:21)
Object.invokePartial (d:\app\node_modules\handlebars\dist\cjs\handlebars\runtime.js:283:12)
Object.invokePartialWrapper [as invokePartial] (d:\app\node_modules\handlebars\dist\cjs\handlebars\runtime.js:68:39)
Object.eval (eval at createFunctionContext (d:\app\node_modules\handlebars\dist\cjs\handlebars\compiler\javascript-compiler.js:254:23), <anonymous>:17:28)
main (d:\app\node_modules\handlebars\dist\cjs\handlebars\runtime.js:175:32)
ret (d:\app\node_modules\handlebars\dist\cjs\handlebars\runtime.js:178:12)

gakimball · 2017-07-13T17:16:34Z

@XmlmXmlmX No worries, thanks for bearing with me! You're one of the first people to use this besides me, so this is very useful.

The way helpers work is changing around in Panini 2.0. Instead of separate #ifPage and #unlessPage block helpers, there's just one inline helper called currentPage. You can use it with the built-in #if and #unless functions in Handlebars.

{{#if (currentPage 'index')}}
{{else}}
{{/if}}

ivanbanov · 2017-12-01T05:57:08Z

I was having problems with the reload on Gulp too. The solution to use the panini.create() worked smoothly but I needed to change the plugin gulp-changed to gulp-changed-in-place to get it working correctly again ;)

shin10kudev · 2018-08-28T14:47:50Z

I have successfully added localization using the above approach. I'm wondering how it's possible to pass the localized strings for meta data as well, which seems to involve passing data from the page or partial to the layout where the meta data exists.

I tried using the {{ translate 'string' }} method

---
layout: region
meta-title: {{ translate 'regions.tohoku.meta.title' }}
---

but I just get [object object] for the returned string. Anybody have experience with this?

shin10kudev · 2018-08-29T01:49:13Z

@gakimball after more digging, looks like my question is answered in here #66 (not possible to use helpers in front matter).

Do you have any other ideas? I'm trying to figure out how to get translations for the meta tags like title/description, etc. which exist in the layout (though it will vary by page)

perhaps if there were a way to make the translation key dynamic, it could be used to create a new key based on a page. maybe using a subexpression to make the key dynamically generated?

shin10kudev · 2018-08-29T04:10:30Z

@gakimball

I was able to resolve by using subexpressions:

simple execution:

<title>{{ translate (for_key 'regions.view.meta.title' ) }}</title>

for_key.js

module.exports = function(key, options) {
  var string = key.replace("view", options.data.root.page);
  return string;
}

basically what it does is dynamically update the key passed to the translate helper based on the page/view context.

Perhaps there are ways to improve this, but it seems to be working for now :)

gakimball · 2018-08-29T17:56:01Z

@eliotc1986 I think that's a solid solution. In Panini, any data/translations you define are global, so there's not really a concept of page-specific data. In some cases this is maybe unwieldy, but Panini is generally designed for small- to medium-sized sites.

You could also use Handlebars inline partials, which would allow you to change the HTML of a layout from within a page:

<!-- Layout -->
<title>{{> page_title}}</title>

<!-- Page -->
{{#* inline 'page_title'}}
  {{ translate 'regions.tohoku.meta.title' }}
{{/inline}}

However, it's a little more verbose.

My only suggestion for your approach would be to change the view part of the string to something that stands out more, like [view], so it's clearly distinct from the other parts.

shin10kudev · 2018-08-30T02:00:19Z

@gakimball Thank you for the reply and for the advice! I really appreciate it. Using a [view] to better distinguish the text is a good idea. I will update my earlier post after testing.
Let me also take the chance to thank you for all your work on panini. It's super versatile and really easy to use. I'm using v.2.0alpha in my project and it has major improvements over previous versions.

you mean to do this right:

{{ translate ( for_key 'meta.[view].title' ) }}

*for_key.js*

module.exports = function(key, options) {
  var string = key.replace("[view]", options.data.root.page);
  return string;
}

gakimball · 2018-08-30T22:43:55Z

you mean to do this right:

Yep, that's the idea.

Let me also take the chance to thank you for all your work on panini. It's super versatile and really easy to use. I'm using v.2.0alpha in my project and it has major improvements over previous versions.

Thank you so much! :) I haven't had time to work on it in a minute, but I'm glad people are still finding it useful. If you have any feedback or bugs, feel free to open a new issue.

Several people asked for multilanguage/i18n solutions for panini (foundation#180, foundation#45). Adding this to the readme can help others.

gakimball added the enhancement label Mar 14, 2016

XmlmXmlmX mentioned this issue Nov 15, 2016

Scan the output of helpers for other helpers, passing in the same page context #69

Closed

gakimball added a commit that referenced this issue Apr 21, 2017

Add localization support #45

8b2d617

gakimball closed this as completed Apr 22, 2017

XAMelleOH mentioned this issue Oct 12, 2017

No way to refresh panini cache [v2.0] #143

Open

ourstudio-erikblomqvist mentioned this issue Mar 25, 2019

Multilingual support? #180

Open

elcaptain added a commit to elcaptain/panini that referenced this issue Dec 18, 2019

Add 'useful plugins' section

0ecf830

Several people asked for multilanguage/i18n solutions for panini (foundation#180, foundation#45). Adding this to the readme can help others.

elcaptain mentioned this issue Dec 18, 2019

Add 'useful plugins' section #202

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multilingual solution based on data language files #45

Multilingual solution based on data language files #45

fdeneux commented Mar 12, 2016

fredericpfisterer commented Apr 14, 2016

giofilo commented Apr 15, 2016 •

edited

Loading

XmlmXmlmX commented Sep 2, 2016 •

edited

Loading

XmlmXmlmX commented Nov 3, 2016

fdeneux commented Nov 3, 2016

XmlmXmlmX commented Nov 15, 2016 •

edited

Loading

shaneog commented Feb 22, 2017

XmlmXmlmX commented Apr 12, 2017 •

edited

Loading

gakimball commented Apr 12, 2017

gakimball commented Apr 21, 2017

gakimball commented Apr 22, 2017

XAMelleOH commented Jul 5, 2017 •

edited

Loading

gakimball commented Jul 5, 2017 •

edited

Loading

XAMelleOH commented Jul 6, 2017 •

edited

Loading

gakimball commented Jul 6, 2017

XmlmXmlmX commented Jul 11, 2017

XAMelleOH commented Jul 11, 2017

savonije commented Jul 11, 2017 •

edited

Loading

gakimball commented Jul 11, 2017

XmlmXmlmX commented Jul 12, 2017 •

edited

Loading

savonije commented Jul 12, 2017

gakimball commented Jul 12, 2017

XmlmXmlmX commented Jul 13, 2017 •

edited

Loading

XmlmXmlmX commented Jul 13, 2017

gakimball commented Jul 13, 2017

ivanbanov commented Dec 1, 2017 •

edited

Loading

shin10kudev commented Aug 28, 2018 •

edited

Loading

shin10kudev commented Aug 29, 2018 •

edited

Loading

shin10kudev commented Aug 29, 2018

gakimball commented Aug 29, 2018

shin10kudev commented Aug 30, 2018 •

edited

Loading

gakimball commented Aug 30, 2018

Multilingual solution based on data language files #45

Multilingual solution based on data language files #45

Comments

fdeneux commented Mar 12, 2016

fredericpfisterer commented Apr 14, 2016

giofilo commented Apr 15, 2016 • edited Loading

XmlmXmlmX commented Sep 2, 2016 • edited Loading

XmlmXmlmX commented Nov 3, 2016

fdeneux commented Nov 3, 2016

XmlmXmlmX commented Nov 15, 2016 • edited Loading

shaneog commented Feb 22, 2017

XmlmXmlmX commented Apr 12, 2017 • edited Loading

gakimball commented Apr 12, 2017

Helper

Locale Strings

gakimball commented Apr 21, 2017

gakimball commented Apr 22, 2017

XAMelleOH commented Jul 5, 2017 • edited Loading

gakimball commented Jul 5, 2017 • edited Loading

XAMelleOH commented Jul 6, 2017 • edited Loading

gakimball commented Jul 6, 2017

XmlmXmlmX commented Jul 11, 2017

Panini Error

Stack Trace:

XAMelleOH commented Jul 11, 2017

savonije commented Jul 11, 2017 • edited Loading

gakimball commented Jul 11, 2017

XmlmXmlmX commented Jul 12, 2017 • edited Loading

savonije commented Jul 12, 2017

gakimball commented Jul 12, 2017

XmlmXmlmX commented Jul 13, 2017 • edited Loading

XmlmXmlmX commented Jul 13, 2017

Panini Error

Stack Trace:

gakimball commented Jul 13, 2017

ivanbanov commented Dec 1, 2017 • edited Loading

shin10kudev commented Aug 28, 2018 • edited Loading

shin10kudev commented Aug 29, 2018 • edited Loading

shin10kudev commented Aug 29, 2018

gakimball commented Aug 29, 2018

shin10kudev commented Aug 30, 2018 • edited Loading

gakimball commented Aug 30, 2018

giofilo commented Apr 15, 2016 •

edited

Loading

XmlmXmlmX commented Sep 2, 2016 •

edited

Loading

XmlmXmlmX commented Nov 15, 2016 •

edited

Loading

XmlmXmlmX commented Apr 12, 2017 •

edited

Loading

XAMelleOH commented Jul 5, 2017 •

edited

Loading

gakimball commented Jul 5, 2017 •

edited

Loading

XAMelleOH commented Jul 6, 2017 •

edited

Loading

savonije commented Jul 11, 2017 •

edited

Loading

XmlmXmlmX commented Jul 12, 2017 •

edited

Loading

XmlmXmlmX commented Jul 13, 2017 •

edited

Loading

ivanbanov commented Dec 1, 2017 •

edited

Loading

shin10kudev commented Aug 28, 2018 •

edited

Loading

shin10kudev commented Aug 29, 2018 •

edited

Loading

shin10kudev commented Aug 30, 2018 •

edited

Loading