Skip to content

Loading…

i18n support #85

Closed
nschonni opened this Issue · 52 comments

7 participants

@nschonni
Assemble member

Currently looking to generate out demos in multiple languages using some string replacements and templates.

  • Strings are currently in a CSV, but can be converted to JSON
  • File names should based on the base page name + 3 letter ISO code

Is this the right project to use or am I better off writing custom code? Also looked at https://github.com/aaaristo/grunt-html-builder, but I'm not sure how finished it is.

@jonschlinkert
Assemble member

@nschonni thanks for bringing this up. we'd like to add i18n support as well, since we need it for our own assemble projects as well.

We can accomplish in a number of different ways, like via simple handlebars helpers and/or underscore for the string replacement, something like:

Handlebars.registerHelper('i18n', function(key) {
  return new Handlebars.SafeString(I18n.t(key))
});

and:

{{i18n "path.to.translation"}}

And then extracting page names should be fairly easy to do (and it's also something we've wanted to add).

Is this close to what you're thinking, or are you looking for something different? let me know if I'm on the right track.

Also, for the data I though I would mention that you can also use YAML if you prefer that format. You would do something like this to add it to the Gruntfile:

 // Project configuration.
grunt.initConfig({
    i18n: grunt.file.readYAML('translations.yml'),

    assemble: {
        options: {
            i18n: '<%= i18n %>'
            ...

Soon the options.data object will support .yml by default, so you won't need to add any additional code to make it work.

@nschonni
Assemble member

I haven't done much with handlebars, but I think this would be a valuable example to add when you create the repo in #86. We're looking to add ~20+ additional language to http://wet-boew.github.com/wet-boew/demos/index-eng.html so I'm trying to find an approach where:

  • base partials are defined with replacement sections for the i18n replacements
  • partials for the various themes
  • separate layouts for ltr and rtl

I'm wondering how you where thinking of structuring the i18n strings if they were in a JSON file?

Please don't take this as a "give me the codez" request :smile:

@jonschlinkert
Assemble member

I think this would be a valuable example

I agree, that's a great idea. And for the record, I know a lot more about handlebars than I do about i18n, so we might be able to help each other out, but I don't want you to get discouraged if I'm not grasping your request right away. We need this too, so once I wrap my head around exactly what you need, we should be able to do something. How fast do you need to turnaround the i18n part of the project.


I just spent some time on the links you provided, but I have some questions/need clarification on a couple of things.

base partials are defined with replacement sections for the i18n replacements

Can you tell me more about the "replacement sections"? The i18n.json config file was helpful, but it might also help to see some HTML for pre- and post-translation just to make sure I understand what you're asking for. We have some other features in the works that might address this as well.

partials for the various themes

Do you have a "theme" schema or definition of some kind that you are comfortable sharing? If you prefer you can email it to me (email is on my profile). That might help, but in the meantime, based on the demos it sounds like what you're saying is theme A will have different styling, AND could have different sections, content or UI components than theme B? (have you seen this project, btw? https://github.com/assemble/assemble-styles. I noticed you use Bootstrap in your UI. I'm on the core team for Less.js as well, so let me know if you need anything on that side of the project as well).

Should the language/code be a root element with the translations as child properties?

I have a model similar to that in my mind, but I'm big on using standards and established conventions wherever possible. So let me research this one and get back to you after we've filled in some gaps here.

Out of curiosity, what translation library are you using? I know i18next actually has handlebars helpers that you can use out of the box: http://jamuhl.github.com/i18next/pages/doc_templates.html. Even if you can't use that solution their examples might help.

@jonschlinkert
Assemble member

actually, is all the code for what you're doing here: https://github.com/wet-boew/wet-boew? I went to the demo link you provided but I didn't notice this repo until now.

@nschonni
Assemble member

No specific time line, we can keep doing it manually till we have something better to replace it with.

Yes, that is the correct repo. The other approach someone is trying with the XSLT. The XSLT might give you an idea on how the themes currently interact (see src/base vs src/theme-*)

There maybe be pieces of Bootstrap-ish styling, but it is not bootstrapped based. We're currently using Sass for the CSS building.

base partials are defined with replacement sections for the i18n replacements

Besides the lack of proofread, I was trying to get at a structure like this:

I'm not sure if this could be done without creating pages for each or if this can be configured some other way. I'm trying to avoid creating 20 index-lang.hbs files in the pages directory. Alternately I guess these could be scaffolded in a separate task.

The sample i18n file I linked to is based on CSV we use to create JS client dictionaries during the build. I think a separate one would be required for the demos so I'm not too attached to format.

@jonschlinkert
Assemble member

Sorry for jumping to assumptions about Bootstrap, it wasn't actually the "appearance" as much as the class names, since I was focused on data :-) and your toolkit goes well beyond bootstrap anyway.

I was trying to get at a structure like this

Thanks for the additional explanation and examples of what goes into a page. What you're trying to accomplish is so aligned with what we're trying to do if you and I keep working together on this I'm pretty confident we'll get this done.

I'm not sure if this could be done without creating pages for each or if this can be configured some other way

No, that wouldn't be much of a solution for you. We'll try to do this in a much DRYer way - so I'll look for ways to leverage data instead of markup wherever possible.

As a first step, I'll try to take some time in the next couple of days to put a couple of examples together based on your code, just so we have something to discuss. Then you can more easily tell me what I'm getting correct or missing, and we'll have much more clarity on what needs to be implemented to get you the rest of the way.

@nschonni nschonni referenced this issue in wet-boew/wet-boew
Closed

Change to the demos #987

@klh

any progress here?

@jonschlinkert
Assemble member

@klh there are helpers and libs that can be used for this. more use cases and scenarios from different devs would help us understand how this would need to be implemented in assemble.

@klh

Usecase

I'm using assemble.io+hbs to render static content pages
& also using clientside dynamic hbs templates injected vis a js from usemin.

STATIC part:

i have a hack running with grunt-gettext extracting to a pot file that can the be translated via XX YY service and returned as a po/json file in various isocode localizations.

Those localizations are then used to generate available languages in build (static dirs):
en-GB
en-US
da-DK
...
the Assembled files are then copied to each directory as a simple copy
and text keys are then replaced from iso.json files.

DYNAMIC part:

hbs templates are rendered into js with i18n strings in them,
on useage the process is (meta code)

page load:
get currentLanguage (en-US)
invalidate language cache
get translation json file

js/app events that requires templates:
renderTemplate("some.template.identifier",{key:value,key:value});

renderTemplate gets a string via currentLanguage -> pops it through handlebars and returns a rendered template.

change language
invalidate language cache
fetch new iso-code

etc.

that help any?

@jonschlinkert
Assemble member

@klh I have to apologize. I don't know how I missed this AND marked it as read. I must have clicked on it on my cell phone and never actually read through it. (my wife and I are also expecting a baby this week, so it's been hectic :-)

In any case, your reply is very helpful, thank you!

i have a hack running with grunt-gettext...

Do you perhaps have a live example of what you're describing? I think most, if not all of this can be achieved with helpers, but it would help to see it running. then I can attempt to put a helper or helpers together.

Same with the dynamic stuff, it would be much easier to try to put something together based on a live example. Then, whatever helpers can't achieve we can look at adding to assemble. does this sound okay?

@scottbert

Hi,

This is also a use case for me. I'd like to be able to develop a set of partials and have grunt set up so on save it:
Scans the i18n/ directory for .yaml files
Outputs a page for each file for each template.

EG with a file layout as such:

i18n/
    - en.yaml
    - de.yaml
    - ru.yaml
partials/
    - nav.hbs
layouts/
    - main.hbs
pages/
    - product.hbs
    - detail.hbs

If I make a change to nav.hbs it creates the following:

dest/
    - en/
        - product.html
        - detail.html
    - de/
        - product.html
        - detail.html
    - ru/
        - product.html
        - detail.html

Using the contents of the three yaml files, creating a separate html file for each yaml file consisting of the layout with the nav and the product/detail content inside of it.

I'm fairly sure this is a similar/the same usecase. Is this simple to create using helpers?

@klh
@LaurentGoderre
Assemble member

I work with @nschonni and our use case is a bit different. We define the language of the page in iso and we would need to reference that in another expression.

---
page.language.iso: en
---
{{i18n.{{page.language.iso}}.key}}
@jonschlinkert
Assemble member

@klh thanks! we had our 2nd baby girl on Friday!

@LaurentGoderre, this is really just a curiosity (and potentially might impact how "convenient" the solution is) are there restrictions regarding data format? Can you use YAML front matter in files? I have some ideas for how to do this but there are so many different conventions, I'm thinking that we might want to implement some helpers along with some utilities for constructing custom helpers when the included ones don't quite meet your needs.

@nschonni
Assemble member

@jonschlinkert can you give an example of "YAML front matter in files"?

The benefit of using CSV is that we store all the translations in a Google Doc that the translators can edit without having to have knowledge of YAML syntax.


We're building our JS dictionaries using node-csv here https://github.com/wet-boew/wet-boew/blob/v4.0/tasks/i18n.js but I would prefer for the demos if we could have something we don't have to write and support all by ourselves


Congratulations on :shipit: :smile:

@LaurentGoderre
Assemble member

Does YAML or JSON matter? Can they be used interchangeably? That would help us stay consistent.

Congrats on the baby!

@jonschlinkert
Assemble member

can you give an example of "YAML front matter in files"?

I'm not sure, I was more curious what the requirements were so that I didn't spend time trying to come up with something that wouldn't work for you; which might include yaml front matter.

The additional info about how you're using CSV and google doc helps. (thanks for the congrats!)

Does YAML or JSON matter?

IMHO, no.

Can they be used interchangeably?

Yes, Assemble allows these formats to be used interchangeably. Beyond Assemble there are a number of libs that make it easy enough to convert from one format to the other. So portability should not be a problem if you decide that Assemble doesn't meet your needs. (I also created a grunt task that does this: https://github.com/assemble/grunt-convert. I have been considering adding CSV as well. I think this conversation demonstrates the potential use case I was looking for: doowb/grunt-convert#7).

However, despite the fact that the data formats are interchangeable, for the sanity of your users I strongly encourage a convention that specifies which format(s) to use for different use cases. For example, on my own projects I typically use YAML for "config" data (which tends to be terse and succinct) and JSON for lengthy or more complicated types of data. I don't know enough about your needs yet, but if I was to generalize it might make sense to use YAML for variables or specifying which language to use ("language config"), and JSON for the actual language definitions. It might be worth mentioning too that YAML front matter is really just plain old YAML that happens to be embedded in a document.

I would love to see Assemble provide support for this, so how about if I start by creating a sandbox repo for us to kick things off? Initially, it would probably be good to focus on reduced use case with the simplest implementation so that we can just get on the same page. and we can go from there...

@LaurentGoderre
Assemble member

Out of curiosity, can you access data from inside a helper?

@LaurentGoderre
Assemble member

I got it! It's quite simple. It assumes that there is a variable on the page called language and a file for each translations (en.json, fr.json or en.yml, fr.yml)

module.exports.register = function(Handlebars, options) {
    Handlebars.registerHelper('i18n', function(context, options) {
        return this[this.language][context];
    });
};
@LaurentGoderre
Assemble member

This allows to override the page language

module.exports.register = function(Handlebars, options) {
    Handlebars.registerHelper('i18n', function(context, options) {
        var language = (typeof options.hash.language === 'string') ? options.hash.language : this.language;
        return this[language][context];
    });
};
{{#i18n "key" language="fr"}}{{/i18n}}
@jonschlinkert
Assemble member

:+1: that's almost exactly what I was about to put together. :-) I was hoping it would be that straightforward.

Yes, helpers are very powerful, and Assemble makes it quite simple to use them with whatever context you require (handlebars has limitations that sometimes make things difficult, but very soon swig will be a viable alternative as a template engine with assemble). It would be great to include these helpers in the [handlebars-helpers library] if you don't mind doing a pull request there (we're removing coffeescript, so if you submit javascript only I'll take care of conversions to coffee until it's removed).

Also see these projects, as they demonstrate how to directly access YAML front matter with helpers, effectively bypassing Assemble's internal methods (this allows you to get very creative):

@LaurentGoderre
Assemble member

Here with proper error messages

module.exports.register = function(Handlebars, options) {
    Handlebars.registerHelper('i18n', function(context, options) {
        var language;

        if (typeof context !== 'string')
            throw "Key must be of type 'string'";

        language = (typeof options.hash.language === 'string') ? options.hash.language : this.language;

        if (typeof language === 'undefined') 
            throw "The 'language' parameter is not defined";

        if (typeof this[language] === 'undefined')
            throw "No strings found for language '" + language + "'";

        if (typeof this[language][context] === 'undefined')
            throw "No string for key '" + context + "' for language '" + language + "'";

        return this[language][context];
    });
};
@LaurentGoderre
Assemble member

Can i pull request this???? :smiley:

@jonschlinkert
Assemble member

absolutely!

@LaurentGoderre
Assemble member

I'm done with the actual helper but I'm struggling a bit with the test suite sop bear with me please (I never actually did this, despite wanting to do it for a long time)

@jonschlinkert
Assemble member

No worries, we're happy to help with anything. I'm not happy with how complicated and inconsistent the helper library is to contribute to, including the tests. I'd rather have contributions that we need to clean up than nothing at all.

@jonschlinkert jonschlinkert referenced this issue in assemble/buttons
Closed

add i18 examples #1

@jonschlinkert
Assemble member

btw, my wife and I are taking care of a newborn and 2-yr old, so there is no hurry. I might not be able to actually merge in the helper for a couple hours anyway.

@jonschlinkert
Assemble member

nvm, you made it easy on me! thanks, will merge now.

@LaurentGoderre
Assemble member

I don't know how to test the exceptions with this framework but we can add that later

@LaurentGoderre
Assemble member

I'm kind of new to npm, how do we leverage this update? Will a npm install cover it?

@nschonni
Assemble member

@LaurentGoderre a new version will have to be published, or you can replace the version number with the github tarball url in the package.json if you can't wait :smile:

@jonschlinkert
Assemble member

I'll be publishing the new version to NPM sometime today... I'll make a note when it's done.

As an aside, if it's okay with you guys I'd like to keep this issue open until we hear feedback from others as well.

@jonschlinkert
Assemble member

btw, @LaurentGoderre to publish new versions to npm is really simple. you just update the version number in the package.json then run npm publish. the time consuming part is making sure it won't break anything :-)

@nschonni
Assemble member

@jonschlinkert I'm fine with leaving this open. Maybe you need to add a "meta" tag to your issues :smile:

@jonschlinkert
Assemble member

add a "meta" tag

done! thanks for the suggestion

And 0.3.4 of handlebars-helpers was published to npm.

@jonschlinkert jonschlinkert referenced this issue in assemble/assemble.io
Open

i18n Helpers #42

@LaurentGoderre
Assemble member

I'm a bit confused by all of this, let me know when I can actually use this and how to get it from my existing instance

@jonschlinkert
Assemble member

@LaurentGoderre,

Assuming you're using Assemble, you should be able to either delete assemble from node_modules and reinstall it, or go into assemble's node_modules and remove the assemble-handlebars dir, then reinstall it.

If you're just using the helpers, you should be able to just do npm install handlebars-helpers to get the latest.

If any of these suggestions give you a problem, please let me know.

@scottbert

Hi,
Any advice for my use case? I have a single template, multiple yaml or JSON i18n files and want to output i18n'd templates from the single template to /template.html - is this something that could be done using helpers?

@LaurentGoderre
Assemble member

Perhaps the owner of this project know better than I do but it seems to go against the model that this project used which is one source file to one output file.

Here's one way to fake it, it's not ideal but it should work.

  1. Create a partial for your content
  2. Create a .hbs file for each language with the following content (substituting the en for the language of the page)
---
language: en
---
{{>mypartial}}
@doowb
Assemble member

@scottbert There is another way to do this that we haven't documented yet since there are still pieces of the feature that we're working on. If you take a look at the Gruntfile.js in the project, there are 3 examples of loading through a json list of defined pages. The pages themselves are stored in this config file and loaded into the Gruntfile at this line

I hope this helps.

@jonschlinkert
Assemble member

@LaurentGoderre

one source file to one output file

Not necessarily, in general my personal preference is to look for ways to use fewer templates, allowing data files to drive the variants.

In fact, this is the exact topic that inspired me to create this repo: https://github.com/assemble/buttons

@scottbert

Before you go the "pages" route that @doowb is recommending, I would suggest looking through the examples in that repo to see what is different from one example to the next, and why. You will likely (hopefully) find a convention that you prefer for your own project. The examples that might be most interesting to you are from button-030 to button-050

@jonschlinkert
Assemble member

btw, this is sincerely what I love about assemble :-) (and why I love contributing to this project). so many different ways to accomplish whatever goal you have

@jonschlinkert
Assemble member

Closing since we can accomplish this with helpers. Please re-open if you think we need to take another look.

@nschonni
Assemble member

Thanks to @jonschlinkert and @LaurentGoderre for picking up the ball on this :+1: :+1: :+1:

@martosoler

Sorry to write here but couldn't find some kind of discussion group for such topic and this issue describe exactly what I would like.
In my case, I would like to have exactly the same behavior than button-i18n project but without having to write an index file for each language. It would be something like iterate over the assemble task changing the language based on a list of files for each locale.

@LaurentGoderre
Assemble member

hey @martosoler I wrote the i18n feature and @nschonni and I are looking at exactly the same use case. We will keep you updated.

@jonschlinkert
Assemble member

perhaps one of us should create a temp repo with some tests and/or fixtures to show before/after/expected result we can collaborate on implementing something. I'd love to see a comprehensive i18 plugin.

@LaurentGoderre
Assemble member

Good idea!

@LaurentGoderre
Assemble member

Do you want to create it in your organization and add @nschonni, @martosoler and I as contributors?

@jonschlinkert
Assemble member

Sure, no prob

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.