Add custom translation function [doc todo: use as sample extension] #51

Open
wants to merge 1 commit into
from

Projects

None yet

4 participants

@saleyn
saleyn commented Nov 12, 2012

Translation functionality can be customized by replacing the hard-coded
erlydtl_i18n:translate/2 with the function controlled by the option:
{i18n_mf, {Module, Function}}.

Update

Not to be merged as is.

@evanmiller
Collaborator

This is great. Can you explain the purpose of included_headers? Also I'd like the config option to have the same convention as the "reader" option since they are both module/function tuples. So either call them "reader_mf" and "translater_mf", or "reader" and "translater", or something.

@saleyn
saleyn commented Jan 18, 2013

I'll rename i18n_mf to translater.

The purpose of included_headers is to let the erlydtl_compiler know which files may contain macro definitions for the emitted code. Why is this needed? The patched README.markdown explains it briefly, but I'll provide more details here. One of the problems I found with gettext application that handled internationalization was that it would always do a run-time lookup of translations in the server. In many cases (such as for button names, etc) the values don't change and can be generated into static code at compile time. So the https://github.com/saleyn/ei18n project addresses that optimization by allowing the user to define a translations XML file (see https://github.com/saleyn/ei18n/blob/master/README.md) and indicate which text blobs in a given language need to be statically defined (by using "static" attribute). The ei18n code generator emits a header file with constants that look like:

-define(HELLO, 1).

and the generated ei18n's translation modules resolve macros into language-specific text binaries, e.g.:

ei18n:get(?HELLO, en) -> <<"Hello">>.

So the modification to erlydtl_compiler is two-fold:

  1. Ensure that it can be provided the list of include files to parse for macros that can be referenced in the body of the DTL template.
  2. Ensure that a DTL template can reference macros in the trans tag in the form: {% trans "?HELLO" %}. So if the compiler sees the text prefixed with a '?', it treats it as a macro.
@evanmiller
Collaborator

Thanks for the additional information. I need to think about the macro stuff some more as it potentially creates conflicts on the template side. I'm trying to decide which is the better solution:

{% trans ?HELLO %}

or

{% trans "?HELLO" %}

If you want to send me a pull request implementing the translater_mf without the macros I will merge it in immediately. However, I'd like to gather feedback from other uses on the macro stuff before committing to it. One consideration is that by using macros outside of strings, they are potentially useful in contexts besides the {% trans %} tag.

@saleyn
saleyn commented Jan 18, 2013

This is exactly what I was debating myself, but decided to avoid changing the parser to accept macros natively, since the change wouldn't be fully DTL syntax compliant, and was worried that it would present an acceptance issue. Though writing ?HELLO rather than "?HELLO" is certainly more elegant.

And if it comes to modifying the parser, perhaps we could introduce a special syntax more simple than {% trans ?MACRO %} or {{ _(?MACRO) }} to deal with translations, e.g. ?MACRO? or {?MACRO}

@saleyn saleyn Add custom translation function
Translation functionality can be customized by replacing the hard-coded
`erlydtl_i18n:translate/2` with the function controlled by the option:
`{translater, {Module, Function}}`.
9b6f73b
@saleyn
saleyn commented Jan 18, 2013

I just repushed a change that renamed i18n_mf into translater. However, this change is not segregated from the rest, so it's ok if it waits until you make a decision to accept the whole patch.

One more note - there's another macro-related option I forgot to mention: translater_macro_prefix documented in the README. The purpose of it was to allow the user to define a prefix to be added to a macro name, so if it is "I18N_", then {% trans "?HELLO" %} would use a macro called ?I18N_HELLO. I had to do this to ensure that there are no macro name collisions, such as if one does {% trans "?EUNIT" %} it doesn't collide with eunit's macros.

@evanmiller
Collaborator

Can you use pre_render_vars to achieve the same goals as the macros?

@saleyn
saleyn commented Jan 21, 2013

If I understand you correctly, you are suggesting to do the following. Parse the include files for the list of macros, and produce a list of vars mapping all macros to underlying value constants. This list would be supplied to erlydtl_compiler at compile time. The DTL template would have to reference surrogate variable names, such as {% trans {{Hello}} %}, corresponding to a macro ?HELLO. However that trans tag syntax is invalid since the tag only accepts a string. Though it seems to me that providing some sort of native support for include_header macros in the template compiler would be beneficial beyond the scope of the {% trans %} tag.

@evanmiller
Collaborator

You've got the right idea -- and the {% trans %} tag does in fact accept variables so there wouldn't be any needed changes to the syntax.

Thinking about it more, I think while macros might be generally useful, using them for translation messages seems like a bad idea. You should probably store a list of messages or message identifiers in a file (e.g. a PO file) rather than pollute your macro namespace. The solution I have in mind would have you create a proplist of messages from a file -- call it "strings" -- and then do

{% trans strings.introductory_message %}

... providing strings at compile-time.

@saleyn
saleyn commented Jan 22, 2013

I'll try this approach and will comment here on any shortcomings. The only thing that I don't like so far is the fact that when building a site using complete internationalization, the {% trans strings.MSG %} or {{_(strings.MSG)}} are too verbose. Though I realize this is Django restriction rather than erlydtl. Ideally I'd like to see something like @MSG or some other simple syntax flavor reserved for translation. Otherwise when a template contains lots of international text it gets cluttered with symbols that provide nothing but attention distraction.

@saleyn
saleyn commented Jan 22, 2013

One problem that I see with this approach is that there needs to be a "pre-compile" hook that would allow the strings proplist to be populated by erlydtl_compiler before compiling a template. The compilation is done by rebar, which uses "erlydtl_opts" option to feed to the erlydtl_compiler. A work-around is to write a custom script that rebar would evaluate to produce the list of erlydtl_opts dynamically at compile time. However, this is inconvenient, since for every user application that uses gettext alternative translation, a custom rebar script would need to be written. A better solution would be to modify the erlydtl_compiler to accept an function that would dynamically populate pre-compile vars instead of feeding it a static list.

So I propose to introduce another compiler option: {dynamic_vars, [{VarName, {M,F,A}}]}. In the initialization of the context, the compiler will join the list provided in 'vars' option with a list produced by calling apply(M,F,A) for every VarName.

What do you think?

P.S. Just implemented a prototype of this approach and regretfully found that neither scoped compile-time variables (i.e. strings.SOME_VARIABLE) nor simple compile-time variables (all passed to the compiler with the {vars, Vars} option) seem to work. The feature works during rendering stage but not during compile-time. I created a separate post/issue #61 regarding compile-time variables.

@kaos kaos was assigned Dec 4, 2013
@kaos
Member
kaos commented Dec 4, 2013

I don't think we will merge this. But I have another idea.

I am working on making it a lot easier to extend/customize erlydtl for those not requiring 100% Django compat and have some custom features they'd like to add to it.

It is still kind of a moving target as I am working on it, but at some point I will have to document it, and it would be great with a sensible useful example to go with it.

And this is where this pull request comes in, as there are a bunch of changes/improvements as well as new features introduced at several levels, I would like to use this as an example for how you can apply some local customizations to erlydtl, without touching any of the erlydtl sources (thus not having to fork and modify it to suit your needs).

@saleyn
saleyn commented Dec 10, 2013

This sounds good. I had to put this project on hold for a while as I've been involved in some other development, but would be glad to come back when erlydtl is ready for supporting implementation of this feature.

@seriyps
Member
seriyps commented Feb 17, 2014

Looks like this one no longer relevant after #132 was merged =) But I just noticed some disadvantages of current compile-time translation API + rebar:

  • you need gettext server to be started before template compilation
  • you need to pass a fun to erlydtl:compile, but fun's are not allowed in rebar.config.

Both this things makes harder to use rebar to compile erlydtl templates in multiple languages.
I think, both can be fixed, if we accept blocktrans_fun :: fun(Key, Lang) | {Module::atom(), Function::atom()} (the second may be internally converted to 1'st by fun Module:Function/2.
So, [{blocktrans_fun, {my_module, my_function}}, {blocktrans_locales, ["en", ru", ...]}] can be added to rebar.config and Module:Function/2 may ensure that gettext server is started before querying it (or it may have hardcoded translations or anything else).

@kaos
Member
kaos commented Feb 17, 2014

First, to be clear. The changes in here are not considered for a merge into mainline.
They're too far away from "the Django way". But I found it to be an interesting exercise to show how it could be supported as an extension (once the extension functionality has been completed and matured a bit).

There are some changes in this pull request however, that I don't think has been resolved by #130/#132, and that is the macro expansion stuff.

@kaos kaos removed their assignment Feb 28, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment