Disclamer: This Gem is not yet production ready, if you are interested in using it please be prepared to become a collaborator. If you are interested, please see Next Steps at the end of this document.
Lingua Franca is a rails i18n plugin. It provides the following services to you app:
Lingua Franca’s most unique feature is that it keeps an up to date translation list by listening to your code during integration tests. When your integration tests begin all translation info is removed, each time the translation method I18n.t is called, that key is added to an info file along with relevant context such the current page. In the end, you will be able to give translators only the translations currently and use, and assuming you have good code coverage for your app, all of the translations in use.
In addition to collecting your translated content, if you use the Lingua Franca driver for Capybara, you will also get failed tests if untranslated content is seen. When calling visit(page) Lingua Franca returns the empty string for all translations including dynamic content. It then stips all HTML tags and if anything is left over, your test fails. You can mark untranslatable content using the _! method.
You don’t need to worry about setting up Lingua Franca for your test suite, all you need to do is write those tests and execute them. RSpec along with Capybara are highly recommended.
Assuming you have a static home page, all you will need to do to collect all of the translations for that page. A test for this could resemble to following:
ENV["RAILS_ENV"] ||= 'test' require File.expand_path("../../config/environment", __FILE__) require 'rspec/rails' require 'capybara/rails' require 'capybara/rspec' require 'capybara/poltergeist' Capybara.configure do |c| c.run_server = true c.javascript_driver = :poltergeist c.default_driver = :poltergeist end feature 'Home page' do scenario 'user visits the home page' do visit '/' end end
Keep in mind that this example test doesn’t actually test your code very well, as long as the page is generated the test will pass. You should accompany the test with some expectations but even so this will at least ensure that your pages are still there and work and ensures that you have all the current translations.
To enable example pages for your translators, replace your default driver with :lingua_franca_poltergeist or :lingua_franca_selenium.
ENV["RAILS_ENV"] ||= 'test' require File.expand_path("../../config/environment", __FILE__) require 'rspec/rails' require 'capybara/rails' require 'capybara/rspec' require 'capybara/poltergeist' Capybara.configure do |c| c.run_server = true c.javascript_driver = :lingua_franca_poltergeist c.default_driver = :lingua_franca_poltergeist end feature 'Home page' do scenario 'user visits the home page' do visit '/' end end
Lingua Franca provides translators with a user interface to add and edit translations. Using a list of collected translation keys you users will be able to select any language that they chose and translate you entire site. The following is a screen capture of the included test_app application. Linguage Franca provides you with terse HTML markup that allows you to style the translation pages just about any way you want, but you do need to style them yourself. Templates are also used so you may override them if you choose.
If you use one of the Lingua Franca drivers for Capybara, you’ll also get previews of each page tested so that your translators can have even better context for the given translation:
By default, Lingua Franca will allow users to select from any locale which already has some based definitions provided by the rails-i18n gem. Currently, only two letter locales are supported, sub locales such as en-GB are not available yet.
A locale becomes enabled once it has met a minimum translation coverage, by default this is set to 80%. So if you have a total of 1000 translation keys, you will need at least 800 of them to be complete in order for users to view your site in that language.
Along with the base translations provided by rails-i18n, Lingua Franca provides a rake task +rake lingua_franca:import+ which collects translations in all available locales from a variety of APIs.
A list of most known languages in all available languages is collected from unicode.org. This repo contains a list which is updated occasionally so the task will select the latest data and import languages into keys in the format languages.code. For example, calling I18n.t('languages.de', locale => :en) will produce 'German' while I18n.t('languages.de', locale => :fr) will produce 'allemand'. The list may include languages which are not available for translations.
A list of countries and their sub-regions (provinces, states, territories, etc.) is provided by geonames.org. To enable this you will need to create an account at geonames and enable the api. Once this is done you will need to set the importer => geonames => username config variable in lingua_franca.yml in your config directory.
The task will provide you with keys and values in the format: geography.countries.COUNTRY_CODE and geography.subregions.COUNTRY_CODE.REGION_CODE. For example calling I18n.t('geography.countries.US', locale => :en) will return 'United States' while calling I18n.t('geography.countries.US', locale => :fr) will return 'États-Unis' and I18n.t('geography.subregions.US.NM', locale => :en) will return 'New Mexico' while calling I18n.t('geography.subregions.US.NM', locale => :fr) will return 'Nouveau-Mexique'.
Lingua Franca makes a record of every translation when it is saved by a translator containing the key, value, time, and translator id. These records can enable change comparisons and well a rolling back translations if required.
Lingua Franca also plugs into your ActiveRecord models using the ‘translates` helper method. When defining your model, include this method to mark which fields can be translated, for example:
class Post < ActiveRecord::Base translates :title, :content belongs_to :user end
This will allow you to save and retrieve the marked fields in any enabled language. As long as a translation exists, no extra effort is required, if for example a Post object was originally created in the en locale, if the user’s language is currently set to fr all you need to do is call mypost.title and the French version of the title is returned. You may also call mypost.title! to retrieve the original untranslated version.
Keep in mind, each translatable model will also need a locale field to store the content’s original locale. This property is updated automatically when the object is first saved using the current user’s current locale.
Dynamic translations are recorded in the database as DynamicTranslationRecord objects which contain the translated content, translator id, date, as well as the model name and id.
No user interface is provided by default for saving and updating translatable content but just like retrieving data, if you update an object while in a locale other than the original locale in which the object was created, only the current locale’s version is updated.
For example, if a Post object was originally created by an English user it may look something like the following:
mypost = { id: 10, title: 'My Title', content: 'My posts\'s content...', created_by: 1, locale: 'en' }
If a French user now updates the object this original object will remain the same in the DB but a set of DynmanicTranslationRecords will be saved that may look like:
mypost_translation_records = [{
id: 1,
model_type: 'post'
model_id: 10,
column: 'title',
content: 'Mon titre',
translator_id: 2,
locale: 'fr'
}, {
id: 2,
model_type: 'post'
model_id: 10,
column: 'content',
content: 'Le contenu de mon post...',
translator_id: 2,
locale: 'fr'
}]
If a French user then retrieve’s the object for display, they should then see an object that appears as:
mypost = { id: 10, title: 'Mon titre', content: 'Le contenu de mon post...', created_by: 1, locale: 'en' }
Lingua Franca will detect the current language using the current URL scheme which you can customize. Here is a recommended resource when considering which technique you should use for your site: support.google.com/webmasters/answer/182192#2.
If the language cannot be detected using the selected method, it will attempt to look at the Accept-Language header and redirect the user to the proper URL for that language and resource. If that fails it will fallback to I18n.config.default_locale.
If the locale is successfully detected but is not yet fully translated, a 404 exception will be thrown allowing the developer a chance to explain to the user that their language is not yet available and a chance to ask to the user to volunteer if possible.
If the locale is successfully detected and is available but it is determined not to be the best language for the user, a banner may be shown
You can set the detection method by setting the I18n.config.language_detection_method to one of the following values:
Example: http://www.yoursite.com/mypage?locale=code
The language may be automatically detected by URL parameters. The parameter need only be provided once per session, the setting is then saved in the user’s session and will not change until the session expires or a different locale parameter is provided.
Example :http://locale-code.yoursite.com
For example, assuming your site is called example.com, if a user visits en.example.com, they will see the site in English, if they visit es.example.com, they will see Spanish. If the user navigates to http://example.com or http://www.example.com they will be redirected to http://es.example.com if that is selected to be the best choice for that user.
Example: http://example.com/locale/mypage
Not Yet Implemented
Lingua Franca modifies the way that the native translator works:
By default, fallback text is generated by looking at the last portion of the key provided with underscores replaced by spaces. For example if you were to execute I18n.t 'my_concern.this_needs_translation' you will receive 'this needs translation' as the output string. This may be useful for developers both for generating useful key names and also see text while developing without providing translations themselves. Because of this, uppercase keys are encouraged, for example:
<h1><% I18n.t 'titles.This_is_the_Page_Title'%></h1>
Thus generating:
<h1>This is the Page Title</h1>
The context parameter allows you to provide alternate fallback text but more importantly, meaningless lorem ipsum text. If context is a string and is not recognized as a valid context, the context itself is returned as the fallback text:
<h1><% I18n.t 'titles.this_page_title', :context => 'My Awesome Page Title' %></h1>
Thus generating:
<h1>My Awesome Page Title</h1>
However, if the context is a recognized context, lorem ipsum text will be returned instead and the context will be recorded during tests to provide translators with more…well, context.
- character, characters, char, c
-
A single random lowercase character, or if
context_sizeis provided a a string of capitalized characters:I18n.t 'concern.a_char' :context => :character --> 'f'I18n.t 'concern.some_chars' :context => :characters, :context_size => 8 --> 'Vel pede' - word, words, w
-
A single word or if
context_sizeis provided a string of words:I18n.t 'concern.a_word' :context => :word --> 'Erat'I18n.t 'concern.some_words' :context => :words, :context_size => 4 --> 'Cubilia curae donec pharetra' - sentence, sentences, s
-
A single sentence or if
context_sizeis provided a string of sentences which should each be delimited by periods:I18n.t 'concern.a_sentence' :context => :sentence --> 'Morbi a ipsum.'I18n.t 'concern.some_sentences' :context => :words, :context_size => 4 --> 'Cubilia curae donec pharetra' - paragraph, paragraphs, p
-
A set of n sentences where n is determined by
I18n.config.default_paragraph_lengthor 10 by default or the number provided bycontext_sizeif provided. Ifcontext_sizeis provided a paragraphs will be determined, each of a random length of sentences:I18n.t 'concern.a_paragraph' :context => :paragraph --> 'In quis justo. Maecenas rhoncus aliquam lacus. Morbi quis tortor id nulla ultrices aliquet. Maecenas leo odio, condiment um id, luctus nec, molestie sed, justo. Pellentesque viverra pede ac diam. Cras pellentesque volutpat dui. Maecenas tris tique, est et tempus semper, est quam pharetra magna, ac consequat metus sapien ut nunc. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Mauris viverra diam vitae quam.'. You should generally avoid expecting more than a single paragraph from translators, which would both be far less manageable for them but may also cause problems parsing output later. - title, titles, t
-
A single sentence of either random length or
context_sizeif provided and transformed into titlecase and does not include a trailing period:I18n.t 'concern.a_title' :context => :title --> 'Proin At Turpis A Pede Posuere Nonummy'
Your main tool for translating should be the underscore helper method _, it is available to all views. It does the work of looking up a translation and surrounding the text with markup to help translators translate the text (when needed). it takes your key as the first parameter and the context and context_size as optional second and thrid parameters.
<h1><% _'page_titles.A_Simple_View', :title %></h1> <p><% _'page_intros.simple_view_description', :paragraph %></p>
For most users, assuming that translations are missing, they will see something like:
<h1> Fusce Posuere Felis Sed Lacus </h1> <p> Praesent id massa id nisl venenatis lacinia. Aenean sit amet justo. Morbi ut odio. Cras mi pede,malesuada in, imperdiet et, commodo vulputate, justo. In blandit ultrices enim. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Proin interdum mauris non ligula pellentesque ultrices. Phasellus id sapien in sapien iaculis congue. Vivamus metus arcu, adipiscing molestie, hendrerit at, vulputate vitae, nisl. Aenean lectus. </p>
For translators on the other hand, they may see markup similar to the following:
<h1>
<span class="translated-content" data-i18n-key="page_titles.A_Simple_View" data-i18n-needs-translation="1">
<a href="/translations/en/?concern=translate#page_titles.A_Simple_View" class="translation-link">
Translate
</a>
Fusce Posuere Felis Sed Lacus
</span>
</h1>
<p>
<span class="translated-content" data-i18n-key="page_intros.simple_view_description" data-i18n-needs-translation="1">
<a href="/translations/en/?concern=translate#page_intros.simple_view_description" class="translation-link">
Translate
</a>
Fusce Posuere Felis Sed Lacus
</span>
</p>
This will allow translators to see which text requires translation and translate it immediately.
Haml is highly encouraged as it, apart from all of the other benefits, greatly decreases the amount you will need to type for each translation and make the markup much more readable
%h1=_'page_titles.A_Simple_View', :t %p=_'page_intros.simple_view_description', :p
Notice also here that we are using the context abbreviations: t and p as opposed to title and paragraph
Sometimes surrounding your translation in HTML will not produce a desired output, in this case you can use code blocks to wrap around elements.
%h1=_'page_titles.My_Image'
=_'images.myimage.alt_text', :t do |my_alt_text|
%img{src: 'myimage.png', alt: my_alt_text}
This will provide the user with:
<h1> My Image </h1> <img src="myimage.png" alt="Duis Ac Nibh">
More importantly this will provide the translators with:
<h1>
<span class="translated-content" data-i18n-key="page_titles.My_Image" data-i18n-needs-translation="1">
<a href="/translations/en/?concern=translate#page_titles.My_Image" class="translation-link">
Translate
</a>
My Image
</span>
</h1>
<span class="translated-content" data-i18n-key="images.myimage.alt_text" data-i18n-needs-translation="1">
<a href="/translations/en/?concern=translate#images.myimage.alt_text" class="translation-link">
Translate
</a>
<img src="myimage.png" alt="Duis Ac Nibh">
</span>
If you need multiple attributes, you may provide an array instead:
%h1=_'page_titles.My_Image'
=_(['images.myimage.alt_text', 'images.myimage.title'], :t) do |my_alt_text, my_title_text|
%img{src: 'myimage.png', alt: my_alt_text, title: my_title_text}
Which may generate something like:
<h1> My Image </h1> <img src="myimage.png" alt="Duis Ac Nibh" title="Proin Interdum Mauris Non Ligula Pellentesque Ultrices">
Lingua Franca also alters the default text provided by form helpers, it will generate default translation keys for you based on the inputs provided:
=form_for @user do |user_form| =user_form.label :username =user_form.text_field :username, placeholder: true =user_form.submit
This provides the label text with the key: forms.labels.user.username, the placeholder for the text field will be given a key of forms.placeholders.user.username, and the submit button’s value will be given a key of forms.actions.user.update. If the submit button is called as such: =user_form.submit :login it’s key will instead be: forms.actions.user.login.
For select fields, you may call options_for_select using an single dimensional array (['a', 'b', 'c'...]) instead of the multi-dimensional array of keys and values normally required ([['a', 1], ['b', 2], ['c', 3]...]). This will then give each option a value equal to that in the array and a key in the form of forms.options.input_name.option_value. For example:
=select_tag :colour, options_for_select(['red', 'green', 'blue'], 'green')
Now if the selected language is French and values are provided for the following keys: forms.options.colours.red, forms.options.colours.green, forms.options.colours.blue. The output will be:
<select name="colours" id="colours"> <option value="red">Rouge</option> <option value="green" selected="selected">Vert</option> <option value="blue">Bleu</option> </select>
These keys may be altered by providing the select_tag with a :scope option. For example, you may which to provide a list of countries which is already provided to you by Franca Lingua in the form of ‘geography.countries.XX’ so calling:
=select_tag :country, options_for_select(['CA', 'MX', 'US'], 'CA'), scope: 'geography.countries'
will instead look up the keys: geography.countries.CA, geography.countries.MX, and geography.countries.US. It will provide French users with the following:
<select name="country" id="country"> <option value="CA" selected="selected">Canada</option> <option value="MX">Mexique</option> <option value="US">États-Unis</option> </select>
I am currently looking for collaborators, this is my first ruby gem and I feel like I could definitely use some experienced gem developers to help make the code follow better practices. In addition I could use help with the following:
This is my current concern, I am working on this but there are a lot of cases to cover, and testing a product that itself integrates with testing creates some interesting problems. Also, just using it it multiple apps and tweaking to be more customizable would be extremely helpful.
I’m starting to feel that the scope of this gem is too large for one product and it should perhaps be split into its various components.
There are things that the UI could deliver that it is currently not, such as providing translations in alternate languages for context and a better change history.
We’re collecting info about languages and geography, we can probably do better, especially since we’re already using {unicode.org}[ftp://unicode.org/Public/cldr/]s vast repository.