Localisation Translation
Moved here. I'll delete this page in due course... please don't add anything else here.
WGACA APP was born en-GB - UK-London based. COVID-19's spread is worldwide and we think the APP and its underlying idea could be useful worldwide. Since WGACA is intended to be used by all kind of people, no english-language knowledge can be assumed on worldwide audience.
So the need for localised versions of WGACA APP.
This wiki page is intended to provide some sort of guidance to people wanting to fork their own country-localised WGACA clone, so to make their experience somewhat smooth and quick.
This kind of audience need to read just State of the Art chapter.
Localisation is not simply "Translation". Translation to another language is just one of the aspects.
We can identify easily at least the following localisation sub-topics:
- language translation
- data formats: e.g. currency, date format
- address data
Only translation has been somewhat managed by now.
Translation itself can be broken down to at least two subtopics:
- ANVIL's IDE design-time text-like components properties
- run-time strings set in code
By now, we've focused on step 1 only.
For the reasons outlined in History chapter, we've chosen to integrate excellent Shaun's Translations ANVIL's library. Currently, this library has two small bugs you'll need to fix yourself. See relative section below.
This library allows you to leave all your ANVIL Component's text properties unchanged, Translations will do the magic runtime.
This library requires you to:
- build a big dictionary, with all the translations you may need
- register all components (and their properties) you want to translate.
Building these two items by hand is long, difficult and error-prone. Se we've developed a small script that can do this automatically, in order to standardize things, reduce errors and guide you with code-snippets and suggestions in the adoption of Translations.
This is a task that cannot be made inside ANVIL's IDE. At lease not yet. You'll need a locally-installed python interpreter on you own.
You'll also need to install required libraries: yaml, dicttoxml and lxml.
Next you'll need to get the whole WGACA APP as a single YAML file. To do so, follow these steps.
First, you'll need to fork a WGACA clone in your ANVIL's account. If you're not new to ANVIL you already know what I'm talking about: just click this WGACA official DEV APP clone link to have it added to your ANVIL's space.
Then open your cloned APP, go to the settings menu:
images/translation/settings_menu.png
click on Share APP and then on Download as a file:
images/translation/download_as_file.png
Now you should have a nice and big .yaml file sitting in your Downloads folder.
Next, fire your preferred Python editor/IDE (e.g. PyCharm Community Edition), create a blank new project and paste this source code:
import yaml
import dicttoxml
from lxml import etree
with open("<your_downloaded_yaml_file_name>.yaml", 'r') as stream:
yaml_object = yaml.safe_load(stream)
xml_object = dicttoxml.dicttoxml(yaml_object)
root = etree.fromstring(xml_object)
tree = etree.ElementTree(root)
nodes = root.xpath("/root/forms//text")
locale = {}
forms = {}
for node in nodes:
node_text = node.text
# skip empty text fields
if (not node_text is None) and (len(node_text) > 0):
# get all sensitive info: form name, component name, component type
node_path = tree.getpath(node)
parent_base_path = '/'.join(node_path.split('/')[0:-2])
component_type_path = parent_base_path + '/type'
component_name_path = parent_base_path + '/name'
form_path = '/'.join(node_path.split('/')[0:4])+'/class_name'
form_name = root.xpath(form_path)[0].text
component_name = root.xpath(component_name_path)[0].text
component_type = root.xpath(component_type_path)[0].text
# create empty form dict if not yet existing
if not form_name in forms.keys():
forms[form_name] = {}
# populate translations dict
if node_text not in locale.keys():
locale[node_text] = 'your translation here'
# populate component's data inside form
if component_name not in forms[form_name].keys():
forms[form_name][component_name] = "Translations.register_translation(self.{cn}, 'text')".format(
cn=component_name)
with open('output.py','w', encoding="utf-8") as output:
output.encoding
output.write('##################################################\n')
output.write('## INSERT THIS DICTIONARY IN YOUR MAIN MODULE ##\n')
output.write('##################################################\n')
output.write('MY_LOCALE = {')
for text_key in locale.keys():
brackets = '"' if text_key.find('\n') == -1 else '"""'
#brackets = '"'
output.write('\t{br}{tk}{br}: {br}{tv}{br},\n'.format(br=brackets,
tk=text_key.replace('"','\\"'), #.replace('\n','\\n'),
tv=locale[text_key]))
output.write('}')
output.write('\n')
output.write('##################################################\n')
output.write('## INSERT THIS CODE IN YOUR MAIN\'s init method ##\n')
output.write('##################################################\n')
output.write('Translations.set_dictionary(\'<your_locale_2_digits_code>\', MY_LOCALE)\n')
output.write('Translations.set_locale(\'<your_locale_2_digits_code>\')\n')
output.write('Translations.initialise(\'<your_locale_2_digits_code>\')\n')
output.write('\n')
for form in forms.keys():
output.write('#################################################\n')
output.write('## INSERT THIS CODE IN FORM: {fn}\'s show method\n'.format(fn=form))
output.write('#################################################\n')
for component in forms[form].keys():
output.write('{cv}\n'.format(cv=forms[form][component]))
output.write('\n')Replace with your values these placeholders:
- <your_downloaded_yaml_file_name>
- <your_locale_2_digits_code>
and execute the script.
At the end, if everything proceeded without errors, you should have a nice output.py new file sitting in your folder.
Now you've built your dict and code snippets, let's see how to use them in your code.
Let's have a look at this file, here's an example from my IT translation.
At the beginning you have a big translation dictionary, with all the english textes found in the APP and a "your translation here" placeholder for your translations.
Please note that if the source english text is a multi-line text, then we suppose the translation will be a multi-line text as well, so triple brackets are used.
You'll need to place this big dictionary in your localised APP's main client module, just after all the import ... statements.
Next in output.py you'll find the code you need to insert in your APP's main client module Init method to initialise Translations library:
##################################################
## INSERT THIS CODE IN YOUR MAIN's init method ##
##################################################
Translations.set_dictionary('<your_locale_2_digits_code>', MY_LOCALE)
Translations.set_locale('<your_locale_2_digits_code>')
Translations.initialise('<your_locale_2_digits_code>')Note: you'll also need to include the appropriate import statement in your APP's main client module and in all the forms cited in your output.py
from Translations import TranslationsNext in output.py you'll find the code you need to insert in your APP forms' Show methods to get the translations magic happen, e.g.:
#################################################
## INSERT THIS CODE IN FORM: HomePage's show method
#################################################
Translations.register_translation(self.menu_about, 'text')
Translations.register_translation(self.menu_my_offers, 'text')
Translations.register_translation(self.menu_my_requests, 'text')
Translations.register_translation(self.menu_my_matches, 'text')
Translations.register_translation(self.menu_my_deliveries, 'text')
Translations.register_translation(self.menu_my_data, 'text')Then you should be done!
This section is needed until Shaun will publish an updated version of the official library, then will be removed. Most of this section's content is taken from this post on ANVIL's forum. In that post you can find all the story but here we're interested only to what you have to do in order to fix those two little bugs.
Open Translations library, head to Translations module code and look for this part:
TRANSLATIONS_LOWER = {}
def initialise(locale=None):
"""Set the library up - construct a lower-case dictionary so we can fall back to case-insensitive comparisons."""
for locale, dictionary in TRANSLATIONS.items():
for original, translated in dictionary.items():
if locale not in TRANSLATIONS_LOWER:
TRANSLATIONS_LOWER[locale] = {}
TRANSLATIONS_LOWER[original.lower()] = translated.lower() # <===== THIS LINE
if locale is not None:
set_locale(locale)Now, if the line highlghted with THIS LINE text is present, please change it to:
TRANSLATIONS_LOWER[locale][original.lower()] = translated.lower()
otherwise the key lookup in translate function will raise an error:
def translate(original):
"""Translate a given string."""
if LOCALE in TRANSLATIONS:
if original in TRANSLATIONS[LOCALE] or original.lower() in TRANSLATIONS_LOWER[LOCALE]:
return TRANSLATIONS[LOCALE][original]
return originalEven if Translations documentation page doesn't mention it, remember to call the initialise method after set_dictionary and set_locale.
Translations.initialise('IT')otherwise the TRANSLATIONS_LOWER dicitonary never gets built.
Not managed. Each localisation will need to change it by hand in cloned-APP code.
Currently WGACA APP is relying on a hard-coded London-only streets DB.
This DB must be localised by hand as well, at least until the main WGACA APP will refer to some kind of country-independent online street data service.
Labels, texts, wordings used on the pages and in most gui-element has a language specific text which is hardcoded in the first MVP version. (made for UK)
Italy, Netherlands and Sweden expressed need to localise.
A manual harcoded effort per country could initially be faster to market but Peter and Aldo had a dialogue regarding automation and building a simplified onboarding experience for new countries and languages.
There seemed to be two different approaches. Two different alternatives where discussed. One was related to using tools for looking through source code to automatically find text for translation. There where a need to perform an update on the MVP to make it language aware.
-
One approach was using dynamic variables and a reference lookup source where the hardcoded text needed to be changed and replaced and the text needed to be put in another central source of reference.
-
Another approach was using a similar one source for reference of translations but that could keep the sourcecode of existing hardcoded text to be unchanged and still be hardcoded. That approach used another way of registering each translatable component in the gui that needed to be translated.
Tommy saw a brilliant post by Saun. A Translation module
A dropdown to select the language for translation
Code snippets needed ( initial ... more was needed ... a bug showed later ...)
Aldo was “done” with a first iteration of a code that used a hybrid - yaml to get text out of source and then generated the Translation according to the post I mentioned from Saun.
In my clone I exported my WGACAUKDEVTEST.yaml file.
I executed specific code from Aldo to read yaml and to produce an output.py file.
It generated an output.py file which I then modified for the Swedish SE Locale and translation.
The outputed py-file than served as the master for the snippets and instructions in how to update the different fields in the right page in the app.
In my fork I exported the project yaml file - execeuted the code to get the output.py generated
Aldo provided an instruction - with a step by step …
Tommy translated the text to swedish in to a copy output_se.py with code and instructions from Aldo. Went to the Translation Core module from Sauns Translation Demo sample
Copy the existing Translation module Created the same Translation module Pasted the Translation code Added a Dependancy to Translation module Added from Translations import Translations in all pages Added the snippets according to the instruction on all pages/forms Addes the dictionary with the translation to the main/init Tested just starting the project First pages looked “OK” Sample of a page translated to Swedish - SE -locale
Aldo found a bug and solution - reported back to Saun
A Translation module bug and solution reported
The Translation module "worked" but had some issues with a "KeyError" message complaining on the locale was missing or incorrect for not succefully mapped entries in the dictionary.
The following 2 lines where also needed:
Translations.set_locale('SE')
Translations.initialise('SE')
Tommy verified the bug - same behaviour - applied Aldos fix which fixed and solved the problem.
We still have to find more fields to translate. Some pages is not yet in the yaml-file. Other components such as Buttons, Dropdowns with text is not within the yaml. They do not have 'text' attribute. Dropdown is a list of items as well and it is not sure how the Translate module yet support that.
Needs to be verified and checked with Saun