Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Internationalize Friendlycode #163

Merged
merged 37 commits into from

3 participants

@toolness
Owner

Here I have implemented a simple, quick solution to bootstrap localization for Friendlycode.

Goals of the bootstrapping:

  • Allow contributors to start localizing ASAP without blocking on us to "perfect" localization.
  • Potentially allow contributors to provide better copy for certain components, such as e.g. the overly-technical HTML/CSS help that was scraped off MDN, by contributing to the en localization.
  • Start with an extremely simple localization infrastructure: just key-value pairs, no initial support for pluralization, gender, or RTL languages. Evolve as needed, preferably without too much churn (e.g., constantly forcing contributors to re-localize the same content).
  • Allow contributors to provide feedback on the localization of an app so we can solve any problems sooner rather than later.
  • Use a simple but flexible data format for the localization that can easily be converted to whatever interchange format we end up using, be it .po files or .properties files or something else entirely.
  • When possible, use other abstractions that allow us to decouple each component of the localization pipeline from one another, allowing us to replace parts with more powerful alternatives as needed.

Ultimately the initial solution involved the following pieces:

  • Use the tiny requirejs i18n plugin and its i18n bundle convention to access localization data in the product's source code.
  • Create "stub" root i18n bundles in the product's source code that provide default data for the localization, "scraping" content from existing code/data files as necessary. This allows both for friendlycode to continue to minimally operate from flat files (without requiring a special server or build step) and also for a simple build-i18n.js script to scrape and generate initial source data for localization tools to consume.
  • Create a html-to-i18n-bundle.js module that converts slowparse's HTML error templates into the JSON format expected by requirejs-style i18n bundles. This effectively "scrapes" the templates, providing initial source data for localization tools to consume.
  • Create and document a tiny inline-l10n.js module that preprocesses HTML templates in friendlycode and outputs HTML templates with localized content. It can also "scrape" the templates and provide initial source data for localization tools to consume.
  • Create and document a simple localization editor, ghetto-l10n, which takes the output of build-i18n.js and allows contributors to easily localize content and quickly see the results of their work.
toolness added some commits
@toolness toolness Converted hacktionary-data to requirejs i18n bundles.
For more information on i18n bundles, see:

http://requirejs.org/docs/api.html#i18n
d533606
@toolness toolness converted MDN_URLS to requirejs i18n bundles. 4799d9a
@toolness toolness Internationalized slowparse-errors.
This is sort of weird and inefficient right now because we convert
from HTML to an i18nbundle and back again. However, in production
builds, we will be replacing the 'nls' paths with pre-built bundles
anyways, so only one direction will be computed in production.

Ideally we might want to move all this i18n stuff into slowparse,
though, and just couple slowparse to requirejs.
6183f65
@toolness toolness moved content in js/fc/nls/root/ into parent dir. 79c179a
@toolness toolness added build-i18n.js, which makes a big JSON blob of all bundles. b72d314
@toolness toolness added custom metadata to some i18n bundles. 9055ecc
@toolness toolness in build-i18n.js, add selective module export and allow underscore im…
…porting.
15ab560
@toolness toolness Added a very simple inline-l10n module. b7b9971
@toolness toolness added js/fc/nls/templates.js and modified template plugin to also loc…
…alize.
a64b158
@toolness toolness internationalized all templates. 80e944c
@toolness toolness added misc i18n bundle and made friendlycode.js use it. 539cdea
@toolness toolness made fc/ui/publish.js use fc/nls/misc. b6d90ab
@toolness toolness internationalized social-media.js. 134f64f
@toolness toolness added some metadata to i18n bundles. fc1d50a
@toolness toolness renamed fc/nls/templates to fc/nls/ui, and merged fc/nls/misc into it. 045f73a
@toolness toolness added descriptions to slowparse error i18n bundles. aa491e0
@toolness toolness for i18n bundle keys that originate in templates, show the template f…
…ilename as part of the key's help metadata.
f45beb0
@toolness toolness documented inline-l10n.js. 4988a38
@toolness
Owner

@stenington, do you think you could take a quick look at this?

@stenington stenington was assigned
@stenington
Owner

I think the code looks good, but there are so many moving parts it's pretty daunting. Maybe provide some information in the README about the i18n approach, and how people can provide localizations if they want to?

I tried to hand-localize a few strings just to make sure it worked, but I ran into a lot of trouble. For example, my understanding is that build-i18n.js should give me the initial localization data to start with, but when I run it I get

$ node build-i18n.js 
{}

I'm not sure if there's a bug, or if I'm supposed to be passing some arguments or something.

@toolness
Owner

Whoa, that is definitely odd--running what you did above should indeed give you a nice hearty JSON blob with tasty localization strings.

I wonder if it's related to the rather wide-open version requirement for requirejs in package.json... What does your node_modules/requirejs/package.json say its version is? Mine is 2.1.2.

@stenington
Owner

2.0.6... Looks like requiring 2.1.x fixes it.

@stenington
Owner

@toolness clarified for me that hand-localizing friendlycode is kind of out of scope for this pull. Integration with the ghetto-l10n thing actually happens by hacking the require config to point the various */nls/ paths at an entirely different url that serves the i18n bundles, and build-i18n.js is script that extracts data in a way that ghetto-l10n needs.

I think this is fine, although it's worth noting that if friendlycode is meant to be localizable by itself, more work may need to be done (instructions on how to do it in the README at the very least). Also, build-i18n.js might make more sense in the ghetto-l10n project rather than being a utility here.

@cmcavoy

Hey, @andrewhayward was looking for something to work on, so I asked him to take a look at this code.

@toolness
Owner

Actually, I am going to work on this a bit more today, by documenting it better and moving ghetto-l10n specific stuff (like build-i18n.js) out of the repository. Hold on a bit, @andrewhayward, this will all be less confusing soon!

toolness added some commits
@toolness toolness force requirement of requirejs 2.1.x in package.json. 3e4eaee
@toolness toolness Add error output for requirejs optimization failure. 625cb1b
@toolness toolness build-require.js works again. 45079a9
@toolness toolness slowparse errors are now stored as raw i18n json bundles in optimized…
… builds.

previously, their raw HTML was being parsed into i18n bundle format
at runtime.
aa6ffad
@toolness toolness refactorings to make l10n infrastructure easier to understand. a43b0fb
@toolness toolness moved build-i18n.js and build-require.js into a 'bin' directory. bb889da
@toolness toolness added node-tap tests for build-i18n.js, and fixed a bug in html-to-i1…
…8n.js.
1205d4f
@toolness toolness 'npm test' now also tests unoptimized and optimized qunit tests, buil…
…d step.
dd2670b
@toolness toolness removed self-test from bin/build-require.js, since node-tap tests obv…
…iate it.
d298c31
@toolness toolness Added .travis.yml. a898b06
@toolness toolness added express to package.json 9a72ebc
@toolness toolness added travis badge to readme. 008aa65
@toolness toolness added a quick-start guide and prereqs to the readme. also added bin/s…
…erver.js.
80c2f3d
@toolness toolness Abnormal termination when CSS compilation fails. ba88096
@toolness toolness move github URL into config instead of hardcoding it 6debbd7
@toolness toolness template plugin and build-i18n no longer hard-code template and i18n …
…dirs.
311384f
@toolness toolness ensure that optimized js and css files are generated by build-require…
….js.
1b695f9
@toolness toolness added 'template' and 'list' sub-cmds to build-i18n, and l10n instruct…
…ions.
7176a2f
@toolness
Owner

Ok @stenington, I did some refactorings to make this stuff hopefully easier to understand, and also turned build-i18n.js into a really simple tool for localizing stuff w/o any other prototypey tools like ghetto-l10n. Instructions for doing this are now in the README: https://github.com/toolness/friendlycode/tree/i18n#localization.

While I was at it, I also made sure that the localization stuff works w/ optimized builds, and added Travis CI integration, which runs the qunit test suite on unoptimized and optimized builds, as well as some other random shit.

@toolness
Owner

I'm pretty confident that this code doesn't affect the stability of the software, and it's easy to change anything that anyone has a problem with after it lands. It also doesn't tie us to a particular l10n strategy--all it does is add a basic lightweight infrastructure for using key-value localization, and it doesn't even commit us to a particular interchange format like .po or .property files (contributions made in the form of requirejs i18n modules can easily be converted to such formats if/when we decide on them).

As such, this pull request was designed to be something we can quickly iterate on--not something that forces all our projects on a path that we can't back away from, that we need to fear landing until representatives from other projects have given it their blessing.

The fact that l10n touches virtually all parts of the codebase means that it's very hard to keep from bit-rotting as other changes are introduced into the project, so I'm going to merge this now and any changes we decide on after evaluating the changes here can be addressed in subsequent commits.

@toolness toolness merged commit ca0b19d into mozilla:gh-pages
This was referenced
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 3, 2012
  1. @toolness

    Converted hacktionary-data to requirejs i18n bundles.

    toolness authored
    For more information on i18n bundles, see:
    
    http://requirejs.org/docs/api.html#i18n
  2. @toolness
  3. @toolness

    Internationalized slowparse-errors.

    toolness authored
    This is sort of weird and inefficient right now because we convert
    from HTML to an i18nbundle and back again. However, in production
    builds, we will be replacing the 'nls' paths with pre-built bundles
    anyways, so only one direction will be computed in production.
    
    Ideally we might want to move all this i18n stuff into slowparse,
    though, and just couple slowparse to requirejs.
  4. @toolness
  5. @toolness
  6. @toolness
Commits on Dec 4, 2012
  1. @toolness
  2. @toolness
  3. @toolness
Commits on Dec 5, 2012
  1. @toolness
  2. @toolness
  3. @toolness
  4. @toolness
  5. @toolness
  6. @toolness
  7. @toolness
  8. @toolness

    for i18n bundle keys that originate in templates, show the template f…

    toolness authored
    …ilename as part of the key's help metadata.
  9. @toolness

    documented inline-l10n.js.

    toolness authored
Commits on Jan 15, 2013
  1. @toolness
  2. @toolness
  3. @toolness

    build-require.js works again.

    toolness authored
  4. @toolness

    slowparse errors are now stored as raw i18n json bundles in optimized…

    toolness authored
    … builds.
    
    previously, their raw HTML was being parsed into i18n bundle format
    at runtime.
  5. @toolness
  6. @toolness
  7. @toolness
  8. @toolness
  9. @toolness
  10. @toolness

    Added .travis.yml.

    toolness authored
  11. @toolness

    added express to package.json

    toolness authored
  12. @toolness

    added travis badge to readme.

    toolness authored
  13. @toolness
Commits on Jan 16, 2013
  1. @toolness
  2. @toolness
  3. @toolness
  4. @toolness
  5. @toolness
  6. @toolness
Something went wrong with that request. Please try again.