Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Internationalize Friendlycode #163

Merged
merged 37 commits into from

3 participants

Atul Varma Mike Larsson Chris McAvoy
Atul Varma
Owner

Here I have implemented a simple, quick solution to bootstrap localization for Friendlycode.

Goals of the bootstrapping:

  • Allow contributors to start localizing ASAP without blocking on us to "perfect" localization.
  • Potentially allow contributors to provide better copy for certain components, such as e.g. the overly-technical HTML/CSS help that was scraped off MDN, by contributing to the en localization.
  • Start with an extremely simple localization infrastructure: just key-value pairs, no initial support for pluralization, gender, or RTL languages. Evolve as needed, preferably without too much churn (e.g., constantly forcing contributors to re-localize the same content).
  • Allow contributors to provide feedback on the localization of an app so we can solve any problems sooner rather than later.
  • Use a simple but flexible data format for the localization that can easily be converted to whatever interchange format we end up using, be it .po files or .properties files or something else entirely.
  • When possible, use other abstractions that allow us to decouple each component of the localization pipeline from one another, allowing us to replace parts with more powerful alternatives as needed.

Ultimately the initial solution involved the following pieces:

  • Use the tiny requirejs i18n plugin and its i18n bundle convention to access localization data in the product's source code.
  • Create "stub" root i18n bundles in the product's source code that provide default data for the localization, "scraping" content from existing code/data files as necessary. This allows both for friendlycode to continue to minimally operate from flat files (without requiring a special server or build step) and also for a simple build-i18n.js script to scrape and generate initial source data for localization tools to consume.
  • Create a html-to-i18n-bundle.js module that converts slowparse's HTML error templates into the JSON format expected by requirejs-style i18n bundles. This effectively "scrapes" the templates, providing initial source data for localization tools to consume.
  • Create and document a tiny inline-l10n.js module that preprocesses HTML templates in friendlycode and outputs HTML templates with localized content. It can also "scrape" the templates and provide initial source data for localization tools to consume.
  • Create and document a simple localization editor, ghetto-l10n, which takes the output of build-i18n.js and allows contributors to easily localize content and quickly see the results of their work.
toolness added some commits
Atul Varma toolness Converted hacktionary-data to requirejs i18n bundles.
For more information on i18n bundles, see:

http://requirejs.org/docs/api.html#i18n
d533606
Atul Varma toolness converted MDN_URLS to requirejs i18n bundles. 4799d9a
Atul Varma toolness Internationalized slowparse-errors.
This is sort of weird and inefficient right now because we convert
from HTML to an i18nbundle and back again. However, in production
builds, we will be replacing the 'nls' paths with pre-built bundles
anyways, so only one direction will be computed in production.

Ideally we might want to move all this i18n stuff into slowparse,
though, and just couple slowparse to requirejs.
6183f65
Atul Varma toolness moved content in js/fc/nls/root/ into parent dir. 79c179a
Atul Varma toolness added build-i18n.js, which makes a big JSON blob of all bundles. b72d314
Atul Varma toolness added custom metadata to some i18n bundles. 9055ecc
Atul Varma toolness in build-i18n.js, add selective module export and allow underscore im…
…porting.
15ab560
Atul Varma toolness Added a very simple inline-l10n module. b7b9971
Atul Varma toolness added js/fc/nls/templates.js and modified template plugin to also loc…
…alize.
a64b158
Atul Varma toolness internationalized all templates. 80e944c
Atul Varma toolness added misc i18n bundle and made friendlycode.js use it. 539cdea
Atul Varma toolness made fc/ui/publish.js use fc/nls/misc. b6d90ab
Atul Varma toolness internationalized social-media.js. 134f64f
Atul Varma toolness added some metadata to i18n bundles. fc1d50a
Atul Varma toolness renamed fc/nls/templates to fc/nls/ui, and merged fc/nls/misc into it. 045f73a
Atul Varma toolness added descriptions to slowparse error i18n bundles. aa491e0
Atul Varma toolness for i18n bundle keys that originate in templates, show the template f…
…ilename as part of the key's help metadata.
f45beb0
Atul Varma toolness documented inline-l10n.js. 4988a38
Atul Varma
Owner

@stenington, do you think you could take a quick look at this?

Mike Larsson stenington was assigned
Mike Larsson
Owner

I think the code looks good, but there are so many moving parts it's pretty daunting. Maybe provide some information in the README about the i18n approach, and how people can provide localizations if they want to?

I tried to hand-localize a few strings just to make sure it worked, but I ran into a lot of trouble. For example, my understanding is that build-i18n.js should give me the initial localization data to start with, but when I run it I get

$ node build-i18n.js 
{}

I'm not sure if there's a bug, or if I'm supposed to be passing some arguments or something.

Atul Varma
Owner

Whoa, that is definitely odd--running what you did above should indeed give you a nice hearty JSON blob with tasty localization strings.

I wonder if it's related to the rather wide-open version requirement for requirejs in package.json... What does your node_modules/requirejs/package.json say its version is? Mine is 2.1.2.

Mike Larsson
Owner

2.0.6... Looks like requiring 2.1.x fixes it.

Mike Larsson
Owner

@toolness clarified for me that hand-localizing friendlycode is kind of out of scope for this pull. Integration with the ghetto-l10n thing actually happens by hacking the require config to point the various */nls/ paths at an entirely different url that serves the i18n bundles, and build-i18n.js is script that extracts data in a way that ghetto-l10n needs.

I think this is fine, although it's worth noting that if friendlycode is meant to be localizable by itself, more work may need to be done (instructions on how to do it in the README at the very least). Also, build-i18n.js might make more sense in the ghetto-l10n project rather than being a utility here.

Chris McAvoy

Hey, @andrewhayward was looking for something to work on, so I asked him to take a look at this code.

Atul Varma
Owner

Actually, I am going to work on this a bit more today, by documenting it better and moving ghetto-l10n specific stuff (like build-i18n.js) out of the repository. Hold on a bit, @andrewhayward, this will all be less confusing soon!

toolness added some commits
Atul Varma toolness force requirement of requirejs 2.1.x in package.json. 3e4eaee
Atul Varma toolness Add error output for requirejs optimization failure. 625cb1b
Atul Varma toolness build-require.js works again. 45079a9
Atul Varma toolness slowparse errors are now stored as raw i18n json bundles in optimized…
… builds.

previously, their raw HTML was being parsed into i18n bundle format
at runtime.
aa6ffad
Atul Varma toolness refactorings to make l10n infrastructure easier to understand. a43b0fb
Atul Varma toolness moved build-i18n.js and build-require.js into a 'bin' directory. bb889da
Atul Varma toolness added node-tap tests for build-i18n.js, and fixed a bug in html-to-i1…
…8n.js.
1205d4f
Atul Varma toolness 'npm test' now also tests unoptimized and optimized qunit tests, buil…
…d step.
dd2670b
Atul Varma toolness removed self-test from bin/build-require.js, since node-tap tests obv…
…iate it.
d298c31
Atul Varma toolness Added .travis.yml. a898b06
Atul Varma toolness added express to package.json 9a72ebc
Atul Varma toolness added travis badge to readme. 008aa65
Atul Varma toolness added a quick-start guide and prereqs to the readme. also added bin/s…
…erver.js.
80c2f3d
Atul Varma toolness Abnormal termination when CSS compilation fails. ba88096
Atul Varma toolness move github URL into config instead of hardcoding it 6debbd7
Atul Varma toolness template plugin and build-i18n no longer hard-code template and i18n …
…dirs.
311384f
Atul Varma toolness ensure that optimized js and css files are generated by build-require…
….js.
1b695f9
Atul Varma toolness added 'template' and 'list' sub-cmds to build-i18n, and l10n instruct…
…ions.
7176a2f
Atul Varma
Owner

Ok @stenington, I did some refactorings to make this stuff hopefully easier to understand, and also turned build-i18n.js into a really simple tool for localizing stuff w/o any other prototypey tools like ghetto-l10n. Instructions for doing this are now in the README: https://github.com/toolness/friendlycode/tree/i18n#localization.

While I was at it, I also made sure that the localization stuff works w/ optimized builds, and added Travis CI integration, which runs the qunit test suite on unoptimized and optimized builds, as well as some other random shit.

Atul Varma
Owner

I'm pretty confident that this code doesn't affect the stability of the software, and it's easy to change anything that anyone has a problem with after it lands. It also doesn't tie us to a particular l10n strategy--all it does is add a basic lightweight infrastructure for using key-value localization, and it doesn't even commit us to a particular interchange format like .po or .property files (contributions made in the form of requirejs i18n modules can easily be converted to such formats if/when we decide on them).

As such, this pull request was designed to be something we can quickly iterate on--not something that forces all our projects on a path that we can't back away from, that we need to fear landing until representatives from other projects have given it their blessing.

The fact that l10n touches virtually all parts of the codebase means that it's very hard to keep from bit-rotting as other changes are introduced into the project, so I'm going to merge this now and any changes we decide on after evaluating the changes here can be addressed in subsequent commits.

Atul Varma toolness merged commit ca0b19d into from
This was referenced
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 3, 2012
  1. Atul Varma

    Converted hacktionary-data to requirejs i18n bundles.

    toolness authored
    For more information on i18n bundles, see:
    
    http://requirejs.org/docs/api.html#i18n
  2. Atul Varma
  3. Atul Varma

    Internationalized slowparse-errors.

    toolness authored
    This is sort of weird and inefficient right now because we convert
    from HTML to an i18nbundle and back again. However, in production
    builds, we will be replacing the 'nls' paths with pre-built bundles
    anyways, so only one direction will be computed in production.
    
    Ideally we might want to move all this i18n stuff into slowparse,
    though, and just couple slowparse to requirejs.
  4. Atul Varma
  5. Atul Varma
  6. Atul Varma
Commits on Dec 4, 2012
  1. Atul Varma
  2. Atul Varma
  3. Atul Varma
Commits on Dec 5, 2012
  1. Atul Varma
  2. Atul Varma
  3. Atul Varma
  4. Atul Varma
  5. Atul Varma
  6. Atul Varma
  7. Atul Varma
  8. Atul Varma

    for i18n bundle keys that originate in templates, show the template f…

    toolness authored
    …ilename as part of the key's help metadata.
  9. Atul Varma

    documented inline-l10n.js.

    toolness authored
Commits on Jan 15, 2013
  1. Atul Varma
  2. Atul Varma
  3. Atul Varma

    build-require.js works again.

    toolness authored
  4. Atul Varma

    slowparse errors are now stored as raw i18n json bundles in optimized…

    toolness authored
    … builds.
    
    previously, their raw HTML was being parsed into i18n bundle format
    at runtime.
  5. Atul Varma
  6. Atul Varma
  7. Atul Varma
  8. Atul Varma
  9. Atul Varma
  10. Atul Varma

    Added .travis.yml.

    toolness authored
  11. Atul Varma

    added express to package.json

    toolness authored
  12. Atul Varma

    added travis badge to readme.

    toolness authored
  13. Atul Varma
Commits on Jan 16, 2013
  1. Atul Varma
  2. Atul Varma
  3. Atul Varma
  4. Atul Varma
  5. Atul Varma
  6. Atul Varma
Something went wrong with that request. Please try again.