Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal for a Simplified I18N Process #41437

Open
HansMuller opened this issue Sep 27, 2019 · 39 comments
Open

Proposal for a Simplified I18N Process #41437

HansMuller opened this issue Sep 27, 2019 · 39 comments

Comments

@HansMuller
Copy link
Contributor

@HansMuller HansMuller commented Sep 27, 2019

Introduction

This is a proposal to provide a tool that simplifies setting up and maintaining an internationalized Flutter app. It is based on the Dart intl package. The overall goal is to reduce the number of steps required to use that package in an app with modest i18n requirements.

Current Internationalization Process

Currently, internationalized apps which are based on the intl package must follow a procedure similar to the one outlined in the stocks example. Roughly speaking there are 5 steps:

  • Add calls to the static Intl.message and Intl.plural methods where message strings are needed by the app.
  • Generate a template ".arb" (application resource bundle) file with intl_translation:extract_to_arb.
  • Make copies of the template ".arb" file for each supported locale, and add message translations.
  • Generate several source files which collectively encode all of the localizations using intl_translation:generate_from_arb.
  • Before the app starts, call initializeMessages() for each supported locale.

Proposal

The proposed approach is to start with handwritten ".arb" files. One of these files will contain the standard per-message metadata required by the intl package.

A new tool will be used to generate a single class that contains one method per message.

Message lookup is based on the inherited locale, using the current BuildContext:

MyLocalizations.of(context).myMessage()

The generated class will provide a LocalizationDelegate which is to be included in the app's list of localization delegates. The generated class itself will depend on intl package; the rest of the app doesn't have to.

This approach, based on segregating the app's localizations in a single class, is similar to the stocks example.

Prototype App and Tool

A prototype of the tool is #41432.

The main elements of the app, including the generated code, are as follows.

The English locale ".arb" file, lib/l10n/demo_en.arb defines the messages. The Spanish locale arb file, lib/l10n/demo_es.arb, just defines message translations.

The generated localizations class, DemoLocalizations in lib/1l0n/demo_localizations.dart, provides all of the messages.

The app includes DemoLocalizations.delegate in its localizationDelegates list, and looks up its messages with DemoLocalizations.of(context).

@escamoteur

This comment has been minimized.

Copy link
Contributor

@escamoteur escamoteur commented Sep 27, 2019

Have you seen this package https://github.com/timfreiheit/r_flutter?
Also just a thought wouldn't it even more comfortable if the input would be one cvs file with all languages in it that easy can be edited with any spread sheet application, so easy to give to translators.
Also have you seen https://marketplace.visualstudio.com/items?itemName=esskar.vscode-flutter-i18n-json

To make is really comfortable you should be able to directly create a new word entry from the IDE.

@alan-knight

This comment has been minimized.

Copy link
Contributor

@alan-knight alan-knight commented Sep 27, 2019

The target audience for this are people who doing their own translations by directly modifying the data files? I would have thought most people were using some sort of tool or translation service, but it seems not. Given that it seems reasonable enough. It seems important to consider migration issues when someone scales up to using a tool/service. I think they could probably still generate that way, but could this reasonably support multiple classes/multiple ARB files? Is that something people need/use in larger Flutter applications?

Intl doesn't currently handle formatting of parameters, so if you have something that takes a date or a number, you either have to format it before passing it in as a parameter or it will get the default format. So the rest of the app might still need to depend on intl for that.

Not directly relevant to this proposal, but Google Translator Toolkit, which is the primary thing that supported the ARB format is deprecated. We will really need to support at least one other format. There is a pending bug on intl_translation to make it so that anyone can write a reader/writer external to the package. There's also a converter someone wrote in JS, between ARB files and "Hierarchical JSON" format and it could probably handle others.

CSV files with Unicode in them makes me nervous of escaping issues, but there are a lot of formats. As above, being able to handle lots of them seems good, but we may want to recommend a default.

Does this/can this reasonably combine with the ability to use native translations on mobile platforms? So, would people be more comfortable with editing e.g. Android resource files directly?

There's been a bit of work ongoing on being able to read ICU MessageFormat strings at runtime and format them, as Closure does. That would be another alternative, although it's an entirely different (and much less typed) API so would be a significant change. And those would still have to be embedded in a file format.

@escamoteur

This comment has been minimized.

Copy link
Contributor

@escamoteur escamoteur commented Sep 27, 2019

@alan-knight what is the problem with CVS and Unicode?
I still think a format that can be read by office applications would make it much easier.

Another point not to forget is how can we support singular / plural versions of words in different languages?

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Sep 27, 2019

It seems important to consider migration issues when someone scales up to using a tool/service. I think they could probably still generate that way, but could this reasonably support multiple classes/multiple ARB files? Is that something people need/use in larger Flutter applications?

The tool could definitely support generating multiple classes for different sets of ARB files. That does seems like a nice kind of flexibility. I don't know if developers need that kind of support now, although I'm hoping we'll hear from them here.

Intl doesn't currently handle formatting of parameters, so if you have something that takes a date or a number, you either have to format it before passing it in as a parameter or it will get the default format. So the rest of the app might still need to depend on intl for that.

Yes, that's correct.

Not directly relevant to this proposal, but Google Translator Toolkit, which is the primary thing that supported the ARB format is deprecated. We will really need to support at least one other format. There is a pending bug on intl_translation to make it so that anyone can write a reader/writer external to the package. There's also a converter someone wrote in JS, between ARB files and "Hierarchical JSON" format and it could probably handle others.
...
Does this/can this reasonably combine with the ability to use native translations on mobile platforms? So, would people be more comfortable with editing e.g. Android resource files directly?

The ARB format isn't essential to this proposal, however it's what we've recommended that Flutter developers use to date. We could support native message catalog formats, per #37031, as well.

There's been a bit of work ongoing on being able to read ICU MessageFormat strings at runtime and format them, as Closure does. That would be another alternative, although it's an entirely different (and much less typed) API so would be a significant change. And those would still have to be embedded in a file format.

It would be nice to support a platform-independent and widely accepted standard. Assuming that the right API to expose to an app was still a set of methods that returned strings, supporting ICU (or ICU too) sounds like a win.

@Tomucha

This comment has been minimized.

Copy link

@Tomucha Tomucha commented Oct 2, 2019

Hi! I'm all for handwritten message files and code generation. The compile-time check of the message existence is great. But don't forget that occasionally you need to decide the message at runtime, maybe like this:

var vehicleTypeMessageKey = "VehicleType.${data['type']'}";
... so having just generated methods is not enough.

You will find a few more ideas like this in my internationalization package :-)

https://pub.dev/packages/i69n

@ashour

This comment has been minimized.

Copy link

@ashour ashour commented Oct 2, 2019

Hello, I hope you're all well. Thanks for your work on Flutter btw. It's an awesome piece of tech and I wish I had Flutter when I was co-founding a mobile app startup years ago. I was recently emailed by @HansMuller, who asked me to chime in. I research and write about i18n. I wrote an article on Flutter + Intl last year (I think that's how Hans found me).

I think this is a good proposal because the current Intl workflow is a bit clunky for smaller projects. This is especially true for projects that don't have automation and sophisticated tooling around i18n. Given this leaner alternative, I think two main pain points from a developer perspective warrant a look:

  • Reducing manual script running as much as possible
  • Using quick, lean file formats for messages that can be easily shared with translators

RE reducing manual script running, I've always wondered while using Flutter + Intl if there could be a watcher that generated and updated message files on the fly while developing. I've actually never seen this in any i18n library and I've always wanted it. I don't know how feasible this is given available Dart code analysis tools VS what would need to be written from scratch for such a watcher.

As for file formats, there are a few I think worth considering (not exclusively, of course, just ones that come to my mind):

  • ICU
  • PO
  • CSV
  • JSON
  • YAML

ICU and PO are widely used. PO has good adoption among i18n tools and platforms for developers & translators. For a smaller team, however, I think the overhead of generating and compiling ICU or PO is more of a hassle than it's worth. The other options are much more lean and to-the-point for smaller projects.

CSV is attractive for sharing with translators, although I can see it being a pain for developers. CSV headers can be far removed from their respective values, which can make managing messages tricky. Think of a plural message in Arabic with the language's 6 plural forms. How could a developer tell them apart? You could add an identifier to each message component, but that would take away from translators' experience.

JSON would be nice for us coders, but it may be a bit of a hassle for translators.

YAML could be a nice in-betweener for both programmers and translators. YAML has a nice, intuitive syntax with significant white space that I think would be picked up easily by translators. It also allows for various types and hierarchies, which make it flexible enough accommodate plural forms and more complex messages.

It could be worthwhile to pull in some translators that work in software and get their thoughts. They may well have their own bread-and-butter tools that, say, work well with JSON and have wide adoption. In that case JSON would be a very attractive option since we'd really only be solving the developers' problem, not the translators'.

These are my two cents so far. I hope they help a bit.

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 2, 2019

@Tomucha - Thanks for the feedback. I think we've taken a similar approach to simplifying building i69n applications :-).

But don't forget that occasionally you need to decide the message at runtime, maybe like this:

var vehicleTypeMessageKey = "VehicleType.${data['type']'}";
... so having just generated methods is not enough.

I agree that this sort of thing isn't necessary to solve with generated methods. I would think the app would have to include a method that switched on vehicleTypeMessageKey and returned a localized string.

String vehicleType(String vehicleTypeMessageKey) {
  final MyLocalizations localizations = MyLocalizations.of(context);
  switch (vehicleTypeMessageKey) {
    case 'sedan': return localizations.sedan();
    case 'truck': return localizations.truck();
    ...
  }
}
@Tomucha

This comment has been minimized.

Copy link

@Tomucha Tomucha commented Oct 2, 2019

At this point i69n generates the exact switch statement you are suggesting ;-)

@szotp

This comment has been minimized.

Copy link

@szotp szotp commented Oct 2, 2019

@ashour what's wrong with .arb files? Flutter already uses them, and they are esentially jsons.

EDIT: Also, the watcher you mentioned kinda exists already. Packages using code generation ( https://github.com/flutter/flutter/wiki/Code-generation-in-Flutter ) already auto update when app is running. The only problem is that analyzer does not seem to pick it up immediately and displays fake errors.

@HansMuller I agree that current state is a mess. Did you consider doing flutter generate builder package? I implemented this in r_flutter and it worked decently well (although IDE & analyzer are not really prepared to handle it). Such package could be eventually embraced by flutter team.

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 2, 2019

@ashour - Your article on internationalizing Flutter apps is an excellent introduction to this topic!

I think two main pain points from a developer perspective warrant a look:

  • Reducing manual script running as much as possible
  • Using quick, lean file formats for messages that can be easily shared with translators

RE reducing manual script running, I've always wondered while using Flutter + Intl if there could be a watcher that generated and updated message files on the fly while developing. I've actually never seen this in any i18n library and I've always wanted it. I don't know how feasible this is given available Dart code analysis tools VS what would need to be written from scratch for such a watcher.

That's an excellent point. We can definitely do this, even in the context-of hot-reload, in roughly the same way that Flutter handles asset changes.

As for file formats, there are a few I think worth considering (not exclusively, of course, just ones that come to my mind):

  • ICU
  • PO
  • CSV
  • JSON
  • YAML

Currently the "ARB" format, which is used by the Dart intl package and the proposed tool, is mostly just a JSON container for a collection of ICU messages. Additional support (beyond what's in the prototype) for dates, currencies, numbers, gender would move us from the realm of "modest" i18n Flutter apps, to more ambitious ones. I think that would be a good direction to take in the long run.

Support for other message catalog formats came up earlier. We could support other formats with additional code generators or perhaps with a scheme that converts other formats to a common one. XLIFF is another popular alternative.

These are my two cents so far. I hope they help a bit.

Thanks for your feedback!

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 2, 2019

@szotp - thanks for your feedback.

I agree that current state is a mess. Did you consider doing flutter generate builder package? I implemented this in r_flutter and it worked decently well (although IDE & analyzer are not really prepared to handle it). Such package could be eventually embraced by flutter team.

I'll take a look at r_flutter.

@escamoteur

This comment has been minimized.

Copy link
Contributor

@escamoteur escamoteur commented Oct 2, 2019

Looking back when I did.Net programming there was a powerful free Application to manage the resource files including text resources in different languages. That could also in and export different formats.

So the used format might not be so important for users if someone writes a flutter desktop app for this.

So maybe

@alan-knight

This comment has been minimized.

Copy link
Contributor

@alan-knight alan-knight commented Oct 2, 2019

Dart has a lot of things that rely on code generation and mechanisms for automatically re-generating on changes, e.g. build_runner. So I would expect automating that to be fairly easy. But I don't think anything is integrated with flutter right now, so I guess you would have to manually run build_runner on the side and have it watch the files.

@jonahwilliams

This comment has been minimized.

Copy link
Contributor

@jonahwilliams jonahwilliams commented Oct 2, 2019

I don't think we want to require a dependency on build_runner to generate translations.

@alan-knight

This comment has been minimized.

Copy link
Contributor

@alan-knight alan-knight commented Oct 2, 2019

No, build_runner is pretty heavy for that. But I don't think it would be difficult to do something custom that did file watching and triggered a re-generate, or to use other mechanisms.

@escamoteur

This comment has been minimized.

Copy link
Contributor

@escamoteur escamoteur commented Oct 2, 2019

Just one thing that came to my mind. If this gets sort of service think about adding an interface that can be easily used by the IDE extensions for VS code and IntelliJ.
With a function in the editor to insert a new identifier together with a default language that then creates an entry in the ARB(or whatever) files would make you really productive with internationalisation.

@bitsydarel

This comment has been minimized.

Copy link

@bitsydarel bitsydarel commented Oct 21, 2019

Reading the proposal i have few questions and propositions.

1- Why not sticking with arb and just make the files generation and intl initialization process almost inexistant?

Arb it’s used in many translation tools and it’s actually a popular format with many features.

Using csv, json, xml we will have to specify a way of defining plurals, gender and ect.

2- Keep the way users specify the properties to be translated (Custom Localization class with properties forwarding to intl.message).

3- Automate the creation of arb file template and the one for each languages that’s defined in the supportedLanguages property of the widgetsApp.

Keep the translations empty in other languages arb files, so the users can fill it.

4- generate the binding code to initialize intl.

5- generate the delegates that will load our localization class and initialize intl.

this allow us as developer to only create the localization class, the properties and all the rest is generated for us and no IDE plugin is required, so i can put it in a CI step.

Why not using build_runner ?

I can’t think of a real production app that does not use code generation... most of the libraries we daily use, require build_runner, example: json code generation, SQLite and more.

Using build_runner watch command, would allow us to interactively update the arb files while we are developing our app, which is amazing for programmer experience and better than native solutions.

Example this is all, I should have to specify

class GlobalAppLocalizations {

  static GlobalAppLocalizations of(final BuildContext context) {
    return Localizations.of<GlobalAppLocalizations>(
      context,
      GlobalAppLocalizations,
    );
  }

  String get applicationTitle {
    return Intl.message(
      "Flutter Localizations App",
      name: "applicationTitle",
      desc: "The title of the application",
    );
  }

  String get homeTitle {
    return Intl.message(
      "Flutter Localizations",
      name: "homeTitle",
      desc: "Home page title",
    );
  }

  String get homeDescription {
    return Intl.message(
      "This text is being translated in the current device language",
      name: "homeDescription",
      desc: "Home page description",
    );
  }

  String get submitButton {
    return Intl.message(
      "Submit",
      name: "submitButton",
      desc: "submit button text",
    );
  }

  String devExperience(final int years) {
    return Intl.plural(
      years,
      name: "devExperience",
      meaning:
          "how experienced the developer is based on how many years of experience he has",
      args: [years],
      zero: "$years of experience, you're still a debutant",
      one: "$years of experience, you're a junior dev",
      few: "$years of experience, you're a middle dev",
      many: "$years of experience, you're a senior dev",
      other: "$years of experience, your're a lead dev",
      examples: const {'years': 5},
    );
  }
}

And the rest would be generated for me.

@hpoul

This comment has been minimized.

Copy link

@hpoul hpoul commented Oct 21, 2019

I personally would vote for native support for PO files (generating to/from dart code), but I guess it would be quite trivial to have a json/po converter. I've got used to PO files from python/django and it seems it has a pretty wide range of tools available (e.g. poedit, transifex, etc.).

In my dream setup the source of truth for string keys would be the dart code. Similar to right now. So during development there is no need to update PO files immediately. Once code is done, extract to PO files, translate the PO files and then generate resulting dart code from those po files.
Imo even in trivial apps it would be very much helpful to allow easy separation into multiple classes. Having all app strings in one file/class can get pretty crowded pretty soon.

@bitsydarel

This comment has been minimized.

Copy link

@bitsydarel bitsydarel commented Oct 21, 2019

You can already use multiple localizations classes @hpoul

The framework only require you to provide delegates for them.

class DBAssistantLocalizations {
  /// List of locale supported by [DBAssistantLocalizations].
  static const List<Locale> supportedLanguages = <Locale>[
    Locale.fromSubtags(languageCode: "en"),
    Locale.fromSubtags(languageCode: "ru"),
    Locale.fromSubtags(languageCode: "uk")
  ];

  /// Get an instance of [DBAssistantLocalizations] from the current [context].
  static DBAssistantLocalizations of(final BuildContext context) =>
      Localizations.of<DBAssistantLocalizations>(
        context,
        DBAssistantLocalizations,
      );

  // ignore: public_member_api_docs
  String get appTitle => Intl.message(
        "DBAssistant",
        name: "app_title",
        desc: "the application title",
      );

  // ignore: public_member_api_docs
  String get profile => Intl.message(
        "Profile",
        name: "profile",
        desc: "Represent user profile",
      );

  // ignore: public_member_api_docs
  String get high => Intl.message(
        "High",
        name: "high",
        desc: "adjective especially of things that are not living",
      );

  // ignore: public_member_api_docs
  String get medium => Intl.message(
        "Medium",
        name: "medium",
        desc: "being in the middle between an upper"
            " and lower amount, size, degree, or value",
      );

  // ignore: public_member_api_docs
  String get low => Intl.message(
        "Low",
        name: "low",
        desc: "adjective below the usual level",
      );

  // ignore: public_member_api_docs
  String get lists => Intl.message(
        "Lists",
        name: "lists",
        desc: "feature of the app",
      );

  // ignore: public_member_api_docs
  String get news => Intl.message(
        "News",
        name: "news",
        desc: "feature of the app",
      );

  // ignore: public_member_api_docs
  String get tv => Intl.message(
        "Tv",
        name: "tv",
        desc: "feature of the app",
      );

  // ignore: public_member_api_docs
  String get couldNotRetrieveWeatherInfo => Intl.message(
        "Could not retrieve weather information",
        name: "couldNotRetrieveWeatherInfo",
        desc: "Message that for some reason weather date could not be retrieve",
      );

  // ignore: public_member_api_docs
  String get noShoppingLists => Intl.message(
        "You currently don't have any shopping lists",
        name: "no_shopping_lists",
        desc: "The message for when the user does not have any shopping lists",
      );
}
class DBAssistantDateTimeLocalizations {

  /// Get an instance of [DBAssistantDateTimeLocalizations]
  /// from the current [context].
  static DBAssistantDateTimeLocalizations of(final BuildContext context) {
    return Localizations.of<DBAssistantDateTimeLocalizations>(
      context,
      DBAssistantDateTimeLocalizations,
    );
  }

  // ignore: public_member_api_docs
  String years(int years) => Intl.plural(
        years,
        one: "$years year",
        other: "$years years",
        name: "formatted_years",
        desc: "year formatter base on the amount of years",
      );

  // ignore: public_member_api_docs
  String months(int months) => Intl.plural(
        months,
        one: "$months month",
        other: "$months months",
        name: "formatted_months",
        desc: "month formatter base on the amount of months",
      );

  // ignore: public_member_api_docs
  String days(int days) => Intl.plural(
        days,
        one: "$days day",
        other: "$days days",
        name: "formatted_days",
        desc: "day formatter base on the amount of days",
      );

  // ignore: public_member_api_docs
  String hours(int hours) => Intl.plural(
        hours,
        one: "$hours hour",
        other: "$hours hours",
        name: "formatted_hours",
        desc: "hour formatter based on the amount of hours",
      );

  // ignore: public_member_api_docs
  String minutes(int minutes) => Intl.plural(
        minutes,
        one: "$minutes minute",
        other: "$minutes minutes",
        name: "formatted_minutes",
        desc: "minute formatter based on the amount of minutes",
      );

  // ignore: public_member_api_docs
  String get fewMoments => Intl.message(
        "few moments",
        name: "few_moments",
        desc: "refer the time elapsed since an event",
      );

  // ignore: public_member_api_docs
  String getLastChanges(final String elapsedTime) => Intl.message(
        "Last changes: $elapsedTime ago",
        name: "last_changes_formatted",
        desc: "Display the last time a change occured on a object",
        args: [elapsedTime],
        examples: const {"elapsedTime": "2h"},
      );
}
@bitsydarel

This comment has been minimized.

Copy link

@bitsydarel bitsydarel commented Oct 21, 2019

And ARB has many features that PO does not support.

@hpoul

This comment has been minimized.

Copy link

@hpoul hpoul commented Oct 21, 2019

@bitsydarel

You can already use multiple localizations classes

I wasn't disputing that, just wanted to raise that issue because the current proposal mentions:

A new tool will be used to generate a single class that contains one method per message.

not sure how much emphasis was on that "single class"..

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 21, 2019

@bitsydarel per #41437 (comment)

1- Why not sticking with arb and just make the files generation and intl initialization process almost inexistant?

Although, in the fullness of time, we hope to provide support for other formats, initially we'll support the arb format.

The "Proposal" section of this issue begins with "The proposed approach is to start with handwritten ".arb" files."

2- Keep the way users specify the properties to be translated (Custom Localization class with properties forwarding to intl.message).

That is similar to what's being proposed here. However in this case the custom localization class is generated rather than hand written. See for example, the code generated by the prototype is based on the sample .arb file.

3- Automate the creation of arb file template and the one for each languages that’s defined in the supportedLanguages property of the widgetsApp.

In this proposal the template arb file is generated by hand. With some IDE integration, we could simplify creating it. Simiarly, we could generate copies of the template arb file, per supported locale.

4- generate the binding code to initialize intl.
5- generate the delegates that will load our localization class and initialize intl.

We are in fact proposing to do this and to do so automatically, when the .arb files change.

I think the main difference between what you're suggesting and what has been proposed is the starting point. We're proposing to start with .arb files, since they're the source of localizations, and generate the rest.

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 21, 2019

@hpoul per #41437 (comment)

I personally would vote for native support for PO files (generating to/from dart code)...

In the fullness of time, we hope to provide support for other formats, initially we'll support the arb format.

Imo even in trivial apps it would be very much helpful to allow easy separation into multiple classes. Having all app strings in one file/class can get pretty crowded pretty soon.

Making it possble to generate more than one localization class sounds like a good idea.

@bitsydarel

This comment has been minimized.

Copy link

@bitsydarel bitsydarel commented Oct 21, 2019

@HansMuller Ho so then i did not fully understand the proposal and I’m happy all the other suggestions are already covered by the initial proposal.

About the starting point, If we have to start by creating an initial arb file this require knowledge of the arb specifications and format.

Which is definetly overwhelming for beginners, people who would like to quickly integrate localization for their apps and someone coming from another platform.

That's why i suggested that it's should be generated based on the content of the custom localization class because this is just dart code that the programmer is already familiar with.

So me as a programmer i just have to worry about writing my custom localization class.

Now for the copies of the template arb for each supported locale, since they would already be generated and pre-filled.
I would just have to fill the translated value i have gotten from a trusted source example: Google translate, translations team, ect.

@bitsydarel

This comment has been minimized.

Copy link

@bitsydarel bitsydarel commented Oct 21, 2019

@HansMuller If in the futur you plan on adding support for other formats, the suggested flow would require less work on the developer side because the only changes would be the type of files generated and how the IDE plugin or generator will handle the binding.

@bitsydarel

This comment has been minimized.

Copy link

@bitsydarel bitsydarel commented Oct 21, 2019

@HansMuller Now that i think of it, the only issue i see with my suggestion is if let say we add support for PO files or any other format, if the developer wrote the custom localization class property using Intl.message, Intl,plural, ect. Can we still do the binding (Intl initialization) properly ? Or is Intl depending on ARB ?

If Intl is depending on ARB then the current proposal is better because the current proposal it does not matter which file format we use, at the end we only need that the custom localization class is properly generated and with proper bindings or initialization so that we could do:

MyLocalizations.of(context).myMessage()
@marcglasberg

This comment has been minimized.

Copy link

@marcglasberg marcglasberg commented Oct 23, 2019

This issue was shown to me by @escamoteur.

I'd like to share with you my approach to translations. I just published it yesterday. It uses the new Dart extensions: https://pub.dev/packages/i18n_extension

Suppose you have this: Text("Hello") and you want to translate it to the current locale. You just need to add .i18n to the string, like this: Text("Hello".i18n).

This would display "Hola" is the locale is Spanish, "Olá" if it's Portuguese, and so on. You can also have plurals, using the .number() modifier. For example Text("There is %d dog".number(2)) will display "There are 2 dogs". And you can create your own custom modifiers, for things like genders etc.

It's specially easy to use for small projects, where you are doing your own translations.

But it also works well for large projects (like the one I'm involved in, which will use this).

In more detail, you have a dart file that creates a Translation object, and this object is optimized for easily creating translations by hand. But it creates them from maps. So if you can create maps from some file you can use that file. In other words, it's very easy to import AND export from/to .json,, .arb etc.

Also, there is no need for build_value.

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 25, 2019

@bitsydarel per #41437 (comment)

... the only issue i see with my suggestion is if let say we add support for PO files or any other format, if the developer wrote the custom localization class property using Intl.message, Intl,plural, ect. Can we still do the binding (Intl initialization) properly ? Or is Intl depending on ARB ?

The Dart intl package depends on the ARB format. That's why this proposal is based on ARB, at least by default. We could support other formats, possibly by converting them to ARB. In that case ARB would just be an intermediate format.

If Intl is depending on ARB then the current proposal is better because the current proposal it does not matter which file format we use, at the end we only need that the custom localization class is properly generated and with proper bindings or initialization so that we could do:

MyLocalizations.of(context).myMessage()

It's true that we could support other formats directly, by generating a MyLocalizations class that did the work for plurals, genders, numbers, currencies, dates/times, etc that the Dart intl package does. That is in fact what the flutter_localizations package does for the Flutter SDK.

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 25, 2019

@marcglasberg per #41437 (comment)

Using static extension methods for simple message translation is a nice idea and it does look pretty slick. Personally, I'm a bit leery of static extensions methods in libraries because, with heavy use, they can make code difficult to read. Glad to hear that it's working for your project!

@marcglasberg

This comment has been minimized.

Copy link

@marcglasberg marcglasberg commented Oct 25, 2019

@HansMuller Instead of your Localization class having ARB as an intermediate format, it could instead have some simple map based format in memory. Then you could define an Importer interface, and anyone could then create importers to any formats. ArbImporter could be the first importer.

This would make it easy to create importers, so that people could choose their own formats, or even publish packages with importers.

Also that would allow for any other internalization libraries like my own to actually use these same importers. For example, in my lib I would call the same ArbImporter to create the in memory format, and then it's easy for me to convert from memory to my own format.

In other words, these importers would become reusable code for other internalization libraries.

@HansMuller

This comment has been minimized.

Copy link
Contributor Author

@HansMuller HansMuller commented Oct 31, 2019

@marcglasberg per #41437 (comment)

Instead of your Localization class having ARB as an intermediate format, it could instead have some simple map based format in memory.

I agree, that would be much better.

@shihaohong shihaohong mentioned this issue Oct 31, 2019
18 of 19 tasks complete
@shihaohong

This comment has been minimized.

Copy link
Contributor

@shihaohong shihaohong commented Nov 4, 2019

Hi all, I've created the tool with @HansMuller's original proposal. All feedback and ideas are welcome. You can also comment on the PR #43934 if there are any questions or concerns.

Tool Functionality

Starting with arb files in lib/l10n, the tool creates the following by default:

  1. An AppLocalizations class containing:
  • an AppLocalizationsDelegate
  • a supportedLocales property that returns a list of all supported locales.
  • a localizationsDelegate property that returns a list containing this delegate along with GlobalMaterialLocalizations.delegate, GlobalCupertinoLocalizations.delegate, and GlobalWidgetsLocalizations.delegate.
  1. One Dart message_<locale>.dart file for each arb file, as well as a messages_all.dart file that performs the locale message lookup. The tool infers the locale of each arb file from the @@Locale property or the arb file's name.
    All feedback and ideas are welcome. You can also comment on the PR #43934 if there are any questions or concerns.
    Once the tool is done running, all you would to do is modify the MaterialApp.localizationsDelegates and MaterialApp.supportedLocales properties.

Tool Demo

  1. Go to https://github.com/shihaohong/i18n_exploration and clone down the repo.
  2. From the project root directory, run:
dart ${FLUTTER_DIR}/dev/tools/localization/gen_l10n.dart
  1. Note that the above files have been generated in the ${PROJECT_REPO}/lib/l10n folder.
  2. The modifications to the app to accept localizations have already been added, but typically, you'd have to make the following changed to your app:
import 'l10n/app_localizations.dart'; // add this line
// ...
class I18NDemoApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      localizationsDelegates: AppLocalizations.localizationsDelegates, // add this line
      supportedLocales: AppLocalizations.supportedLocales, // add this line
      home: Home(),
    );
  }
}
  1. You should then be able to access localized messages depending on the locale of your simulator or physical device:
class Home extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Column(
          mainAxisSize: MainAxisSize.min,
          children: <Widget>[
            Text(AppLocalizations.of(context).helloWorld),
            Text(AppLocalizations.of(context).hello('International World')),
            Text(AppLocalizations.of(context).greeting('Greetings', 'Earthlings')),
            Text(AppLocalizations.of(context).helloWorlds(100)),
          ],
        ),
      ),
    );
  }
}

Tool Options

There are a couple of modifiable options for the tool:

  1. --arb-dir: the path to your l10n directory. The tool defaults to /lib/l10n/.
  2. --template-arb-file: the arb file to use as a template to generate these files. The tool defaults to app_en.arb.
  3. --output-localization-file: The filename of the Dart file containing your localization and localizations delegate classes. The tool defaults to app_localizations.dart.
  4. --output-class: The Dart class name to use for your localization class. The tool defaults to AppLocalizations.
  5. --preferred-supported-locales: Takes a list of locales that is preferred for the Flutter application. For example, calling the tool with --preferred-supported-locales=["en_US"] would place that at the top of the MaterialApp.supportedLocales list and have the app default to American English if a device supports it.
@shihaohong

This comment has been minimized.

Copy link
Contributor

@shihaohong shihaohong commented Dec 13, 2019

I've added proposals to a new GitHub issue to how we could handle DateTime, Numbers and Currencies to the tool.

Please take a look and provide any feedback, suggestions, or nice-to-haves you'd like to see if you have any :)

@goranluledzija

This comment has been minimized.

Copy link

@goranluledzija goranluledzija commented Dec 28, 2019

We published Flutter Intl extension for VS Code that watches ARB files and generates localization code automatically.

So after adding string keys to ARB files, key getters are immediately available for autocomplete in Dart code.

Would like to hear your feedback and suggestions. :-)

@shihaohong

This comment has been minimized.

Copy link
Contributor

@shihaohong shihaohong commented Jan 2, 2020

I just added an enhancement to the tool to support a preferred locales option. By default, the supported locale list is sorted by alphabetical order. This option allows developer to set a prioritized list of locales to be placed at the beginning of the list over the rest of the sorted locales.

I added my original comment with this functionality as well

@mehmetf

This comment has been minimized.

Copy link
Contributor

@mehmetf mehmetf commented Jan 15, 2020

  • Have you looked into how this proposal will work in google3?
  • Does this proposal take into account that we will make embedder be more responsible about locale selection? #47087
@jmagman jmagman added this to Awaiting Triage in Mobile - localization & intl Feb 25, 2020
@GaryQian GaryQian moved this from Awaiting Triage to Engineer Reviewed in Mobile - localization & intl Mar 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Mobile - localization & intl
  
Engineer Reviewed
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.