Skip to content

L10nConfigPaths file format

Jump to bottom
Eemeli Aro edited this page Feb 14, 2025 · 2 revisions

The configuration files are in TOML format. Below, we document the semantics of a well-formed TOML file, and how they are validated.

Paths can include environment references using the syntax {reference}. Undefined references are replaced with an empty string.

The moz.l10n implementation of this format currently has the following differences to what is documented here:

  • [build] is ignored
  • [[excludes]] are not supported
  • [[filters]] are ignored

Top-level Elements

  • basepath: Any relative path in the configuration is resolved against this path.

  • locales: List of locale codes to enable for the described project. This is a list of BCP 47 locale identifiers, with the notable exception of ja-JP-mac, which we temporarily need to keep for compatibility reasons.

  • env: Table of environment aliases, which allows to define short-hands to be used in the configuration file. Example:

    [env]
    l = "{locale}/"

  • build: Table of build configuration entries. This is a good location to store locale lists for special builds. Example:

    [build]
    exclude-multi-locale = [
        "bn-BD",
        "zam",
    ]
    test-single-locale = [
        "it",
        "he",
        "zh-TW",
    ]

    This could for example mean that from all the locales that are in a project (think Firefox for Android), some are excluded from a multi-locale build and only part of the single-locale builds. In addition, we could have a short list of locales that we always want to generate single-locale builds for to make sure they don't break? The builds to actually use and test would be the multi-locale build, though.

    The entries in build are project specific.

Includes

The l10n configuration files can include external configuration files to make the whole configuration system modular. This is done by adding includes tables to the file.

[[includes]]
    path = "some/file/path.toml"

The paths are resolved like this (filedir being the path to the directory of the configuration file): filedir/{basepath}/path.

The global localization scope is defined by a root configuration file, which utilizes includes to define projects. Projects themselves can refer to further configuration files, to reuse the l10n configuration of shared code fragments.

Excludes

Top-level configuration files can also exclude an external configuration file. This can be used to define localization projects where one excludes the other. When excluding a configuration with includes, all child configurations of that are also excluded.

It's an error to use an exclude in a child configuration.

If an included configuration (or the top-level configuration) explicitly includes an excluded sub-configuration file, the file is included, i.e., includes have precedence over excludes.

Paths

This is the heart of the actual configuration, describing where to find the reference files, and where to put the localized files. In defining the localization paths, use the {locale} variable to denote the locale code. Use * to denote a single level wildcard, or ** to denote all files in the given directory hierarchy.

Example:

[[paths]]
    reference = "locales/en-US/my-file.ftl"
    l10n = "locales/{locale}/my-file.ftl"

Both reference and l10n paths are required.

To limit a particular path pattern to a subset of locales, you can specify an optional locales entry:

[[paths]]
    reference = "locales/en-US/my-file.ftl"
    l10n = "locales/{locale}/my-file.ftl"
    locales = [
        "de",
        "fr",
    ]

The list of locales for a particular path is assumed to be a subset of the top-level locales entry. Additional locales will be ignored. If a path matches multiple [[paths]] entries, the locales of the last match will override earlier ones.

Filters

In some cases localizations can differ from the reference locale. For example it can be fine for some files to be missing, for particular entries to be missing in a file, or to have extra entries. Filters allow the configuration to specify what's acceptable. The filter table entries are defined as follows

  • path: A full file path, or a list of file paths. Wildcards (*, **) and expansions are allowed.

  • key: A key name, or list of key names that are ignored. If you want to specify a set of keys, you can use regular expressions by prefixing the key entry with re:. This entry is optional and, if missing, the filter will only apply to missing or additional files in the localization. If you want to ignore all entries in a file, specify a generic key, re:..

  • action: What to do when the filter matches. The value should be one of

    • error: Report the missing or obsolete string or file. This is the default behavior, but it can be useful to make decision on particular files or keys before taking more generic actions.

    • ignore: Don't report the missing or obsolete string or file.

    • warning: Report the missing string or file but, for example, don't take action when doing l10n-merge. This used to be called report in filter.py.

For any particular project, a string or a file of a localization has a status within that particular project, and a status within the global configuration.

As the global l10n scope and the projects scopes are defined by aggregating individual configuration files, the status of a string or file is also determined by how the statuses of the string from each configuration aggregate.

Evaluation

In a single configuration file, the filter is evaluated as the action of the first rule with a matching path and key pattern. Both path and key can be arrays, which is just a shorthand for a filter rule for each combination.

[[filters]]
    path = [
        "browser",
        "toolkit",
    ]
    action = "ignore"

is a shorthand for

[[filters]]
    path = "browser"
    action = "ignore"

[[filters]]
    path = "toolkit"
    action = "ignore"

If none of the path and key combinations match, there is no resulting action.

The status of a file or string in a project is ignore if any filter returns ignore.

The final status of a file or string is ignore if all project filters return ignore. If all project filters return ignore or warning, the status is warning, otherwise error. Thus, a string or file that's not covered by any filter has an error status.

Here's pseudo code in Python that demonstrates the algorithm.

def filter(config, path, key=None):
    """Determine the status of a file or string
    in a configuration file.
    """
    for rule in config.filter_rules:
        if not rule.path.matches(path):
            continue
        if key is None and rule.key is None:
            return rule.action
        if key is not None and rule.key is not None:
            if rule.key.matches(key):
                return rule.action

def project_status(project, path, key=None, global=True):
    """Determine the status of a file or string
    in a project with a set of configuration files.
    """
    default_action = 'error' if global else 'ignore'
    return aggregate(
        (filter(config, path, key) for config in project.configs),
        'ignore', default=default_action)

def global_status(global_scope, path, key=None):
    """Determine the status of a file or string
    in the global context.
    """
    return aggregate(
        (project_status(project, path, key) for project in global_scope.projects),
        'error', default='error')

def aggregate(actions_iterable, stop_action, default=None):
    """Helper function to aggregate actions over an
    iterable.
    """
    actions = set()
    for action in actions_iterable:
        if action == stop_action:
            return stop_action
        actions.add(action)
    for aggregated_action in ('error', 'warning', 'ignore'):
        if aggregated_action in actions:
            return aggregated_action
    return default

This evaluation of filters builds a system that allows for strings and files to be ignored for a particular project, but if any project needs the string, it needs to be translated. At the same time, files or strings not used anywhere are reported as obsolete, and can be removed.

Meta Data

Meta data for a project is defined in the configuration repository only, and maintained by l10n-drivers. It's used to describe project management-related data about a project, like priorities, schedules, descriptions. Meta data can also be restricted to particular files or locales.

Example:

[[project]]
    name = "Firefox"
    description = """The mozilla Desktop User Agent"""
    [[project.import]]
        repository = "https://hg.mozilla.org/mozilla-central"
        revision = "default"
        path = "browser/locales/l10n.toml"
        vcs = "mercurial"

    [[project.metadata]]

        priority = 2

        flags = [
        ]

        [[project.metadata.schedule]]
            when = 2017-02-29T00:00:00Z
            title = "Code freeze"

        [[project.metadata.schedule]]
            when = 2017-03-29T00:00:00Z
            title = "Go-Live to production"
            notify = 2017-03-22T00:00:00Z

    [[project.metadata]]

        locales = [
            "de",
        ]
        priority = 1

    [[project.metadata]]

        locales = [
            "fr",
        ]
        path = "{l}browser/some/directory/file.properties"
        priority = 1
Clone this wiki locally