annotation: Multilingual meta data

[handrews]

This was originally proposed on the old wiki at https://github.com/json-schema/json-schema/wiki/multilingual-meta-data-(v5-proposal) by @geraintluff with further contributions from @brettz9 and @sonnyp

The translation alternative discussed at the end of this comment was originally proposed at https://github.com/json-schema/json-schema/wiki/translations-(v5-proposal) by @geraintluff based on an email thread with @fge.

Proposed keywords

This proposal modifies the existing properties:

• title
• description

This proposal would also apply to the named enumerations proposed in issue #57 , if that makes it in.

Purpose

This modification would allow inclusions of multiple translated values for the specified properties.

Currently, schemas can only specify meta-data in one language at a time. Different localisations may be requested by the client using the HTTP Accept-Language header, but that requires multiple (largely redundant) requests to get multiple localisations, and is only available over HTTP (not when pre-loading schemas, for instance).

Values

In addition to the current string values (which are presumed to be in the language of the document), the values of these keywords may be an object.

The keys of such an object should be IETF Language Tags, and the values must be strings.

Behaviour

When the value of the keyword is an object, the most appropriate language tag should be selected by the client, and the string value used as the value of the keyword.

Example

{
    "title": {
        "en": "Example schema",
        "de": "..."
    }
}

Concerns

Schemas with many languages could end up quite bulky.

In fact, the Accept-Language option is in many ways more elegant, as the majority of the time only one language will be used by the client (and the other localisations will simply be noise). However, this option is not available in all situations. One might also avoid the extra bulk by using JSON references (and thereby also enable localisation files to contain all translatable text).

An alternative approach to the above would be to reserve localeKey as a property for any schema object or sub-object and localization-strings as a top-level property:

{
    "localization-strings": {
        "en": {
            "example": {
                "title": "Example schema",
                "description": "Example schema description"
            }
        },
        "de": {
            "example": {}
        }
    },
    "type": "object",
    "localeKey": "example"
}

The advantage to this approach would be that, as typically occurs with locale files (for reasons of convenience in independent editing by different translators), all language strings could be stored together. Thus, if leveraging JSON references, it would be a simple matter of:

{
    "localization-strings": {
        "en": {
            "$ref": "locale_en-US.json"
        },
        "de": {
            "$ref": "locale_de.json"
        }
    },
    "type": "object",
    "localeKey": "example"
}

or yet simpler:

{
    "localization-strings": {"$ref": "locales.json"},
    "type": "object",
    "localeKey": "example"
}

Alternative: translation objects

This alterantive proposes a translations keyword which would be alongside the title and `descript

translation object Values

The value of translations would be an object. The values would be a JSON schema meta keyword and would themselves be objects,
where each property key MUST be in accordance with RFC 3066

Example translation object

When translating title and description, you can easily write an object where the meta keywords are RFC3066 conformal:

{
    "title": "postal code",
    "description": "A postal code.",
    "translations": { 
        "title": { 
            "en-GB": "postcode",
            "en-US": "zip code",
            "de": "Postleitzahl",
            "fr": "code postal"
        },
        "description": {
            "en-GB": "A Royal Mail postcode.",
            "en-US": "An USPS ZIP code.",
            //  ...
        }
    }
    //  ...
}

Translation object concerns: where to apply?

"What would be left to specify is of course what "relevant" is here.
Apart from "title", there is "description". But I don't think we want any other keyword to be affected."

[handrews]

My personal preference would be to separate I18N/L10N from the JSON Schema specification. A companion specification could describe how to build locale files by associating the file with the URI of the schema being localized, and associating each set of localized strings with a point in that schema through JSON pointer. The JSON pointer would simply be appended to the schema URI as a fragment, as is done for all other uses of JSON pointer to reference internal contents of the schema.

This would be as powerful as the proposal above but would not burden JSON Schema at all. It would also easily allow alternate approaches to translations, which may be preferable in some implementation environments.

[akuckartz]

Please try to avoid reinventing wheels which are already specified in JSON-LD.

[awwright]

@akuckartz I really like JSON-LD, can you point to some specific examples/alternatives?

[akuckartz]

@awwright See https://www.w3.org/TR/json-ld/#string-internationalization and https://www.w3.org/TR/activitystreams-core/#naturalLanguageValues

[handrews]

@akuckartz Ad noted, I'd prefer not to solve this within JSON Schema at all. If JSON-LD provides a way to do this orthogonal to JSON Schema, I'm all for that.

[brettz9]

JSON-LD appears to have a robust mechanism for i18n, and it can nicely be reused for non-schema JSON as well, but it lacks the feature mentioned in the OP of having a localeKey to allow a standardized means of referencing keys within simple key-value, single-language modular JSON locale documents (e.g., flat files) without any proprietary (or necessarily server-side) means of substitution, embedding, or transformation.

JSON-LD also lacks any means to plug in a variable default locale. One fairly clean, intelligible way to add this support is to leverage the reserved tildes of JSON Reference for variable substitutions, in our case for adding in a locale (e.g., as mentioned at whitlockjc/json-refs#47 (comment) and subsequently potentially supported through whitlockjc/json-refs#54 )

So, using both JSON-LD and JSON Reference (and @handrews suggestion for JSON pointer (or JSON reference)), we could do:

{
    "@context": {"@language": {"$ref": "~$locale~"}},
    "localization-strings": {"$ref": "locale-~$locale~.json"},
    "type": "object",
    "title": {"$ref": "#localization-strings/myTitle"},
    "description": {"$ref": "#localization-strings/myDescription"}
}

~$locale~ could be substituted with the language code from an Accept-Language header or as otherwise determined by the application.

This format is superior to using JSON-LD alone in that it allows us to maintain locale files independent for each language (as is convenient when having multiple translators) like locale-en-US.json with keys like:

{
     "myTitle": "My title",
     "myDescription": "My description"
}

The aforementioned localeKey was more succinct than using JSON references as we do above, but this JSON References approach at least uses the draft JSON Reference spec rather than a proprietary property; though the variable substitutions we use within the JSON reference are non-standard, they at least make clear that pre-processing is required given their use of reserved tildes (with a 0 or 1 following) precludes them from being valid JSON references.

[handrews]

Note that I just edited the initial comment to include the translation alternative, as I've generally been consolidating proposals targeting the same problem into a single issue for discussion.

[brettz9]

Not a fan of the translations approach (at least alone), as it, as with JSON-LD, doesn't allow for separate language files.

Regarding the question in the OP regarding other candidates for i18n besides title and description, some new fields enumTitles would be very useful (and maybe enumDescriptions for parity), so that, for example, items in a pull-down could be made human-readable, and in an i18n-izable fashion (I think this proposal was brought up somewhere else already).

[handrews]

@brettz9 for enumTitles and enumDescriptions see issue #57 (named enumerations), particularly the surprisingly concise oneOf-based solution that @awwright proposes later in the comments. It would reduce the problem back to the regular title and description.

[handrews]

Regarding the question of external files, that's a big part of why I don't think that JSON Schema should be in the business of specifying I18N/L10N at all. I18N/L10N is an issue for JSON-formatted data independent of JSON Schema.

It should be solved independently, as well. It may be solved in part or entirely by JSON-LD, but whatever is not covered there should not be pulled into JSON Schema. I do support using IRIs instead of URIs as appropriate to enable I18N ( issue #59 ), but that is the extent to which I think JSON Schema should concern itself with this sort of thing.

I18N/L10N is a critically important topic, and deserves better than the be shoehorned into a specification that is primarily focused on very different things.

[handrews]

@brettz9 are you OK with folding this topic from https://github.com/json-schema/json-schema/wiki/Content-language-proposal into this issue?

As with HTML's lang/xml:lang properties, it would be useful to indicate the content language of a particular field in a standard manner. This might be used for proper font display (as in CJK languages) or for selectively showing content to users based on their locale.

I think a name "contentLang" for the property would avoid confusion at (falsely) thinking that this was necessarily indicating the language of the "title" or "description" itself.

I think this similarly lies outside the scope of JSON Schema, both because of my opinion that I18N should be handled separately and because this particular part is primarily geared towards displaying in a UI, which is not what JSON Schema is trying to solve ( see issue #55 for a discussion of such principles).

For now I'm going to point readers of that wiki page here, but I'd be happy to file a separate issue (or you are welcome to yourself- please prefix it with "v6 annotation:" if you do.

[awwright]

@handrews I don't have any use for the naming scheme myself, we've got milestones and tags if we need to distinguish between different types of issues, and we can filter by those to boot.

[handrews]

@awwright well yeah, but I can't apply tags or milestones so the naming scheme help me :-)
If you want to go through and tag them as valdiation/annotation/hyper-schema and set the v5 and v6 milestones I will be happy to change the titles and drop the prefixes. I just figured I was bugging you about enough things already.

[brettz9]

@handrews : To me, although it is probably not a feature that would be used for validation (since looking at code points might not be reliable or valid such such detection), contentLang does describe the data, placing constraints on how it is to be understood which brings it more into the world of schemas, as I see it (even more so than the i18n of "title", etc. which doesn't pertain to the data). And this is not something that can be solved by JSON-LD unless one uses JSON-LD in the instance documents (which defeats the purpose of being able to add this kind of pseudo-constraint at the schema level).

[handrews]

@brettz9 Oh I see what you mean, I think I misinterpreted it. Yeah, I should file that separately as it is about data and not meta-data. This issue should just be about the meta-data.