Enable translation workflow for a "hamonization" quality gate

[rhofer]

Weblate integration into SW project(s)

Desired translation process

Template (or SWEnglish or Developer-English)

As shown in the sketches, template has the following semantic.

Owner

• Developer

Language

• It is a en
• Written by developer (ideally together with UXer / UX writing), hence also provides developer changes (terms, semantic)
• Can have typos, may does not yet comply with domain specific terms
• Quality: as good as possible, but can always be reasonable low.

Technically

• Provides all valid and used keys in SW (component), hence also provides developer changes in context
• Can be file in repo or is a build artifact (generated)
• Can be any format, as digestible by weblate (json, resx, android, ...)
• Base input for Weblate
• Editable in Weblate: NO
  • Since provisioning as build artifact is possible, editing of template in Weblate makes no sense

English en

English is created in the first step in Weblate, based on template

Owner

• Translator in step Harmonize

Language

• It is proper en
• Written by translator, hence also provides harmonization changes (terms, semantic)
• Encounters for domain specific terms (align, harmonize)
• Semantic in close collaboration of SW Project team
• Quality: Is used as polished and clean base for any further translations. The higher the quality the easier to translate.

Technically

• Is the key provider for any further subsequent languages, but always based on template
• Is of same format as according template
• Editable in Weblate: YES

Any further language

These again are created based on English en.

Owner

• Translator in step Translate

Language

• Any further language
• Written by translator, based on en

Technically

• Is based on the keys provided with en
• Is of same format as according template (and en)
• Editable in Weblate: YES

Requirements

• Developer shall be able to act independently from translation (Process: template approach)
• Developer input shall pass a quality gate first and not being directly exposed to translations (Process: harmonize approach)
• Multi SW project and multi component from same domain shall be aligned in terminology (Process: harmonize approach)
• Avoid merge conflicts (template one way only to translation, results one way only back to SW project)

In order to achieve this process, the following is required:

• Template becomes the source for en in the Harmonize step
• en becomes the source for any further language in Translate step
• Strings in template shall not enrich TM
• Template based creation of en shall be supported by built in MT

Work-around's

Create template as an additional language

Create two components, both pointing at the same SW component with:

• ComponentA in Project1 with source language template >> used for harmonization
• ComponentA' in Project2 with source language en >> used for translation into further languages

Cons:

• Suggest and comment: where to do this for en since the exact same appears in ComponentA and ComponentA' ?
• template was needed to be created as natural language in the list of languages, but actually it isn't, really. Based on project settings it is developer kind of en

Solution Approaches

• Provide secondary source language: Provide a secondary source language in case "Monolingual base language file" is marked as editable #3257
• Extend usage of template and Translation States: Extend the usage of "Template for new translations:" and according Translation States #3273

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been put aside. Currently, it is unclear whether it will be ever implemented as it seems to cover too narrow use case or doesn't seem to fit into Weblate. Please try to clarify the use case or consider proposing something more generic to make it useful to more users.

[nijel]

We certainly need something like this. It just needs some time to properly evaluate this to make it fit into all workflows.

I'd love to see this for Weblate translations as well, but the secondary source approach is not really usable with bilingual translation files.

[rhofer]

If I understood correctly the translation workflow today (as-is) is working as depicted in the following:

Following the gettext documentation , literally the following statements can be found in overview:

• ... The letter t in .pot marks this as a Template PO file, not yet oriented towards any particular language ...
• ... The first time through, there is no lang.po yet, so the msgmerge step may be skipped and replaced by a mere copy of package.pot to lang.po, where lang represents the target language ...
• ... package.pot file is a created file by xgettext ...

From these statements I derive that

• the pot file is not necessarely a language and this is only a specific implementation in weblate that when project is set to en that in bi-lingual case the pot is assumed to be en
• the lang.po could also be lang=en means en.po also in bi-lingual case. This currently weblate doesn't allow in bi-lingual case when project is set to en and an according .pot is provided. (but it still allows sub-languages as en_US or en_UK etc.)
• package.pot is created, following the rules of not checking in build artifacts also this shouldn't.

From this point of view the overall workflow could be (to-be) as following:

• introducing new options (blue) on component configuration level. If these options are set to false it shall behave as-is.
• blue written statements in data and UI layer are different from as-is (above)

With this approach:

• There is no need to define an artificial source language as template, neither a secondary source language needs to be indicated. All is derived by project settings (e.g. en).
• Most of the optimizations would be in UI layer and not on data.
• Introduction and treatment of SWEnglish would be consistently based on TEMPLATE for all kind of formats
• bi-lingual and mono-lingual cases need to be distinguished and treated differently but with a very minor variability / divergence

@nijel If this would be kind of correct and applicable thinking, we'd also need to analyse towards further bi-lingual formats. (Translation state dependencies are not shown in the pictures).

[rhofer]

Proposal from @nijel :

• For monolingual components there will be an option to configure "Intermediate language file" (tentative name, feel free to propose better). In case it is configured it will be shown when editing source strings.
• New per project configuration to enable source strings review (similarly as we do with current review process).
• If source review is enabled, once new source string is imported, it's would be marked as needs review, at this point all translations of this string will be read only.
• At approval of the source string the state changes and translations are opened up for editing.

[rhofer]

In order to have a precise design statement, I'd separate the discussion of source/intermediate language file concept from review concept (quality gate).

Intermediate language file

Generally, you mentioned explicitely ... for monolingual components ... . Why does it not work for bilingual components?

Beyond this, the current component configuration (django admin interface), has the following essentials:

Wouldn't it be sufficient to simply provide a check-box for "Intermediate language file" to say "Use Template for new translations: as intermediate language file"?

Pros:

• No need to have more files specified as the component configuration already provides today
• Only check-box to be added, whether template shall be used as "Intermediate language file" , or not.
  • If not checked, template is only donator for keys for new translations
  • If checked, template is donator for keys for new translations AND provider of SWEnglish terms

Cons:

• This merges two concepts into one configuration and does not allow to treat it separately. Hence, it would not be possible to NOT have a template as key donator specified, but still have an "Intermediate language file" specified as SWEnglish provider.

Conclusion:
In order to clearly separate these concepts and to allow for the full variability, a simple check-box is not sufficient and it is needed to provide two separate file specifications:

• Template for new translations: ...
• Intermediate language file: ...

Concerns:
Weblate shall assure that the filename of Intermediate language file: is NOT added to the list of available languages. In the example above, with filemask *.json the filename template found in the same folder as all other translations, would be added to the global list of available languages, what would be bad. (If this can't be avoided, the work around is by convention to tell any weblate user to not use template in any case (User profile settings, create new translations, ...))

Review process

• Project level configuration: Add an additonal check box to enable source string review: yes, I agree
• Block or open for translation depending on source string review state: yes, I agree

I just wonder on how many review states we need and upon which state translations are blocked or opened.

New source string

For this use case, two states might be sufficient, to say

• Waiting for review... tells the translator, hey, there is a new source string, please harmonize first and approve. As long as approval is not given, any further translations for this specific string are blocked.
• Approved ... tells the translator, that harmonization is done and any further translations for this specific string are now open.

Change in source string

This use case states, that a source string already existed and potentially already was approved and further translations might be ongoing. Then suddenly the source string changed. This can be simply due to the developers decision to merge back the current harmonization results from en to his SWEnglish or the developer may was fixing a typo in his SWEnglish. In this case, the semantic / context didn't change, hence such a source string change should not block any further translations.

Compared to use case New source string, this use case may requires more than only two states.

Conclusion

We need three Review states for the harmonization of the source strings. Therefore, I'd stick to the set we already have and assign block/open behavior as following:

• Needs editing: block for further translation (Harmonization was not done at all.)
• Waiting for review: open for further translation (Harmonization was majorly done. Potentially, fine tuning will happen, but basically this is no blocker for further translations. In case fine tuning (or similar) applies, the translator for a further language anyways get a notification/marker for "Failed check: Has been translated", what should be treated accordingly)
• Approved: open for further translation (Harmonization was fully done.)

With the following decision tree:

• If source string is new, set Review state to Needs editing and block further translations
• If source string has changed:
  • If this string has been in Review state: Needs editing, keep this state and keep blocking further translations
  • If this string has been in Review state: Waiting for review, keep this state and keep open for further translations
  • If this string has been in Review state: Approved, set this state back to Waiting for review and keep it open for further translations

Remark: Today, upon a source string change, weblate sets the Review state back to "Needs editing" and adds (default) a failing check "Has been translated".

Additionally, the translator taking care of harmonization can manually decide to set the Review state of a specific string back to Needs editing via Weblate's UI in order to block further translations.

@nijel does this reflect your proposal?

[nijel]

In order to clearly separate these concepts and to allow for the full variability, a simple check-box is not sufficient and it is needed to provide two separate file specifications

That's why I proposed it this way. We used to have one field for monolingual template and new language and that didn't work well as well.

Weblate shall assure that the filename of Intermediate language file: is NOT added to the list of available languages.

This is also the case for the monolingual base, the logic will be similar here.

I will get back to the review states later, but I think it follows what I've designed.

[nijel]

For the states, I'd prefer to keep it simple and I'm not sure that third state brings enough benefits. My intention is:

• If source string is new, set Review state to Needs editing and block further translations
• If source string has changed and it now matches English, keep current state
• If source string has changed and is still different from English, set Review state to Needs editing and block further translations

The motivation is to avoid additional work in case developers synchronize with English, but once they do other change, it most likely should be done on English as well.

Additionally, the translator taking care of harmonization can manually decide to set the Review state of a specific string back to Needs editing via Weblate's UI in order to block further translations.

See also #2772

[rhofer]

@nijel regarding the states, this is fine, as well. So we can keep the current Weblate behavior.

[rhofer]

@nijel As discussed, this will go in 4.0 release, correct? Can we remove then the "undecided" label?

[github-actions]

Thank you for your report, the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see similar problem, please open separate issue.
• If you are happy with the outcome, consider supporting Weblate by donating.