Research on the feasibility of using string hashing as the skeleton and unit ID for XLIFF files

[wmhn1872265132]

Related issues, PRs or discussions

None

What is the current state of the codebase?

Currently, the unit IDs of document translation strings are randomly generated.

Why are changes required?

This has the following disadvantages:

• Every time a new duplicate string appears, it needs to be retranslated, for example:
  • changes.md:
    ### Important notes
### New Features
### Changes
### Bug Fixes
### Changes for Developers
Please refer to [the developer guide](https://download.nvaccess.org/documentation/developerGuide.html#API) for information on NVDA's API deprecation and removal process.
#### Deprecations
  • userGuide.md:
    | Name |Desktop key |Laptop key |Description|
| Name |key |Description|
| Option | Behaviour |
| Options | Disabled, Enabled |
| Default | Disabled |
| Default | Enabled |
• When content is moved to another section, it needs to be retranslated, such as commit 6172254
• I am unsure whether deleting some content and then restoring it would preserve the translations of those strings in Crowdin. If preservation is possible, this would also be a applicable scenario.
• In extreme cases, if the English XLIFF document is lost and the file is regenerated, all previous translations for that file will be lost (this has already occurred during the migration of the current add-ons translation system).

What technical changes are required?

When a new string appears, use a hash of the English string (such as MD5, SHA1, or SHA256) as the filler content for the skeleton part and as the unit ID of the translated string. To ensure sufficient translation accuracy, the hash calculation should include the prefix and suffix of the string in that line. For compatibility reasons, currently translated strings can remain as they are.

Are the proposed technical changes API breaking?

No

Are there potential risks or issues with the proposed implementation?

Uncertain

[CyrilleB79]

For a bit of context:
When NVDA translation framework was moved to Crowdin/xliff, @michaelDCurran asked if this ID was necessary or not. An argument for this ID was that sometimes, we may have twice the same string in English translated differently in other languages.

In the GUI for example, we have "Off" translated differently depending on the context (pgettext usage in the code). This collision may happen in documentation too, hence the IDs.

Though, in practice, I realize that documentation strings are much longer. I'm not sure if there are such translation conflicts in practice. So maybe it's just possible to remove IDs. Or better, remove them but implement a disambiguation possibility only when needed.

[wmhn1872265132]

I think using string hashes as IDs can fully meet these requirements:

• First, cases where identical lines (including prefixes and suffixes) need to be translated into different content should be very rare.
• For identical lines that need to be translated into identical content, since the string hashes are the same, no retranslation is required.
• For identical lines that need to be translated into different content, since the ID is the hash of the line's string, you only need to slightly adjust the content of that line (e.g., add comments/tags, similar to the case with |None |--enable-start-on-logon=True|False |When installing, enable NVDA's [Use NVDA during Windows sign-in](#StartAtWindowsLogon)| <!-- markdownlint-disable-line MD055 MD056 -->) and then compute the hash of the full string to obtain a new ID.

[seanbudd]

I've converted this to a discussion because as-is this cannot be triaged. Feel free to perform this research in a separate repo / crowdin set up. Our current understanding is the approach might be worse. Without clear feasibility testing we can't triage this.