Skip to content

Rework Fluent file path and yaml references#21622

Merged
PunkPun merged 9 commits intoOpenRA:bleedfrom
pchote:remove-magic-translation-paths
Nov 3, 2024
Merged

Rework Fluent file path and yaml references#21622
PunkPun merged 9 commits intoOpenRA:bleedfrom
pchote:remove-magic-translation-paths

Conversation

@pchote
Copy link
Copy Markdown
Member

@pchote pchote commented Oct 15, 2024

Followup to #21602 and implements most of the remaining points from the previous Discord discussion.

The motivation for these changes is to remove the magic file path replacements and the specific references to "en" in the file paths. The exact shape of the future ingame translation system is still up for debate, but i'm fairly confident that it will end up as some variation of one of the following approaches (likely the first):

  1. Dynamic file overrides for mod-defined paths - e.g. via a downloadable translation bundle that contains a manifest saying "remap file ra|fluent/rules.ftl to langbundle|rules.ftl. In this case it would be incorrect to specify en in the mod-defined paths because the files that they load may not be in english.

  2. Explicitly defined file paths for each language at the point of usage - in which case we will need to update all of these file references when future languages are supported, meaning that nothing is lost by removing "en" from the paths in the meantime.

The incomplete user-level language selection is replaced by a mod-level language key. This enables modders and downstream repackagers to ship non-english languages and not have Fluent misbehave due to assuming the "en" culture.

For the time being, this PR updates the logic added in #21601 for passing strings to the content installer, but I intend to file a third PR in this series that will remove a lot of that magic in favour of an explicit mod.yaml definition for each content installer mod.

@pchote pchote added this to the Next Release milestone Oct 15, 2024
Copy link
Copy Markdown
Member

@PunkPun PunkPun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UserInterface.String() is definitely weird, we should do something different. At the minimum UserInterface.GetString()

I recon the best might be FluentProvider.GetString()?

Comment thread OpenRA.Game/Manifest.cs Outdated
Copy link
Copy Markdown
Member

@PunkPun PunkPun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while FluentStrings is a passable name, I feel like strings is the wrong term here. We are expecting .ftl files, not strings

FTL stands for Fluent Translation List, and it's the expected format of the field. Perhaps a better name would be FluentTranslationFiles, following the already existing convention.

@pchote
Copy link
Copy Markdown
Member Author

pchote commented Oct 17, 2024

Actually, the various nodes in map.yaml do expect raw file contents as child nodes, with the ability to reference external files being an odd implementation detail.

The fluent docs define that the basic unit of translation as a "message", so we should probably be replacing all uses of GetString etc with GetMessage.

@pchote
Copy link
Copy Markdown
Member Author

pchote commented Oct 17, 2024

while FluentStrings is a passable name, I feel like strings is the wrong term here. We are expecting .ftl files, not strings

The terminology in Linguini and our wrapping of that is to use "Bundle" to refer to a collection of messages. So would FluentBundles be an acceptable replacement?

@pchote
Copy link
Copy Markdown
Member Author

pchote commented Oct 17, 2024

I'll wait for #21623 and #21570 before rebasing.

@pchote pchote force-pushed the remove-magic-translation-paths branch from 4f9f121 to 4794b5e Compare October 19, 2024 14:36
@pchote
Copy link
Copy Markdown
Member Author

pchote commented Oct 19, 2024

Rebased on #21623 and reworked based on feedback above and in Discord.

This now also fixes #21618 by updating the --map-rules utility command that is used by the RC to populate the "rules" yaml block to include inline ftl. e.g. for allies-02, it now includes

FluentMessages: ra|fluent/lua.ftl, ra|fluent/campaign.ftl
	base64: ZHJvcGRvd24tZGlmZmljdWx0eSA9CiAgICAubGFiZWwgPSBEaWZmaWN1bHR5CiAgICAuZGVzY3JpcHRpb24gPSBUaGUgZGlmZmljdWx0eSBvZiB0aGUgbWlzc2lvbgoKb3B0aW9ucy1kaWZmaWN1bHR5ID0KICAgIC5lYXN5ID0gRWFzeQogICAgLm5vcm1hbCA9IE5vcm1hbAogICAgLmhhcmQgPSBIYXJkCiAgICAudG91Z2ggPSBSZWFsIHRvdWdoIGd1eQo=

and MapPreview/Map will decode this to include in the map bundle.

@pchote pchote force-pushed the remove-magic-translation-paths branch from 4794b5e to 52ef68b Compare October 20, 2024 12:05
Comment thread OpenRA.Mods.Common/Scripting/Global/UserInterfaceGlobal.cs Outdated
{
{ "Rules", map.RuleDefinitions },
{ "Translations", map.TranslationDefinitions },
{ "FluentMessages", map.FluentMessageDefinitions },
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also worse. We don't call it MiniYamlRules but just Rules. Please keep this as is for consistency.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is definitely confusing. I almost feel like just having it be Fluent would be better

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Fluent, the basic unit of translation is called a message. Messages are containers for information. You use messages to identify, store, and recall translation information to be used in the product.

This node is listing filenames containing/inline definitions of fluent messages. This is the same as "Sequences" listing filenames containing/inline definitions of sequences, etc.

There is no logic here to handle multiple languages, which I feel is a basic requirement for calling something "Translations".

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The definition already says that it is TranslationMessages. People who don't know that @projectfluent is a translation framework will be irritated. Ideally you should revert the renamings from #21602 as well.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would conflict with the naming of things in the actual translation system that manages the sets of language-specific fluent / sound / sprite files.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a yaml level I think this is the only place it is called fluent? So for non-power users both the concept of fluent and messages might be foreign

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for more advanced modders the other place will be #21622 (comment). I think we should make sure these are named well.

Comment thread OpenRA.Game/Manifest.cs
Comment on lines +89 to +90
// TODO: This should be controlled by a user-selected translation bundle!
public readonly string FluentCulture = "en";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it should be controlled by a user settings like it already is.

Copy link
Copy Markdown
Member

@PunkPun PunkPun Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be controlled by the bundles. The user should be able to select the language from installed languages eventually yeah. It should give the pop-up window to reload the game same way changing display size does

Comment thread OpenRA.Mods.Common/Lint/CheckFluentReferences.cs
Comment thread OpenRA.Game/FluentProvider.cs
@pchote pchote force-pushed the remove-magic-translation-paths branch 3 times, most recently from 36d5ded to bf4a684 Compare October 23, 2024 18:10
@pchote pchote force-pushed the remove-magic-translation-paths branch from bf4a684 to 30e4228 Compare October 28, 2024 21:25
@PunkPun PunkPun dismissed Mailaender’s stale review October 29, 2024 09:14

It should be exposed to settings menu, but a ton of work is yet to be done to properly implement it into setting menu

Copy link
Copy Markdown
Member

@PunkPun PunkPun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes done here should be good enough for playtest

@PunkPun PunkPun merged commit 147cb56 into OpenRA:bleed Nov 3, 2024
@PunkPun
Copy link
Copy Markdown
Member

PunkPun commented Nov 3, 2024

changelog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MP maps with custom lobby options need to be tested wrt language strings

3 participants