Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change primary folder structure for reliable sorting and better culture-specific labels #18

Closed
iJungleboy opened this issue May 18, 2021 · 24 comments

Comments

@iJungleboy
Copy link
Contributor

The current folder structure will not scale well when a lot of languages are added, because sorting will make it hard to spot the right languages. I recommend the primary identifier comes first - being the language code itself with an addition. Optionally with the localized term in brackets.

So I recommend:

de-CH German-Switzerland (Deutsch-Schweiz)
de-DE German-Germany (Deutsch-Deutschland)
nl-NL Dutsch-Netherlands (Nederlands-Nederland)

This ensures that it's always consistent from A-Z sorting and it's clear if a language pack has been added. It also ensures that similar language packs are side-by-side as they will start with the same language code.

@iJungleboy iJungleboy changed the title Change primary folder structure for reliable sorting and better cultur-specific labels Change primary folder structure for reliable sorting and better culture-specific labels May 19, 2021
@hishamco
Copy link
Collaborator

Make sense, but for packages we agreed with @sbwalker to use Oqtane.{project}.{culture-Name}.nuget if I'm not wrong

@iJungleboy
Copy link
Contributor Author

@hishamco I don't see a problem with having the packages having a longer, more explicit name. What would the {culture-name} part look like? en-US or English-United States or something else?

@hishamco
Copy link
Collaborator

Oqtane.Client.de-CH.nuget

@iJungleboy
Copy link
Contributor Author

@hishamco that's perfect IMHO.

@sbwalker
Copy link
Member

sbwalker commented May 28, 2021

I posted a lot of thoughts about the current state of Localization in this issue:

#16

  1. Framework - an initial attempt has been made to implement localization throughout the framework. It is possible that there are a few items which have been missed, or even a few scenarios which are not supported currently (which need to be documented somewhere). Going forward we need to ensure that all new features are localizable.

I don't think a general sweep has been made yet to find any missing items requiring localization

  1. Resource files - need to be maintained by version, as the set of resources are dependent on a specific version of the Oqtane framework. A set of resources should exist for each official release version of the framework from version 2.1.0 forward.

IMHO requires a restructuring of the Translations repo as the current structure does not handle versions.

  1. Translators - in order to maintain the resource files we need to try to minimize the effort required by translators. @hishamco has created a tool to automate the extraction of resources from the framework. This tool does not currently handle all scenarios - but it needs to as soon as possible or else people will attempt to maintain resources manually. Since resource files are just text files it should be possible to do a comparison of the resources extracted for the latest version with the resources for the prior version, so that a translator can easily identify the specific modifications required without having to do a full translation for every new release.

I am not aware of any progress on the extraction tool. Based on other conversations it will need to handle Shared resources as well as the immutable key concept

  1. Translators - should be able to create a Nuget package which includes the compiled satellite assembly for their culture. This package should be created for each specific framework version and should be made available on nuget.org. Detailed instructions and a template should be created to explain how to create these packages.

instructions have not been created anywhere - however I know that @leigh-pointer created a Dutch satellite assembly and as able to install it

  1. Framework - Oqtane should allow non-technical users to install translation packages into Oqtane similar to how Modules or Themes are installed. I will add this capability.

done - I added support for installing satellite assemblies via Nuget.org or upload in 2.1

@iJungleboy
Copy link
Contributor Author

@sbwalker Based on this my understanding is that there are a few more challenges, but my question for the main folder-names hasn't been answered. But to combine this, I believe the best folder structure would be

  • de-CH German-Switzerland (Deutsch-Schweiz)
    • 02.00.02
      • Client
    • 02.01.00
      • Client
  • de-DE German-Germany (Deutsch-Deutschland)
  • nl-NL Dutsch-Netherlands (Nederlands-Nederland)

Just as an input: I believe Peter Donker created a system to manage resources across versions for DNN. Something like a Database to manage stuff and compare versions. It was blogged about https://www.dnnsoftware.com/community-blog/cid/144731/introducing-dnn-translator-a-new-translation-tool-for-dotnetnuke and is currently on github here: https://github.com/DNN-Connect/DNN-Translator

Donno if this helps.

@sbwalker
Copy link
Member

@hishamco @iJungleboy now that 2.1 is out we need to reorganize this repo so that the language resources are organized by version... and we should create individual nupkg packages for each languge so they are installable.

@sbwalker
Copy link
Member

@iJungleboy I took a look at the code for Peter Donker's tool and it appears that it would take a lot of effort to make it work for Oqtane. Have you ever looked at ResXManager: https://marketplace.visualstudio.com/items?itemName=TomEnglert.ResXManager - it is a popular open source project which is a member of the .NET Foundation. It does a nice job of comparing RESX files in folders and identifying missing keys. It has a snapshot feature as well so you can compare to a previous version.

image

@sbwalker
Copy link
Member

@iJungleboy Most tools operate based on folder hierarchy so the repo organization should be structured in a way which optimizes the main localization management use cases. The main use case I can imagine is within a specific Oqtane version you want to compare the RESX files for various languages ( usually comparing a custom language to the neutral language ). This allows the translator to ensure that their language resources contain all of the resource keys for that version. In order to accomplish this it would seem to make more sense to organize files by version first and then language:

  • 02.00.02
    • en Neutral (English)
      • Client
    • de-CH German-Switzerland (Deutsch-Schweiz)
      • Client
    • de-DE German-Germany (Deutsch-Deutschland)
      • Client
  • 02.01.00
    • en Neutral (English)
      • Client
    • de-CH German-Switzerland (Deutsch-Schweiz)
      • Client
    • de-DE German-Germany (Deutsch-Deutschland)
      • Client

This also makes it very easy to identify which languages have been translated for a specific version, and which have not.

Obviously for each new Oqtane release, a new version folder would need to be created and a subfolder would need to be created for the neutral language resource files. These resource files would need to contain the full set of resource keys for that specific release - and if we adopt the immutable(static) key approach discussed on #17 then the neutral resources would need to be naturally maintained as part of the standard development effort ( eliminating the need for any tool to extract resources from the code itself ).

And if a translator then wanted to create the language resources for a new version they could copy the folder from their previous version as a starting point and then run a tool like ResXManager which allows them to visualize which keys are missing by comparing their new folder to the neutral language folder for that same version.

I am interested in hearing your thoughts...

@hishamco
Copy link
Collaborator

IMHO the repo should have the latest translation, leave NuGet package keep track the versioning (no need to reivent the wheel)

@sbwalker
Copy link
Member

@hishamco I am not following your comment. Are you saying that whenever a new Oqtane version is released ( or 1 week after ) Nuget packages should be generated for each language managed in this repo? And these nuget packages would be stamped with the framework version number? I would imagine that some language resources will be maintained regularly by translators and others will not - which means the quality of the version based nuget packages will be very inconsistent.

@sbwalker
Copy link
Member

sbwalker commented Jun 11, 2021

We need a real world work flow which will facilitate ongoing language resource maintenance and package creation. I am open to suggestions...

@hishamco
Copy link
Collaborator

I would imagine that some language resources will be maintained regularly by translators and others will not

You are right, so the generated NuGet packges should be based on some critieria one of them the maintainability. If there's no activity from previous release to upcoming one no need to publish a translation NuGet package, this way the maintained one will be published regularly

1 similar comment
@hishamco

This comment has been minimized.

@sbwalker
Copy link
Member

So if I look at the current repo I see folders with a number of languages - but I have no idea which Oqtane version the resources were created for... so I can't create installable nuget packages and I have no idea if the resources I clone will work with my version of Oqtane. At the very least I need to be able to easily determine which Oqtane version that resources relate to.

@hishamco
Copy link
Collaborator

Regarding the package creation it could be automated via PowerShell scripts, I already did similar thing, but we could do that manually for the upcoming release. Let me know if I can handle or assist in this step

For versioning we could utilize the tagging feature in GitHub too to archive the translation versions, but I'm still see publishing a NuGet packages will be valuable for active translations

@hishamco
Copy link
Collaborator

so I can't create installable nuget packages and I have no idea if the resources I clone will work with my version of Oqtane. At the very least I need to be able to easily determine which Oqtane version that resources relate to

You are totally right, but I think this will introduce an overhead to maintain all the versions. Tagged version may usefual in this case, but we enchourage people to you use the latest and greates

@sbwalker
Copy link
Member

sbwalker commented Jun 11, 2021

Each language in this repo will be in a different state of maintenance in regards to being updated to the latest version at any given point in time. So you are saying that once a translator has updated their language resources for a specific Oqtane version they would need to remember to tag only their own folder with the appropriate version number. And other people interested in utilizing those resources would need to examine the tags on specific folders to determine the version. That does not sound like a good solution... and it is not even technically possible with Git.

@sbwalker
Copy link
Member

sbwalker commented Jun 11, 2021

The requirements I am looking to satisfy are:

Whether you are a developer, translator, or user...

and whether you are using raw language resources or nuget packages containing satellite assemblies...

you need to easily be able to quickly and easily identify the resources for a specific language that matches your installed version of Oqtane.

This repo should be organized in a way that satisfies this requirement.

And like I said I am open to suggestions. The only thing that is clear to me right now is that the current workflow for resource management is not functional.

@hishamco
Copy link
Collaborator

According what you said we need sort of resource extractor that's adapte in every release, then the maintainer needs to fill in the missing translation as fast as possible before the next version is release. This is may make things complicated

IMHO the repor should focus on the latest translations (Last release)

you need to easily be able to quickly and easily identify the resources for a specific language that matches your installed version of Oqtane.

Look for the NuGet package that match your version

At the end NuGet can simplify the versioning for such scenario

@sbwalker
Copy link
Member

sbwalker commented Jun 12, 2021

You make a good point... in all other cases a repo represents the latest version of assets ( unless you are using multiple branches ). So this repo should contain the latest versions of resources for various languages. And to satisfy the requirement of being able to determine the current "state" of the resources for a specific language ( ie. the Oqtane version they related to ) I think we should follow the example already set by @iJungleboy with his README file in the German folders. Its very easy to look at the README and see information about the resources for a language. I also went ahead this morning and created a functional nuspec and package.cmd which I placed in the de-DE folder as an example ( these files should be created in each language folder ). The idea is that when someone copies the language folder to the Oqtane.Client/Resources folder it contains the assets needed to package the satellite assembly into a nuget package with the proper naming, etc... So basically they copy the folder, build Oqtane, and then run the package.cmd and it creates the nupkg.

@sbwalker
Copy link
Member

The process for a translator to determine the missing keys will become much easier once we transition to using static keys and we ensure that a complete set of the neutral (EN) resources are constantly being maintained for the current framework.

@iJungleboy
Copy link
Contributor Author

For anybody following this thread, the last comment of @sbwalker refers to #17 to use keys for resources instead of the original English text.

@sbwalker I believe publishing nuget packages for each language would help sites a lot - because most will just want to install or update the language pack and forget about it a moment later. I see some challenges there, but I thought it's better to create a separate issue for this aspect #25

@leigh-pointer
Copy link
Collaborator

No further action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants