Language Editor: new "3rd-party reference translation" tool

One of my biggest challenges is keeping PhotoDemon language files up-to-date after contributors drop out of the project.  I never want to abandon people's localization work, but if I don't speak a language it's incredibly hard to know how to maintain a localization, especially given the number of features added to a typical PD release.

I've already added an integrated online translation tool to the Language Editor, but that can only go so far - especially for menu text like "Copy".  Is that a verb?  A noun?  If it *is* a verb, which tense should I use?  I have no GD idea.

So I've now added a helpful localization feature: an option to load a .po file from other software as an additional translation reference.  When a phrase from PD matches a phrase from that software, PD can now notify you of the 3rd-party translation choice so you can make an informed decision.

For a non-native speaker like me, this is a huge help when trying to figure out how to update a localization.

To make this work, I basically wrote a quick-and-dirty .po parser, and it is indeed quick (and dirty).  For example, GIMP ships with some 5,500+ phrases in a typical .po and PD will load and parse the full file in < 0.1s.  Localized strings are stored in a homebrew hash table and it all works nicely.

I have deliberately written this new tool so that you *cannot* bulk-copy translations from a reference file.  Translations are typically copyright their original authors and I don't want to encourage anyone to steal or use them inappropriately.

So the way PD's tool works is that the 3rd-party translations are only provided as a reference.  To replace an existing PD translations with a 3rd-party one, you must manually review the translations one-by-one and interact with the dialog to swap the text.  This is by design to try and make this feature helpful but ethically okay.

I honestly don't know if anyone else uses PD's language editing tool (most localizers use text editors AFAIK) but as long as I'm writing these features for myself, I may as well make 'em available to others.

With this, I think I have everything I need to get un-maintained localizations up-to-date for a 9.0 release.

Author: tannerhelland

App/PhotoDemon/Languages/README.md

@@ -1,66 +1,96 @@
 ## About PhotoDemon localization
 
-Thank you for your interest in PhotoDemon's language support.
+Thank you for your interest in PhotoDemon localization.
 
-This README provides a quick overview of PhotoDemon's localization process.  Importantly, it provides critical instructions to prevent you from losing any updates you make to PhotoDemon's language files.
+This README provides a quick overview of PhotoDemon's localization process.  Importantly, there are **critical instructions** you must follow to avoid losing changes to these language files.
 
-Please read this entire document before editing any files in this folder.
+Please read this entire document before editing PhotoDemon language files.
 
 ### Quick overview
 
-The XML files in this folder (`/App/PhotoDemon/Languages`) are PhotoDemon's "official" language files.  They are included in all PhotoDemon downloads.
+The XML files in this folder (`/App/PhotoDemon/Languages`) are PhotoDemon's "official" language files.  They ship with all PhotoDemon downloads.
 
-When PhotoDemon applies an automatic update, these files will be replaced with newer versions from the PhotoDemon update server.  For nightly builds in particular, language files are frequently updated, so this replacement process could happen at any time.
+When PhotoDemon self-updates, these XML files are automatically replaced with newer versions from the update server.  For nightly builds in particular, language files are frequently updated, so this replacement process could happen at any time.
 
-Because these language files can be replaced at any time, you **MUST NOT** edit them directly.
+Because the language files in this folder can be replaced at any time, **you MUST NOT edit them directly**.
 
-Instead, modified language files must be saved to the `/Data/Languages` folder.  The `/App` subfolder is reserved for PhotoDemon itself, but the `/Data` subfolder is your user data folder.  Files in the `/Data` folder will never be erased by PhotoDemon updates.
+Instead, any modified language files should be saved to the `/Data/Languages` folder.  The `/App` subfolder is reserved for PhotoDemon itself, but the `/Data` subfolder is your user data folder.  Files in the `/Data` folder will not be erased by PhotoDemon updates.
 
-So please, please, please remember to place any modified language files in the `/Data/Languages` folder, **not** the `/App/PhotoDemon/Languages` folder.
+So please, please, please remember to place any modified language files in the `/Data/Languages` folder, **not** this `/App/PhotoDemon/Languages` folder.
 
-### Editing an existing language file
+### Editing an existing language file in any text editor
 
-Editing an existing language file is easy!  
+Editing existing language files is easy.
+
+**Short version:**
+
+PhotoDemon's language files are plain-text XML-like files.  You can edit them in any text editor.  After making changes, send the updated file to me and I will add it to PhotoDemon!
+
+**Long version:**
 
 1) Copy the language file you want to edit into the `/Data/Languages` folder, and modify its filename to something like `German_new.xml`.
-2) Open the language file in any text editor.  Many translators use the free Notepad++ app: https://notepad-plus-plus.org/
-3) Inside the language file, you will notice a few tags at the top of the file.  These tags have names like `<langid>` or `<author>`.  
-4a) Ensure the `<langid>` matches the two-letter ISO language code (https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) and two-letter ISO country code (https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) that you want to provide.  PhotoDemon uses these codes to auto-suggest languages to new users, by matching these codes to the language and country code of each user's PC.  
-4b) (Optional)  You can also modify the language version (to ideally match the PhotoDemon version you are working on), language status (e.g. "complete" or "in-progress"), and author tags, as relevant.
-5) All phrases in the program are stored as `<original>` and `<translation>` pairs.  You *must not* modify any text inside `<original></original>` tags.  PhotoDemon requires these tags to precisely match on-screen interface elements - that's how it locates specific translations inside each file.
+2) Open the language file in any text editor.  For beginners, I recommend the free Notepad++ app: https://notepad-plus-plus.org/
+3) Inside the language file, you will see a collection of tags.  Tags are special text enclosed by angle brackets, with names like `<langid>` or `<author>`.  Do not edit text within < and > characters.  Tag names are used by PhotoDemon to locate translation data.  Only translate text *between* tags - for example, if you see `<translation>text goes here</translation>` you can freely edit the `text goes here` portion.
+4a) Ensure the `<langid>` text at the top of the file matches the two-letter ISO language code (https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) and two-letter ISO country code (https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) are accurate.  PhotoDemon uses these codes to auto-suggest languages to new users (by matching the codes to the language and country code of a user's PC).
+4b) (Optional)  You can also modify the language version to match the PhotoDemon version you are working on, and the language status (e.g. "complete" or "in-progress") and author tags to reflect their current state.
+5) All phrases in the program are stored as `<original>` and `<translation>` pairs.  You *must not* modify any text inside `<original></original>` tags.  PhotoDemon requires these tags to precisely match on-screen interface elements.
 6) You can freely modify any text between `<translation>` and `</translation>` tags.  Empty tags (e.g. translation tags with no text between them) have not yet been translated.  This is common for newly added features.  You can use this to quickly locate text that needs to be translated, by searching for `<translation></translation>`.
-7) Where possible, PhotoDemon tries to match naming conventions with other popular photo editors, like Adobe Photoshop or GIMP.  If a menu command in PhotoDemon does not make sense, you might check what text GIMP uses for their corresponding feature: https://gitlab.gnome.org/GNOME/gimp/-/tree/master/po (click on any language to see the GIMP translations for that language).  I must also request that you *do not* blindly copy text from other software's translations.  I provide the GIMP link only as a reference for comparison on challenging phrases, *not* as a way to steal their translation work.
+7) Where possible, PhotoDemon tries to use similar naming conventions to other popular photo editors, like Adobe Photoshop and GIMP.  If a menu command in PhotoDemon does not make sense, you might see what text Photoshop or GIMP uses for their corresponding feature.  (Later on, this document will describe an automated way to do this.)
 8) Sometimes, PhotoDemon needs to insert dynamic text at run-time.  This is usually a number or percent, like `"Step 1 of 3"`.  Dynamic text is flagged in translations using a % prefix, so the example phrase `"Step 1 of 3"` would appear as `<original>Step %1 of %2</original>` in a PhotoDemon language file.  In your translation, place the same `%1` and `%2` markers in your translation wherever they make sense.
 9) When you are satisfied with your changes, save your work and contact me.  Filing a pull request or new issue on GitHub and attaching your updated file is the fastest way to get your changes merged into the main application: https://github.com/tannerhelland/PhotoDemon .  If you are afraid of GitHub, alternate means of contacting me are available here: https://photodemon.org/about/ 
 
 ### Starting a new language file
 
-Starting a new PhotoDemon language file is exactly the same as editing an existing one.  The only difference is that instead of copying an existing language file into the `/Data/Languages` folder, you will instead copy PhotoDemon's master English text file from `/App/PhotoDemon/Languages/Master/MASTER.xml` into the `/Data/Languages` folder.  Rename the file with an appropriate language name, then follow the steps given in `Editing an existing PhotoDemon language file`, above.
+Starting a new PhotoDemon language file is exactly the same as editing an existing one.  The only difference is that instead of copying an existing language file into the `/Data/Languages` folder, you will instead copy PhotoDemon's master English text file from `/App/PhotoDemon/Languages/Master/MASTER.xml` to the `/Data/Languages` folder.  Rename the file with an appropriate language name, then follow the steps given above in `Editing an existing PhotoDemon language file`.
 
-### PhotoDemon's built-in Tools > Language Editor menu
+### Using PhotoDemon's built-in Tools > Language Editor menu
 
 Some translators prefer to use PhotoDemon's built-in `Tools > Language editor` tool.  It can simplify the language editing process, especially for beginners.
 
-PhotoDemon's Language Editor tool is largely self-explanatory, but one item deserves extra explanation.  On the first page of the editor, there is a box titled `(optional) free DeepL.com API key for translation suggestions`.  
+PhotoDemon's Language Editor tool is largely self-explanatory, but two items deserve extra explanation.  
+
+#### Automatic translation suggestions (via the online DeepL service)
 
-DeepL.com (https://www.deepl.com/translator) is a free, high-quality translation service.  It is not connected to or affiliated with PhotoDemon in any way, but PhotoDemon's Language Editor can interface with DeepL.com to automatically suggest translations for you.  This is especially helpful for longer phrases, like error messages, which can be tedious to translate manually.
+On the first page of the editor, there is a box titled `(optional) free DeepL.com API key for translation suggestions`.  
 
-Like most online services, DeepL requires you to set up a free user account before using their translation service.  As of June 2022, the "Free" button at this link is the fastest way to setup a new, free account:
+DeepL.com (https://www.deepl.com/translator) is a free, high-quality translation service.  DeepL is not connected to or affiliated with PhotoDemon in any way, but PhotoDemon's Language Editor can interface with DeepL.com to automatically suggest translations for you.  This is especially helpful for longer phrases, like error messages, which can be tedious to translate manually.
+
+Like most online services, DeepL requires you to set up a free user account before using their translation service.  As of June 2022, the "Free" button at this link is the fastest way to setup a new account:
 
 https://www.deepl.com/pro-api?cta=header-pro-api/
 
-After setting up an account, you will be provided with a unique API key.  Simply copy and paste that key into the matching box in PhotoDemon's Language Editor, and PhotoDemon can now auto-suggest translations for you.  PhotoDemon will also save the pasted API key to your user preferences file, so you only need to enter it once.
+After setting up an account, you will receive a unique API key.  Simply copy and paste that key into the matching box in PhotoDemon's Language Editor, and PhotoDemon can now auto-suggest translations for you.  PhotoDemon will also save the pasted API key to your user preferences file, so you do not need to enter it again.
+
+Again, this feature **is not required** to edit PhotoDemon translations.  It is simply provided as a convenience for those who want it.  I initially added it to help me update language files that no longer have active contributors, and because I use it frequently I wanted to share it with others.  
+
+If other automated translation services are more appropriate for your language, please contact me and I'll see if I can add them to the tool as well.
+
+#### Comparing translations to other open-source software
 
-Again, this feature **is not required** to edit PhotoDemon translations.  It is simply provided as a convenience for those who want it.  I initially added it to help me update language files that no longer have active contributors, and because I use it so frequently I thought it might also be useful to others.  If other automated translation services are more appropriate for your language, please contact me and I'll see if I can add them to the tool as well.
+Where possible, PhotoDemon tries to use the same terminology as other popular photo editors.  This reduces the learning curve for new users and makes it easier to switch between PhotoDemon and other software.
+
+To help achieve this, the first page of PhotoDemon's Language Editor provides a box titled `(optional) 3rd-party language file (.po) to compare`.  Click the `...` button to select a language file from any other software.  For example, GIMP's language files are freely downloadable from this link:
+
+https://gitlab.gnome.org/GNOME/gimp/-/tree/master/po 
+
+If you provide a translation file from another app, PhotoDemon will also display that app's translations on the translation panel (when available).  This is especially helpful for common terms like menu and tool names.
+
+While this feature is very helpful, I must request that you *do not* blindly copy text from other software's translations.  These translations are typically copyright by their original authors and should be used as a reference only.  PhotoDemon's Language Editor will not allow you to bulk-copy these translations, by design.  In particular, I provide the GIMP link only as a helpful reference for challenging phrases, *not* as a way to steal anyone else's translation work.
+
+Again, note that this feature is 100% optional and you do not need to use it to translate PhotoDemon text.  It only exists as an extra help for those who want it.
 
 ### Conclusion
 
-Thank you again for helping me improve PhotoDemon's language files.  If you have any other questions, please contact me directly.  My contact info is available here:
+Thank you again for helping me improve PhotoDemon's language files.  If you have any other questions, please contact me.  My contact info is available here:
 
 http://photodemon.org/about/contact/
 
+I am very excited to merge your work into the project!
+
 Kind regards,
+
 Tanner Helland
+
 (PhotoDemon developer)
 
 Last modified: 28 June 2022

Classes/pdAutoLocalize.cls

@@ -66,6 +66,10 @@ Private m_ServiceAvailable As Boolean
 'The user needs to supply an API key for the target translation service; without this, the class WILL NOT WORK
 Private m_apiKey As String
 
+Public Function GetAPIKey() As String
+    GetAPIKey = m_apiKey
+End Function
+
 Public Sub SetAPIKey(ByRef srcKey As String)
     m_apiKey = srcKey
 End Sub

Classes/pdStringHash.cls

@@ -15,8 +15,8 @@ Attribute VB_Exposed = False
 'Fast String-based collection, using hash-table + separate chaining (linked-list) for collisions
 'Copyright 2022-2022 by Tanner Helland
 'Created: 07/March/22
-'Last updated: 21/March/22
-'Last update: wrap up initial build, including official integration into pdTranslate
+'Last updated: 30/June/22
+'Last update: fix overflow table bug when re-adding the same key multiple times
 '
 'PhotoDemon's run-time localization engine has to manage a ton of strings.  Historically, we've just
 ' pulled strings on-demand from the base XML string.  This is very fast (especially since localization
@@ -49,6 +49,7 @@ Attribute VB_Exposed = False
 '***************************************************************************
 
 Option Explicit
+Option Compare Binary
 
 'Item entry in the table.  Key and value are stored together, alongside a link to the next item in this chain.
 Private Type MergedEntry
@@ -141,7 +142,7 @@ Friend Function AddItem(ByRef srcKey As String, ByRef srcItem As String) As Bool
 
                     'Key already exists in the table.  Overwrite it with the new value, then exit.
                     If (m_overflowTable(idxTable).sKey = srcKey) Then
-                        m_hashTable(idxTable).sItem = srcItem
+                        m_overflowTable(idxTable).sItem = srcItem
                         Exit Function
 
                     Else