Skip to content

Conversation

@XavSPM
Copy link
Contributor

@XavSPM XavSPM commented Nov 12, 2025

Checklist

  • Testing:

    • The feature has automated tests
    • Tests were run
    • If not, explain how you tested your changes
  • Documentation:

    • The feature is documented
    • The documentation is up to date
    • Release notes:
      • Added an entry if the change is breaking or significant
      • Added an entry when adding a new feature

@georgweiss
Copy link
Collaborator

Just a hint: I think Serveur de file d\u2019attente might be written as Serveur de file d'attente in the messages file. Characters like é, è, â, ï should also work in the messages files.

@XavSPM
Copy link
Contributor Author

XavSPM commented Nov 12, 2025

@georgweiss

In the past, I encountered issues when trying to use accented characters directly in .properties files without escaping them in Unicode.

Has this behavior been fixed since Java 17? In other words, is it now possible to use UTF-8 encoded .properties files without any reading issues?

I have to admit, this would make translation work much easier.

@georgweiss
Copy link
Collaborator

@XavSPM, I would assume support for UTF-8 has been around before Java 17. That said, I'm not sure if accented characters will survive the Github roundtrip :-).
In any case, to make your life simpler, why not try it?

@kasemir
Copy link
Collaborator

kasemir commented Nov 12, 2025

It might not be about Java 17. UTF-8 has been around longer, but we didn't always include -Dfile.encoding=UTF-8 in the launcher scripts.

@shroffk
Copy link
Member

shroffk commented Nov 12, 2025

@XavSPM do you want to try to us the accented char's in this PR... I will wait before merging it.

@XavSPM
Copy link
Contributor Author

XavSPM commented Nov 13, 2025

@kasemir
@shroffk
@georgweiss

After running several tests, it turns out that it is no longer necessary to use Unicode escape sequences for accented characters.
I confirmed this behaviour with Java 17 and Java 21, and I also developed a small Python script to automatically update the messages_fr.properties files.

The simplest solution, in my view, would be to:

  1. validate the commits I have already made;

  2. then create a new commit dedicated to updating all the messages_fr.properties files.

Python script:

#!/usr/bin/env python3
import os
import re

def decode_unicode_escapes(text: str) -> str:
    """Converts \\uXXXX sequences into real Unicode characters."""
    return re.sub(r'\\u([0-9a-fA-F]{4})',
                  lambda m: chr(int(m.group(1), 16)),
                  text)

def convert_file(path: str):
    """Converts a .properties file in place."""
    with open(path, 'r', encoding='utf-8', errors='ignore') as fin:
        original = fin.read()

    converted = decode_unicode_escapes(original)

    # Rewrite only if content has changed
    if converted != original:
        with open(path, 'w', encoding='utf-8', newline='\n') as fout:
            fout.write(converted)
        print(f"✅ Updated: {path}")
    else:
        print(f"↔️  Unchanged: {path}")

def main(root_dir: str):
    """Recursively scans all subdirectories for messages_fr.properties files."""
    for dirpath, _, filenames in os.walk(root_dir):
        for name in filenames:
            if name == "messages_fr.properties":
                full_path = os.path.join(dirpath, name)
                convert_file(full_path)

if __name__ == "__main__":
    import sys
    if len(sys.argv) != 2:
        print("Usage: python3 convert_utf8_replace.py <root_directory>")
        sys.exit(1)
    main(sys.argv[1])
"""

@georgweiss
Copy link
Collaborator

@XavSPM, do you wish to run that script on all messages file for FR and update the PR?

@shroffk shroffk merged commit 609c2ea into ControlSystemStudio:master Nov 13, 2025
2 checks passed
@shroffk
Copy link
Member

shroffk commented Nov 13, 2025

ffd7085

@XavSPM can you verify... I used your script to fix the fr messages

I am sorry but I accidentally pushed the changes to the master ( thanks intellij... this after all the praise I shared for you during the developers meeting )

@XavSPM
Copy link
Contributor Author

XavSPM commented Nov 13, 2025

@shroffk

I’ll take a look as soon as possible.
At first glance, it doesn’t seem like anything was broken.

I’m just a bit surprised that the file
app/display/model/src/main/resources/org/csstudio/display/builder/model/messages.properties
was completely rewritten. Maybe an encoding issue?

When I run:

file app/display/model/src/main/resources/org/csstudio/display/builder/model/messages.properties

I get:

ASCII text

I’m glad you like my work, but I’m only doing the translations.
Still, it’s nice to hear.

@shroffk
Copy link
Member

shroffk commented Nov 13, 2025

I think the messages.properties was changed cause the EOL changed from CRLF to LF

@XavSPM XavSPM deleted the patch-10 branch November 14, 2025 06:16
@XavSPM
Copy link
Contributor Author

XavSPM commented Nov 14, 2025

Hello @shroffk ,

I reviewed all the modified files, and there wasn’t much that needed to be changed. Most of the work had already been handled correctly by my script 😅.

I’ve opened pull request #3645 with my updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants