[Discussion] Crowdin and github integration. #119

Kuzma30 · 2022-04-23T12:05:54Z

I propose to discuss the integration of translation updates.
I've tried something, and there are some results.
What I did

Created an empty mail for tests, I can provide login and password if needed
Under this email I created a test project on https://crowdin.com/freecadtest. Tests decided to make on fork https://github.com/Kuzma30/FreeCAD. Tests conducted in one language (Ukrainian) and on one file (Spreadsheet).
The configuration file https://github.com/Kuzma30/FreeCAD/blob/master/crowdin.yml was created (online when the integration was connected) and supplemented.
After synchronization, I received partially translated file (translation was obtained automatically from Spreadsheet_uk.ts file from github.
What are the problems when importing
a) Loss of translation author
b) loss of Approved status
c) loss of translated variants
This error may be solved (I think) with translation memory. I can't test this on empty project..
The following errors were also noticed. For example
https://github.com/FreeCAD/FreeCAD/blob/f8ae185c15480b1b6cc4209d33a8e726d3a7ba80/src/Mod/Spreadsheet/Gui/Resources/translations/Spreadsheet.ts#L662
Line

<source>&lt;html&gt;&lt;head/&gt;&lt;body&gt;&lt;p&gt;The symbol is used to separate lines, usually a single quote (&apos;) or a double quote (&amp;quot;). Must be a single character.&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;</source>

On the server it (https://crowdin.com/translate/freecad/27907/en-uk?filter=basic&value=0#q=Character%20used%20to%20delimit%20strings) looks like

<html><head/><body><p>The character used to separate lines, usually a single quote (') or a double quote (&quot;). It must be a single character.</p></body></html>.

Accordingly, a translation was made for this line.
On the test server (https://crowdin.com/translate/freecadtest/88/en-uk?filter=basic&value=3#q=Character%20used%20to%20delimit%20strings,%20) it looks like in the original .ts file. The problem is that in this case translation for such strings is lost.

Then on local machine I run command to update Spreadsheet.ts. New lines with untranslated text appeared in the file. They also appeared on the translation server after the synchronization.
Crowdin create pull request in separate branch. Then it can be merged.
For automation update .ts files on github and auto create .qm after update I think we can use Actions mechanism ( test this later)

The text was updated successfully, but these errors were encountered:

luzpaz · 2022-04-24T10:02:05Z

CC @yorikvanhavre

Kuzma30 · 2022-04-24T18:16:59Z

Looks like official script start work better and write "more".
Is there official script for making .qm files?

Kuzma30 · 2022-04-25T08:17:08Z

A few words about commits from crowdin. In the settings, you can choose the frequency of commits from an hour to one day. Commits are created in a separate branch. Created separately for each language and translation file. Then they can be squashed into one through the github menu. This has been tested on my fork.

yorikvanhavre · 2022-04-29T13:29:32Z

I'm all for using the github integration system too, but there are several things we need to check too:

what happens the day we move to another platform: gitlab seems ok for now, but what if we have our own git server?
qm files should indeed be removed from source and built at build time. Probably we need a cmake macro for that. Maybe that could be a first point to address, as it doesn't depend from the crowdn stuff?

I'll have a look at your fork ASAP...

Kuzma30 · 2022-04-29T16:56:22Z

In this situation we can use https://support.crowdin.com/cli-tool/. You can disable github integration at any time without lost of translations memory.
I am thinking on it and also wanted to suggest this. I will try realize this, but I am bad programmer ;)
I can modify my version of script, but I don't know how to run it during cmake/build.

Kuzma30 · 2022-04-30T13:02:08Z

I modify updatets.py to run lrelease in directories. Its works on local machine. But I don't know how to run it during build or configure stage (with cmake).

cat updateqm.py 
#!/usr/bin/python3

# -*- coding: utf-8 -*-
# (c) 2010 Werner Mayer LGPL

#***************************************************************************
#*                                                                         *
#*   Copyright (c) 2010 Werner Mayer <wmayer@users.sourceforge.net>        *
#*                                                                         *
#*   This program is free software; you can redistribute it and/or modify  *
#*   it under the terms of the GNU Library General Public License (LGPL)   *
#*   as published by the Free Software Foundation; either version 2 of     *
#*   the License, or (at your option) any later version.                   *
#*   for detail see the LICENCE text file.                                 *
#*                                                                         *
#*   This program is distributed in the hope that it will be useful,       *
#*   but WITHOUT ANY WARRANTY; without even the implied warranty of        *
#*   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the         *
#*   GNU Library General Public License for more details.                  *
#*                                                                         *
#*   You should have received a copy of the GNU Library General Public     *
#*   License along with this program; if not, write to the Free Software   *
#*   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  *
#*   USA                                                                   *
#*                                                                         *
#***************************************************************************

# Changelog:
# 0.1 Initial Release, based on updatets script

from __future__ import print_function

Usage = """updateqm - run lrelease for all .ts files found in the source directories

Usage:
   updateqm

Authors:
  (c) 2022 FreeCAD Volunteers
  Licence: LGPL

Version:
  0.1
"""

import os, sys

directories = [
        {"tsname":"FreeCAD", "workingdir":"./src/Gui/Language"},
        {"tsname":"AddonManager", "workingdir":"./src/Mod/AddonManager/Resources/translations"},
        {"tsname":"Arch", "workingdir":"./src/Mod/Arch/Resources/translations"},
        {"tsname":"Draft", "workingdir":"./src/Mod/Draft/Resources/translations"},
        {"tsname":"Drawing", "workingdir":"./src/Mod/Drawing/Gui/Resources/translations"},
        {"tsname":"Fem", "workingdir":"./src/Mod/Fem/Gui/Resources/translations"},
        {"tsname":"Image", "workingdir":"./src/Mod/Image/Gui/Resources/translations"},
        {"tsname":"Mesh", "workingdir":"./src/Mod/Mesh/Gui/Resources/translations"},
        {"tsname":"MeshPart", "workingdir":"./src/Mod/MeshPart/Gui/Resources/translations"},
        {"tsname":"OpenSCAD", "workingdir":"./src/Mod/OpenSCAD/Resources/translations"},
        {"tsname":"PartDesign", "workingdir":"./src/Mod/PartDesign/Gui/Resources/translations"},
        {"tsname":"Part", "workingdir":"./src/Mod/Part/Gui/Resources/translations"},
        {"tsname":"Path", "workingdir":"./src/Mod/Path/Gui/Resources/translations"},
        {"tsname":"Points", "workingdir":"./src/Mod/Points/Gui/Resources/translations"},
        {"tsname":"Raytracing", "workingdir":"./src/Mod/Raytracing/Gui/Resources/translations"},
        {"tsname":"ReverseEngineering", "workingdir":"./src/Mod/ReverseEngineering/Gui/Resources/translations"},
        {"tsname":"Robot", "workingdir":"./src/Mod/Robot/Gui/Resources/translations"},
        {"tsname":"Sketcher", "workingdir":"./src/Mod/Sketcher/Gui/Resources/translations"},
        {"tsname":"Spreadsheet", "workingdir":"./src/Mod/Spreadsheet/Gui/Resources/translations"},
        {"tsname":"Start", "workingdir":"./src/Mod/Start/Gui/Resources/translations"},
        {"tsname":"TechDraw", "workingdir":"./src/Mod/TechDraw/Gui/Resources/translations"},
        {"tsname":"Test", "workingdir":"./src/Mod/Test/Gui/Resources/translations"},
        {"tsname":"Tux", "workingdir":"./src/Mod/Tux/Resources/translations"},
        {"tsname":"Web", "workingdir":"./src/Mod/Web/Gui/Resources/translations"}
        ]

LRELEASE = ""

def find_tools():

    print(Usage + "\nFirst, lets find lrelease tool on your system")
    global LRELEASE
    if (os.system("lrelease -version") == 0):
        LRELEASE = "lrelease"
    elif (os.system("lrelease-qt5 -version") == 0):
        LRELEASE = "lrelease-qt5"
    else:
        raise Exception("Cannot find lrelease")
    print("\nQt tool lrelease have been found!\n",
          "\t" + LRELEASE   + "\n")
    print("==============================================\n")

def make_qmfiles(entry):

    global LRELEASE
    print ("\n\n=============================================")
    print (f"MAKING QM files FOR {entry['tsname']}")
    print ("=============================================")
    cur = os.getcwd()
    log_redirect = f" 2>> {cur}/qm_create_stderr.log 1>> {cur}/qm_create_stdout.log"
    os.chdir(entry["workingdir"])

    execline = []
    execline.append (f"{LRELEASE} *.ts {log_redirect}")
    print(f"Executing commands in {entry['workingdir']}:")
    for line in execline:
        print (line)
        os.system(line)
    print()
    os.chdir(cur)

def main(mod=None):

    find_tools()
    path = os.path.realpath(__file__)
    path = os.path.dirname(path)
    os.chdir(path)
    os.chdir("..")
    os.chdir("..")
    print("\n\n\n BEGINNING MAKING QM FILES\n\n")
    if mod:
        for i in directories:
            if i["tsname"] == mod:
                print("WARNING - Running QT lrelease for ",mod,"ONLY")
                make_qmfiles(i)
                break
    else:
        for i in directories:
            make_qmfiles(i)
    print("\nIf updateqm.py was run successfully. Now you have latest translations ")
    print("stderr output from lrelease can be found in qm_create_stderr.log")
    print("stdout output from lrelease can be found in qm_create_stdout.log")

if __name__ == "__main__":
    if len(sys.argv[1:]) > 0:
        main(sys.argv[1])
    else:
        main()

Kuzma30 · 2022-04-30T17:58:47Z

CC @yorikvanhavre
Can I make pull request with this version of script?

Kuzma30 · 2022-05-02T15:03:16Z

2. qm files should indeed be removed from source and built at build time. Probably we need a cmake macro for that. Maybe that could be a first point to address, as it doesn't depend from the crowdn stuff?

Done.
https://github.com/Kuzma30/FreeCAD/commits/qm_create_during_configure
ping @yorikvanhavre

Kuzma30 · 2022-05-05T06:56:19Z

@yorikvanhavre can you just update .ts files and Crowdin strings (In the usual way)?
According to my test crowdin server looks like it will be 1k+ new string to translate.
May be translator can do this before v0.20 release?

yorikvanhavre · 2022-05-05T12:13:08Z

I think qm files should be built through a cmake macro or function, I am not sure it can use your py script. In any case, it's best to do this after the 0.20 release, as we don't add new features at the moment.
We're still fixing details with the translations scripts with @chennes, I'll do a new crowdin merge ASAP.

kaktusus · 2022-05-05T12:14:28Z

We all look forward to the Crowdin update

The new 0.2 release has a chance to stop being a hole and have nicely translated user interfaces in many languages.

Kuzma30 · 2022-05-05T15:43:29Z

I think qm files should be built through a cmake macro or function, I am not sure it can use your py script. In any case, it's best to do this after the 0.20 release, as we don't add new features at the moment.
We're still fixing details with the translations scripts with @chennes, I'll do a new crowdin merge ASAP.

In comment with link to fork you can see how to run script during cmake configure. I test this on my local Ubuntu 22 mashine.

yorikvanhavre · 2022-05-09T14:09:28Z

Ok I'll have a look!

andrii-bodnar · 2022-06-08T12:45:29Z

Hey everyone,

There is a Crowdin GitHub Action that allows you to easily automate source files pushing to Crowdin and translations download.

I can do a pull request with the integration setup. I would only need a green light from you and the minimal setup of this repo (create a Crowdin project and add some Actions secrets).

The Action is based on Crowdin CLI, so the existing crowdin.yml config can be reused.

luzpaz · 2022-06-08T12:58:08Z

Thank @andrii-bodnar we appreciate the volunteering. Out of curiosity is there a way to do dry-run testing?

Kuzma30 · 2022-06-14T10:45:44Z

@yorikvanhavre
As we have 0.20 release, what is your decision with github integration?
Did you test script for creation of qm files?
Kuzma30/FreeCAD@6cda6e3

yorikvanhavre · 2022-06-14T12:26:15Z

Let's wait until the release is complete please 😅 I'll attack that ASAP

Kuzma30 · 2022-06-19T11:13:16Z

@yorikvanhavre
I test adding tags and changing default commit text.

pull_request_title: '[skip ci] Crowdin upload'
pull_request_labels:
  - crowdin
  - l10n

Adding this to config file (https://github.com/Kuzma30/FreeCAD/blob/master/crowdin.yml) produce such PR
https://github.com/Kuzma30/FreeCAD/pulls
I also test how to save approved translation. (I think it will be work for your too)
I change in settings
Duplicate Strings to Show, but auto-translate them
Then, when I add new file to translate (from realthunder branch) it save translations and approve status.

Kuzma30 · 2022-06-23T16:15:41Z

@chennes Please, can you test this script on Windows?
Kuzma30/FreeCAD@6cda6e3

Idea - creating qm files during cmake configure process

chennes · 2022-06-23T18:34:09Z

It doesn't work if you don't have the official Python installed from the Microsoft Store -- you need to modify your command to use the version of Python that the configuration found.

Kuzma30 · 2022-06-23T18:44:03Z

It doesn't work if you don't have the official Python installed from the Microsoft Store -- you need to modify your command to use the version of Python that the configuration found.

Like last updatets.py version?

chennes · 2022-06-23T18:58:18Z

I don't know what you mean: cMake will have found some version of Python, your command should use that one, not try to use whatever is in PATH

Kuzma30 · 2022-06-24T13:42:21Z

@chennes Made small modification, please test (on Windows)
FreeCAD/FreeCAD@0f6f3cc
If this will work I made some cleanup to script and add new directory to list.

Kuzma30 · 2022-07-08T15:49:46Z

Let's wait until the release is complete please sweat_smile I'll attack that ASAP

@yorikvanhavre Is v.20 release complete?

Kuzma30 · 2022-07-26T12:57:21Z

@yorikvanhavre Still waiting (for feedback as minimum)

yorikvanhavre self-assigned this Apr 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Discussion] Crowdin and github integration. #119

[Discussion] Crowdin and github integration. #119

Kuzma30 commented Apr 23, 2022 •

edited

luzpaz commented Apr 24, 2022

Kuzma30 commented Apr 24, 2022

Kuzma30 commented Apr 25, 2022

yorikvanhavre commented Apr 29, 2022 •

edited

Kuzma30 commented Apr 29, 2022 •

edited

Kuzma30 commented Apr 30, 2022

Kuzma30 commented Apr 30, 2022

Kuzma30 commented May 2, 2022 •

edited

Kuzma30 commented May 5, 2022

yorikvanhavre commented May 5, 2022

kaktusus commented May 5, 2022

Kuzma30 commented May 5, 2022

yorikvanhavre commented May 9, 2022

andrii-bodnar commented Jun 8, 2022 •

edited

luzpaz commented Jun 8, 2022

Kuzma30 commented Jun 14, 2022

yorikvanhavre commented Jun 14, 2022

Kuzma30 commented Jun 19, 2022

Kuzma30 commented Jun 23, 2022 •

edited

chennes commented Jun 23, 2022

Kuzma30 commented Jun 23, 2022

chennes commented Jun 23, 2022

Kuzma30 commented Jun 24, 2022 •

edited

Kuzma30 commented Jul 8, 2022

Kuzma30 commented Jul 26, 2022

[Discussion] Crowdin and github integration. #119

[Discussion] Crowdin and github integration. #119

Comments

Kuzma30 commented Apr 23, 2022 • edited

luzpaz commented Apr 24, 2022

Kuzma30 commented Apr 24, 2022

Kuzma30 commented Apr 25, 2022

yorikvanhavre commented Apr 29, 2022 • edited

Kuzma30 commented Apr 29, 2022 • edited

Kuzma30 commented Apr 30, 2022

Kuzma30 commented Apr 30, 2022

Kuzma30 commented May 2, 2022 • edited

Kuzma30 commented May 5, 2022

yorikvanhavre commented May 5, 2022

kaktusus commented May 5, 2022

Kuzma30 commented May 5, 2022

yorikvanhavre commented May 9, 2022

andrii-bodnar commented Jun 8, 2022 • edited

luzpaz commented Jun 8, 2022

Kuzma30 commented Jun 14, 2022

yorikvanhavre commented Jun 14, 2022

Kuzma30 commented Jun 19, 2022

Kuzma30 commented Jun 23, 2022 • edited

chennes commented Jun 23, 2022

Kuzma30 commented Jun 23, 2022

chennes commented Jun 23, 2022

Kuzma30 commented Jun 24, 2022 • edited

Kuzma30 commented Jul 8, 2022

Kuzma30 commented Jul 26, 2022

Kuzma30 commented Apr 23, 2022 •

edited

yorikvanhavre commented Apr 29, 2022 •

edited

Kuzma30 commented Apr 29, 2022 •

edited

Kuzma30 commented May 2, 2022 •

edited

andrii-bodnar commented Jun 8, 2022 •

edited

Kuzma30 commented Jun 23, 2022 •

edited

Kuzma30 commented Jun 24, 2022 •

edited