Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bad translation XLIFF file reconstruction with multi tag file #4953

Open
apourche opened this issue Dec 1, 2020 · 2 comments
Open

Bad translation XLIFF file reconstruction with multi tag file #4953

apourche opened this issue Dec 1, 2020 · 2 comments
Labels
enhancement Adding or requesting a new feature. translate-toolkit Issues which need to be fixed in the translate-toolkit
Projects
File format support

Comments

@apourche
Copy link

apourche commented Dec 1, 2020 •
edited

Describe the bug

The reconstruction of the xliff translation files (Files > Download translation files as ZIP file) produces an incorrect structure due to the evolution of the <file> tags in the source language.

To Reproduce the bug

  1. Import "source string" file :
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" xmlns:scxlf="extXliff">
    <file original="/rep1/myFile1.xml"  source-language="fr" datatype="plaintext">
        <body>
            <trans-unit id="str1">
                <source xml:space="preserve">str1</source>
            </trans-unit>
            <trans-unit id="strA">
                <source xml:space="preserve">strA</source>
            </trans-unit>
        </body>
    </file>
    <file original="/rep1/myFile2.xml"  source-language="fr" datatype="plaintext">
        <body>
            <trans-unit id="str1">
                <source xml:space="preserve">str1</source>
            </trans-unit>
            <trans-unit id="strB">
                <source xml:space="preserve">strB</source>
            </trans-unit>
        </body>
    </file>
</xliff>
  1. Start new translation in EN, and translate all sources
  2. [OK] The export of XLIFF translation file (Files> Download translation files as ZIP file) provides us a correct and full xliff :
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:scxlf="extXliff" version="1.2">
  <file original="/rep1/myFile1.xml" source-language="fr" datatype="plaintext" target-language="en">
    <body>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strA">
        <source xml:space="preserve">strA</source>
        <target>strAEN</target>
      </trans-unit>
    </body>
  </file>
  <file original="/rep1/myFile2.xml" source-language="fr" datatype="plaintext">
    <body>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strB">
        <source xml:space="preserve">strB</source>
        <target>strBEN</target>
      </trans-unit>
    </body>
  </file>
</xliff>
  1. Import the new "source string" file again, which remove '<file original="/rep1/myFile1.xml"...' and wich add <file original="/rep1/myFile3.xml"... :
<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2" xmlns:scxlf="extXliff">
    <file original="/rep1/myFile2.xml"  source-language="fr" datatype="plaintext">
        <body>
            <trans-unit id="str1">
                <source xml:space="preserve">str1</source>
            </trans-unit>
            <trans-unit id="strB">
                <source xml:space="preserve">strB</source>
            </trans-unit>
        </body>
    </file>
    <file original="/rep1/myFile3.xml"  source-language="fr" datatype="plaintext">
        <body>
            <trans-unit id="str1">
                <source xml:space="preserve">str1</source>
            </trans-unit>
            <trans-unit id="strC">
                <source xml:space="preserve">strC</source>
            </trans-unit>
        </body>
    </file>
</xliff>

  1. Translate new strings in existing translation EN

  2. [KO] The export of XLIFF translation file (Files> Download translation files as ZIP file) provides us a bad xliff :

<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:scxlf="extXliff" version="1.2">
  <file original="/rep1/myFile1.xml" source-language="fr" datatype="plaintext" target-language="en">
    <body>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strA">
        <source xml:space="preserve">strA</source>
        <target>strAEN</target>
      </trans-unit>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strC">
        <source xml:space="preserve">strC</source>
        <target>strCEN</target>
      </trans-unit>
    </body>
  </file>
  <file original="/rep1/myFile2.xml" source-language="fr" datatype="plaintext">
    <body>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strB">
        <source xml:space="preserve">strB</source>
        <target>strBEN</target>
      </trans-unit>
    </body>
  </file>
</xliff>

Expected behavior

<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" xmlns:scxlf="extXliff" version="1.2">
  <file original="/rep1/myFile2.xml" source-language="fr" datatype="plaintext">
    <body>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strB">
        <source xml:space="preserve">strB</source>
        <target>strBEN</target>
      </trans-unit>
    </body>
  </file>
  <file original="/rep1/myFile3.xml" source-language="fr" datatype="plaintext" target-language="en">
    <body>
      <trans-unit id="str1">
        <source xml:space="preserve">str1</source>
        <target>str1EN</target>
      </trans-unit>
      <trans-unit id="strC">
        <source xml:space="preserve">strC</source>
        <target>strCEN</target>
      </trans-unit>
    </body>
  </file>
</xliff>

Server configuration and status

  • Weblate: 4.3.2
  • Django: 3.1.3
  • siphashc: 2.1
  • Whoosh: 2.7.4
  • translate-toolkit: 3.2.0
  • lxml: 4.6.1
  • Pillow: 8.0.1
  • bleach: 3.2.1
  • python-dateutil: 2.8.1
  • social-auth-core: 3.3.3
  • social-auth-app-django: 4.0.0
  • django-crispy-forms: 1.9.2
  • oauthlib: 3.1.0
  • django-compressor: 2.4
  • djangorestframework: 3.12.1
  • django-filter: 2.4.0
  • django-appconf: 1.0.4
  • user-agents: 2.2.0
  • filelock: 3.0.12
  • setuptools: 40.8.0
  • jellyfish: 0.8.2
  • openpyxl: 3.0.5
  • celery: 4.4.7
  • kombu: 4.6.11
  • translation-finder: 2.5
  • weblate-language-data: 2020.11
  • html2text: 2020.1.16
  • pycairo: 1.16.2
  • pygobject: 3.30.4
  • diff-match-patch: 20200713
  • requests: 2.24.0
  • django-redis: 4.12.1
  • hiredis: 1.1.0
  • sentry_sdk: 0.19.2
  • Cython: 0.29.21
  • misaka: 2.1.1
  • GitPython: 3.1.11
  • borgbackup: 1.1.14
  • pyparsing: 2.4.7
  • Python: 3.7.3
  • Git: 2.20.1
  • psycopg2: 2.8.6
  • psycopg2-binary: 2.8.6
  • phply: 1.2.5
  • chardet: 3.0.4
  • ruamel.yaml: 0.16.12
  • tesserocr: 2.5.1
  • akismet: 1.1
  • boto3: 1.16.12
  • zeep: 4.0.0
  • aeidon: 1.7.0
  • iniparse: 0.5
  • mysqlclient: 2.0.1
  • Mercurial: 5.6
  • git-svn: 2.20.1
  • git-review: 1.28.0
  • Redis server: 6.0.9
  • PostgreSQL server: 13.1
  • Database backends: django.db.backends.postgresql
  • Cache backends: default:RedisCache, avatar:FileBasedCache
  • Email setup: django.core.mail.backends.smtp.EmailBackend: relay-01.kelis.fr
  • OS encoding: filesystem=utf-8, default=utf-8
  • Celery: redis://cache:6379/1, redis://cache:6379/1, regular
  • Platform: Linux 5.4.0-54-generic (x86_64)

Thank you for your answer
@nijel
Copy link
Member

nijel commented Dec 1, 2020

This has most likely same cause as #2482.

@nijel nijel added enhancement Adding or requesting a new feature. translate-toolkit Issues which need to be fixed in the translate-toolkit labels Dec 1, 2020
@nijel nijel added this to TODO in File format support via automation Dec 1, 2020
@github-actions
Copy link

github-actions bot commented Dec 1, 2020

The issue you've reported needs to be addressed in the translate-toolkit. Please file the issue there, and include links to any relevant specifications about the formats (if applicable).

@apourche apourche mentioned this issue Dec 2, 2020
Open
@jcolladosp jcolladosp mentioned this issue May 17, 2022
Closed
2 tasks
@nijel nijel mentioned this issue May 17, 2022
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Adding or requesting a new feature. translate-toolkit Issues which need to be fixed in the translate-toolkit
Projects
File format support
  
TODO
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nijel @apourche