Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loss of preservation of spaces in xliff target #5026

Closed
apourche opened this issue Dec 12, 2020 · 4 comments · Fixed by translate/translate#4214
Closed

Loss of preservation of spaces in xliff target #5026

apourche opened this issue Dec 12, 2020 · 4 comments · Fixed by translate/translate#4214
Assignees
@nijel
Labels
bug Something is broken. translate-toolkit Issues which need to be fixed in the translate-toolkit
Milestone
4.4.1

Comments

@apourche
Copy link

apourche commented Dec 12, 2020 •
edited

Describe the bug

The xliff export does not preserve line breaks or tabs in <target> fields: the @xml:space="preserve" attribute is absent.

To Reproduce the bug

  1. Create a component with the source :
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
  <file original="file1" source-language="" datatype="plaintext" target-language="fr">
    <body>
      <trans-unit id="unit1">
        <source xml:space="preserve">Test...</source>
      </trans-unit>
    </body>
  </file>
</xliff>

  1. Translate it by adding newlines and tabs :
    bug2

  2. Download translation file return an xliff file in which these characters are lost :

<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2">
  <file original="file1" source-language="" datatype="plaintext" target-language="en-US">
    <body>
      <trans-unit id="unit1">
        <source xml:space="preserve">Test...</source>
        <target>aaa bbb ccc</target>
      </trans-unit>
    </body>
  </file>
</xliff>

Expected behavior

Systematize xml:space="preserve" on tag <target> (or only when it is essential) :

<xliff xmlns="urn:oasis:names:tc:xliff:document:1.2" version="1.2">
  <file original="file1" source-language="" datatype="plaintext" target-language="en-US">
    <body>
      <trans-unit id="unit1">
        <source xml:space="preserve">Test...</source>
        <target  xml:space="preserve">aaa
bbb 
      ccc</target>
      </trans-unit>
    </body>
  </file>
</xliff>

workaround

If the source xliff file contains, <target xml:space="preserve"/>, @xml:space="preserve" is preserved.

Server configuration and status

  • Weblate: 4.3.2
  • Django: 3.1.3
  • siphashc: 2.1
  • Whoosh: 2.7.4
  • translate-toolkit: 3.2.0
  • lxml: 4.6.1
  • Pillow: 8.0.1
  • bleach: 3.2.1
  • python-dateutil: 2.8.1
  • social-auth-core: 3.3.3
  • social-auth-app-django: 4.0.0
  • django-crispy-forms: 1.9.2
  • oauthlib: 3.1.0
  • django-compressor: 2.4
  • djangorestframework: 3.12.1
  • django-filter: 2.4.0
  • django-appconf: 1.0.4
  • user-agents: 2.2.0
  • filelock: 3.0.12
  • setuptools: 40.8.0
  • jellyfish: 0.8.2
  • openpyxl: 3.0.5
  • celery: 4.4.7
  • kombu: 4.6.11
  • translation-finder: 2.5
  • weblate-language-data: 2020.11
  • html2text: 2020.1.16
  • pycairo: 1.16.2
  • pygobject: 3.30.4
  • diff-match-patch: 20200713
  • requests: 2.24.0
  • django-redis: 4.12.1
  • hiredis: 1.1.0
  • sentry_sdk: 0.19.2
  • Cython: 0.29.21
  • misaka: 2.1.1
  • GitPython: 3.1.11
  • borgbackup: 1.1.14
  • pyparsing: 2.4.7
  • Python: 3.7.3
  • Git: 2.20.1
  • psycopg2: 2.8.6
  • psycopg2-binary: 2.8.6
  • phply: 1.2.5
  • chardet: 3.0.4
  • ruamel.yaml: 0.16.12
  • tesserocr: 2.5.1
  • akismet: 1.1
  • boto3: 1.16.12
  • zeep: 4.0.0
  • aeidon: 1.7.0
  • iniparse: 0.5
  • mysqlclient: 2.0.1
  • Mercurial: 5.6
  • git-svn: 2.20.1
  • git-review: 1.28.0
  • Redis server: 6.0.9
  • PostgreSQL server: 13.1
  • Database backends: django.db.backends.postgresql
  • Cache backends: default:RedisCache, avatar:FileBasedCache
  • Email setup: django.core.mail.backends.smtp.EmailBackend: relay-01.kelis.fr
  • OS encoding: filesystem=utf-8, default=utf-8
  • Celery: redis://cache:6379/1, redis://cache:6379/1, regular
  • Platform: Linux 5.4.0-54-generic (x86_64)

Thank you for your answer
@nijel nijel added the bug Something is broken. label Dec 12, 2020
@nijel
Copy link
Member

nijel commented Dec 12, 2020

The whole stack seems to rely on xml:preserve being set on the unit element, but it doesn't enforce it. I'm not confident whether it's more reasonable to handle this in Weblate or in the translate-toolkit...

@nijel nijel added the translate-toolkit Issues which need to be fixed in the translate-toolkit label Dec 12, 2020
@nijel nijel added this to the 4.4 milestone Dec 12, 2020
@github-actions
Copy link

The issue you've reported needs to be addressed in the translate-toolkit. Please file the issue there, and include links to any relevant specifications about the formats (if applicable).

@nijel nijel modified the milestones: 4.4, 4.4.1 Dec 14, 2020
@nijel nijel self-assigned this Dec 22, 2020
nijel added a commit to nijel/translate that referenced this issue Dec 22, 2020
@nijel
xliff: Ensure xml:space="preserve" is set when updating target
a3bfc1e 
Otherwise the whitespace might be lost.

Fixes WeblateOrg/weblate#5026
@nijel nijel mentioned this issue Dec 22, 2020
Merged
@nijel
Copy link
Member

nijel commented Dec 22, 2020

Should be fixed by translate/translate#4214

nijel added a commit to translate/translate that referenced this issue Dec 22, 2020
@nijel
xliff: Ensure xml:space="preserve" is set when updating target
a17c23b 
Otherwise the whitespace might be lost.

Fixes WeblateOrg/weblate#5026
@github-actions
Copy link

Thank you for your report, the issue you have reported has just been fixed.

  • In case you see a problem with the fix, please comment on this issue.
  • In case you see a similar problem, please open a separate issue.
  • If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nijel nijel

Labels
bug Something is broken. translate-toolkit Issues which need to be fixed in the translate-toolkit
Projects
None yet
Milestone
4.4.1
Development

Successfully merging a pull request may close this issue.

xliff: Ensure xml:space="preserve" is set when updating target nijel/translate
2 participants
@nijel @apourche