HTML filter: existing translations are not imported #7929

dleidert · 2022-07-26T13:51:12Z

Describe the issue

I have a few HTML files in multiple languages (a testcase is attached). When I create a component for a file, for which translations already exist, those existing translations are dismissed except if they are the same word/phrase as in the base file, in which case they are marked as "needs editing". Consider these two files main.en.html and main.de.html

<!DOCTYPE html>
<html>
  <head>
    <title>Test</title>
  </head>
  <body>
    <p>I</p>
    <p>am</p>
  </body>
</html>

<!DOCTYPE html>
<html>
  <head>
    <title>Test</title>
  </head>
  <body>
    <p>Ich</p>
    <p>bin</p>
  </body>
</html>

If I import them as a component using the HTML filter, the existing translations for the paragraphs have been dismissed. Instead, I expect them to be there, but maybe marked as "needs editing".

I already tried

I've read and searched the documentation.
I've searched for similar issues in this repository.

Steps to reproduce the behavior

Use the two files shown above, add them to a repository, create a new translation project. For this project, create a component and use the HTML filter and the existing main.*.html files. Check the status of the German translation after finishing the compionent creation. The title string is shown in the user interface and marked as "needs editing". For the two paragraphs, they are shown as untranslated with no German string whatsoever.

Expected behavior

The component should contain three translatable strings, and all strings should already have a German translation in this case. These strings could be marked as "needs editing", but they must not be empty.

Screenshots

Fiilter:

Result:

Translation is missing:

Exception traceback

No response

How do you run Weblate?

Docker container

Weblate versions

$ weblate list_versions
 * Weblate: 4.13.1
 * Django: 4.0.6
 * siphashc: 2.1
 * translate-toolkit: 3.7.1
 * lxml: 4.6.5
 * Pillow: 9.2.0
 * bleach: 5.0.1
 * python-dateutil: 2.8.2
 * social-auth-core: 4.3.0
 * social-auth-app-django: 5.0.0
 * django-crispy-forms: 1.14.0
 * oauthlib: 3.2.0
 * django-compressor: 4.0
 * djangorestframework: 3.13.1
 * django-filter: 22.1
 * django-appconf: 1.0.5
 * user-agents: 2.2.0
 * filelock: 3.7.1
 * jellyfish: 0.9.0
 * openpyxl: 3.0.10
 * celery: 5.2.7
 * kombu: 5.2.4
 * translation-finder: 2.13
 * weblate-language-data: 2022.5
 * html2text: 2020.1.16
 * pycairo: 1.21.0
 * pygobject: 3.42.1
 * diff-match-patch: 20200713
 * requests: 2.28.1
 * django-redis: 5.2.0
 * hiredis: 2.0.0
 * sentry_sdk: 1.6.0
 * Cython: 0.29.30
 * misaka: 2.1.1
 * GitPython: 3.1.27
 * borgbackup: 1.2.1
 * pyparsing: 3.0.9
 * pyahocorasick: 1.4.4
 * python-redis-lock: 3.7.0
 * charset-normalizer: 2.1.0
 * Python: 3.10.5
 * Git: 2.30.2
 * psycopg2: 2.9.3
 * psycopg2-binary: 2.9.3
 * phply: 1.2.5
 * ruamel.yaml: 0.17.21
 * tesserocr: 2.5.2
 * boto3: 1.24.23
 * zeep: 4.1.0
 * aeidon: 1.11
 * iniparse: 0.5
 * mysqlclient: 2.1.1
 * Mercurial: 6.1.4
 * git-svn: 2.30.2
 * git-review: 2.3.1
 * Redis server: 6.2.7
 * PostgreSQL server: 14.4
 * Database backends: django.db.backends.postgresql
 * Cache backends: default:RedisCache, avatar:FileBasedCache
 * Email setup: django.core.mail.backends.smtp.EmailBackend: mail.wgdd.de
 * OS encoding: filesystem=utf-8, default=utf-8
 * Celery: redis://cache:6379/1, redis://cache:6379/1, regular
 * Platform: Linux 5.10.0-14-amd64 (x86_64)

Weblate deploy checks

$ weblate check --deploy
System check identified some issues:

INFOS:
?: (weblate.I021) Error collection is not set up, it is highly recommended for production use
	HINT: https://docs.weblate.org/en/weblate-4.13.1/admin/install.html#collecting-errors

System check identified 1 issue (1 silenced).

Additional context

No response

The text was updated successfully, but these errors were encountered:

nijel · 2022-07-26T19:56:45Z

Might be addressed by #7928

dleidert · 2022-07-26T22:38:30Z

@nijel Thanks for the prompt reply. I'd like to test your changes. Is it enough to apply them inside the docker container, or do I have to restart any of the services running as well?

nijel · 2022-07-27T09:48:58Z

Yes, you need to restart them. The docker container uses supervisor inside.

github-actions · 2022-07-28T08:57:12Z

Thank you for your report; the issue you have reported has just been fixed.

In case you see a problem with the fix, please comment on this issue.
In case you see a similar problem, please open a separate issue.
If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

dleidert · 2022-07-30T11:42:34Z

@nijel This commit does unfortunately not resolve the issue. I applied your changes and restarted the container. The issue is still the same. Can you please test yourself with the testcase I provided?

dleidert · 2022-07-30T11:59:00Z

I created a test-repository: https://github.com/dleidert/weblate-issue-7929-test

nijel · 2022-07-30T18:17:47Z

If you restart the container, any changes in it are discarded

dleidert · 2022-07-30T23:48:46Z

@nijel I adjusted /usr/local/lib/python3.10/site-packages/weblate/formats/convert.py and did docker restart weblate-docker-compose_weblate_1. If I look into convert.py again, my changes are still there.

dleidert · 2022-08-03T10:14:35Z

@nijel Do you have any feedback for me? Were you able to test it yourself with the test-case I supplied? This issue is really a showstopper because we have quite a lot of already translated HTML content and we "loose" it all on import. If this can or is fixed, this would mean a lot to us, and it would spare us that we have to manually use html2po and po2html to deal with translating HTML content.

nijel · 2022-08-10T13:04:44Z

If you do restart the container, any changes are discarded. The easiest approach to test these changes is to use bleeding tag to get the latest changes, see https://docs.weblate.org/en/latest/admin/install/docker.html#choosing-docker-hub-tag

dleidert · 2022-08-15T00:27:20Z

@nijel I tested today with the bleeding edge container. The issue is unfortunately not resolved. When I load the test case (https://github.com/dleidert/weblate-issue-7929-test), the existing translations are missing.

github-actions · 2022-08-15T08:36:13Z

Thank you for your report; the issue you have reported has just been fixed.

In case you see a problem with the fix, please comment on this issue.
In case you see a similar problem, please open a separate issue.
If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

nijel · 2022-08-15T08:37:32Z

Thanks for providing the test files, it should be now fixed for good.

dleidert · 2022-08-16T20:02:20Z

@nijel Tested again. This time it worked. Thanks!

nijel added this to the 4.14 milestone Jul 27, 2022

nijel self-assigned this Jul 27, 2022

nijel added the bug Something is broken. label Jul 27, 2022

nijel linked a pull request Jul 27, 2022 that will close this issue

formats: Fixed parsing of source strings for converted formats #7928

Merged

5 tasks

nijel closed this as completed in #7928 Jul 28, 2022

nijel reopened this Aug 15, 2022

nijel closed this as completed in abedcba Aug 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HTML filter: existing translations are not imported #7929

HTML filter: existing translations are not imported #7929

dleidert commented Jul 26, 2022 •

edited

nijel commented Jul 26, 2022

dleidert commented Jul 26, 2022

nijel commented Jul 27, 2022

github-actions bot commented Jul 28, 2022

dleidert commented Jul 30, 2022

dleidert commented Jul 30, 2022

nijel commented Jul 30, 2022

dleidert commented Jul 30, 2022

dleidert commented Aug 3, 2022

nijel commented Aug 10, 2022

dleidert commented Aug 15, 2022

github-actions bot commented Aug 15, 2022

nijel commented Aug 15, 2022

dleidert commented Aug 16, 2022

HTML filter: existing translations are not imported #7929

HTML filter: existing translations are not imported #7929

Comments

dleidert commented Jul 26, 2022 • edited

Describe the issue

I already tried

Steps to reproduce the behavior

Expected behavior

Screenshots

Exception traceback

How do you run Weblate?

Weblate versions

Weblate deploy checks

Additional context

nijel commented Jul 26, 2022

dleidert commented Jul 26, 2022

nijel commented Jul 27, 2022

github-actions bot commented Jul 28, 2022

dleidert commented Jul 30, 2022

dleidert commented Jul 30, 2022

nijel commented Jul 30, 2022

dleidert commented Jul 30, 2022

dleidert commented Aug 3, 2022

nijel commented Aug 10, 2022

dleidert commented Aug 15, 2022

github-actions bot commented Aug 15, 2022

nijel commented Aug 15, 2022

dleidert commented Aug 16, 2022

dleidert commented Jul 26, 2022 •

edited