Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Translations for strings added in Weblate get reset shortly after creation #8074

Closed
2 tasks done
EndOfTheSpline opened this issue Sep 1, 2022 · 13 comments
Closed
2 tasks done
Assignees
Labels
bug Something is broken.
Milestone

Comments

@EndOfTheSpline
Copy link

Describe the issue

When creating a new string in a component backed by monolingual resx files and translating it into other languages using Weblate, it sometimes (frequently?) resets the translations back to an empty string a while after its creation. The original string is not changed during all of this.

This seems to only happen once after creation, which makes me believe that it might writes the new strings into the base file, then reads the base file, sees those new strings, and gets a bit confused about it.

The remote repository is only written by weblate. There was no change by a developer that did not happen over Weblate. In our setup, we're using Weblate exclusively to manage translations, and the repository itself also only contains translations (plus the minimum boilerplate to use it in our application).

I've taken the liberty to anonymize the project, component and string names, as well as URLs and translations in the logs below.

The complete changes.csv for the day, showing two occurrances (for text1 and text2):

time                                action                          user   "url"                                      target        edit_distance
2022-08-30T13:04:15.746857+00:00  Pushed changes                           /root-component/                                         0
2022-08-30T14:57:11.944802+00:00  New string added                  UserA  /child-component/de_CH/text1               de-TEXT1      7
2022-08-30T14:57:11.944871+00:00  New string added                  UserA  /child-component/fr_CH/text1                             0
2022-08-30T14:57:11.944912+00:00  New string added                  UserA  /child-component/it_CH/text1                             0
2022-08-30T14:57:11.946517+00:00  New string to translate           UserA  /child-component/de_CH/?q=is:untranslated                0
2022-08-30T14:57:17.450355+00:00  String updated in the repository         /child-component/de_CH/text4               de-TEXT4      0
2022-08-30T14:57:17.450421+00:00  String updated in the repository         /child-component/de_CH/text1               de-TEXT1      0
2022-08-30T14:57:17.450460+00:00  Resource update                          /child-component/de_CH/                                  0
2022-08-30T14:57:17.522385+00:00  Resource update                          /child-component/fr_CH/                                  0
2022-08-30T14:57:17.589576+00:00  Resource update                          /child-component/it_CH/                                  0
2022-08-30T14:57:21.908783+00:00  Pushed changes                           /root-component/                                         0
2022-08-30T14:57:40.196101+00:00  New translation                   UserA  /child-component/it_CH/text1               it-TEXT1      11
2022-08-30T14:57:57.679158+00:00  New translation                   UserA  /child-component/fr_CH/text1               fr-TEXT1      7
2022-08-30T14:59:04.773872+00:00  New string added                  UserA  /child-component/de_CH/text2               de-TEXT2      15
2022-08-30T14:59:04.773941+00:00  New string added                  UserA  /child-component/fr_CH/text2                             0
2022-08-30T14:59:04.773983+00:00  New string added                  UserA  /child-component/it_CH/text2                             0
2022-08-30T14:59:04.775669+00:00  New string to translate           UserA  /child-component/de_CH/?q=is:untranslated                0
2022-08-30T14:59:08.106882+00:00  String updated in the repository         /child-component/de_CH/text2               de-TEXT2      0
2022-08-30T14:59:08.106953+00:00  Resource update                          /child-component/de_CH/                                  0
2022-08-30T14:59:08.182720+00:00  String updated in the repository         /child-component/fr_CH/text1                             7
2022-08-30T14:59:08.182791+00:00  Resource update                          /child-component/fr_CH/                                  0
2022-08-30T14:59:08.251645+00:00  String updated in the repository         /child-component/it_CH/text1                             11
2022-08-30T14:59:08.251712+00:00  Resource update                          /child-component/it_CH/                                  0
2022-08-30T14:59:08.262374+00:00  New string to translate                  /child-component/fr_CH/?q=is:untranslated                0
2022-08-30T14:59:08.263300+00:00  New string to translate                  /child-component/it_CH/?q=is:untranslated                0
2022-08-30T14:59:12.658624+00:00  Pushed changes                           /root-component/                                         0
2022-08-30T14:59:22.257753+00:00  New translation                   UserA  /child-component/fr_CH/text2                fr-TEXT2     14
2022-08-30T14:59:37.657320+00:00  New translation                   UserA  /child-component/it_CH/text2                it-TEXT2     15
2022-08-30T15:00:02.129416+00:00  New string added                  UserA  /child-component/de_CH/text3                de-TEXT3     13
2022-08-30T15:00:02.129482+00:00  New string added                  UserA  /child-component/fr_CH/text3                             0
2022-08-30T15:00:02.129524+00:00  New string added                  UserA  /child-component/it_CH/text3                             0
2022-08-30T15:00:02.130972+00:00  New string to translate           UserA  /child-component/de_CH/?q=is:untranslated                0
2022-08-30T15:00:11.942827+00:00  String updated in the repository         /child-component/de_CH/text3                de-TEXT3     0
2022-08-30T15:00:11.942898+00:00  Resource update                          /child-component/de_CH/                                  0
2022-08-30T15:00:12.018783+00:00  String updated in the repository         /child-component/fr_CH/text2                             14
2022-08-30T15:00:12.018852+00:00  Resource update                          /child-component/fr_CH/                                  0
2022-08-30T15:00:12.095705+00:00  String updated in the repository         /child-component/it_CH/text2                             15

If we take for example the first string:

UserA creates the string:

2022-08-30T14:57:11.944802+00:00  New string added                  UserA  /child-component/de_CH/text1               de-TEXT1      7
2022-08-30T14:57:11.944871+00:00  New string added                  UserA  /child-component/fr_CH/text1                             0
2022-08-30T14:57:11.944912+00:00  New string added                  UserA  /child-component/it_CH/text1                             0
2022-08-30T14:57:11.946517+00:00  New string to translate           UserA  /child-component/de_CH/?q=is:untranslated                0
2022-08-30T14:57:17.450355+00:00  String updated in the repository         /child-component/de_CH/text4               de-TEXT4      0
2022-08-30T14:57:17.450421+00:00  String updated in the repository         /child-component/de_CH/text1               de-TEXT1      0
2022-08-30T14:57:17.450460+00:00  Resource update                          /child-component/de_CH/                                  0
2022-08-30T14:57:17.522385+00:00  Resource update                          /child-component/fr_CH/                                  0
2022-08-30T14:57:17.589576+00:00  Resource update                          /child-component/it_CH/                                  0
2022-08-30T14:57:21.908783+00:00  Pushed changes                           /root-component/                                         0

UserA adds the translations for the string:

2022-08-30T14:57:40.196101+00:00  New translation                   UserA  /child-component/it_CH/text1               it-TEXT1      11
2022-08-30T14:57:57.679158+00:00  New translation                   UserA  /child-component/fr_CH/text1               fr-TEXT1      7

Weblate wipes the string:

2022-08-30T14:59:08.106953+00:00  Resource update                          /child-component/de_CH/                                  0
2022-08-30T14:59:08.182720+00:00  String updated in the repository         /child-component/fr_CH/text1                             7
2022-08-30T14:59:08.182791+00:00  Resource update                          /child-component/fr_CH/                                  0
2022-08-30T14:59:08.251645+00:00  String updated in the repository         /child-component/it_CH/text1                             11
2022-08-30T14:59:08.251712+00:00  Resource update                          /child-component/it_CH/                                  0

WEBLATE_LOGLEVEL is set to DEBUG, but there's nothing that I could find that explains the behaviour. All log times here are UTC+2 (... for some reason). These are the docker logs, minus heartbeats, from 14:57:16Z until 15:00:16Z

gunicorn stderr | [2022-08-30 16:57:16,999: DEBUG/298] project/root-component: running post_commit add-on: weblate.git.squash
gunicorn stderr | [2022-08-30 16:57:17,219: INFO/298] project/root-component: checking Repository/RootComponent/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:57:17,231: INFO/298] project/root-component: checking Repository/RootComponent/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:57:17,239: INFO/298] project/root-component: checking Repository/RootComponent/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:57:17,247: INFO/298] project/root-component: checking triggered alerts
gunicorn stderr | [2022-08-30 16:57:17,251: INFO/298] project/root-component: updating linked project project/child-component-3 [1/4]
gunicorn stderr | [2022-08-30 16:57:17,254: INFO/298] project/child-component-3: checking Repository/child-component-3/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:57:17,263: INFO/298] project/child-component-3: checking Repository/child-component-3/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:57:17,270: INFO/298] project/child-component-3: checking Repository/child-component-3/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:57:17,278: INFO/298] project/child-component-3: checking triggered alerts
gunicorn stderr | [2022-08-30 16:57:17,283: INFO/298] project/child-component-3: updating completed
gunicorn stderr | [2022-08-30 16:57:17,284: INFO/298] project/root-component: updating linked project project/child-component-2 [2/4]
gunicorn stderr | [2022-08-30 16:57:17,287: INFO/298] project/child-component-2: checking Repository/child-component-2/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:57:17,296: INFO/298] project/child-component-2: checking Repository/child-component-2/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:57:17,304: INFO/298] project/child-component-2: checking Repository/child-component-2/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:57:17,314: INFO/298] project/child-component-2: checking triggered alerts
gunicorn stderr | [2022-08-30 16:57:17,319: INFO/298] project/child-component-2: updating completed
gunicorn stderr | [2022-08-30 16:57:17,320: INFO/298] project/root-component: updating linked project project/child-component-4 [3/4]
gunicorn stderr | [2022-08-30 16:57:17,323: INFO/298] project/child-component-4: checking Repository/child-component-4/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:57:17,336: INFO/298] project/child-component-4: checking Repository/child-component-4/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:57:17,344: INFO/298] project/child-component-4: checking Repository/child-component-4/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:57:17,352: INFO/298] project/child-component-4: checking triggered alerts
gunicorn stderr | [2022-08-30 16:57:17,358: INFO/298] project/child-component-4: updating completed
gunicorn stderr | [2022-08-30 16:57:17,359: INFO/298] project/root-component: updating linked project project/child-component [4/4]
gunicorn stderr | [2022-08-30 16:57:17,362: INFO/298] project/child-component: checking Repository/ChildComponent/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:57:17,377: INFO/298] project/child-component/de_CH: processing Repository/ChildComponent/Resources.resx, content changed, 162 strings
gunicorn stderr | [2022-08-30 16:57:17,451: INFO/298] project/child-component/de_CH: updating completed
gunicorn stderr | [2022-08-30 16:57:17,453: INFO/298] project/child-component: checking Repository/ChildComponent/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:57:17,469: INFO/298] project/child-component/fr_CH: processing Repository/ChildComponent/Resources.fr-ch.resx, content changed, 162 strings
gunicorn stderr | [2022-08-30 16:57:17,523: INFO/298] project/child-component/fr_CH: updating completed
gunicorn stderr | [2022-08-30 16:57:17,524: INFO/298] project/child-component: checking Repository/ChildComponent/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:57:17,535: INFO/298] project/child-component/it_CH: processing Repository/ChildComponent/Resources.it-ch.resx, content changed, 162 strings
gunicorn stderr | [2022-08-30 16:57:17,590: INFO/298] project/child-component/it_CH: updating completed
gunicorn stderr | [2022-08-30 16:57:17,592: INFO/298] project/child-component: checking triggered alerts
gunicorn stderr | [2022-08-30 16:57:17,597: INFO/298] project/child-component: running source checks for 2 strings
gunicorn stderr | [2022-08-30 16:57:17,602: INFO/298] project/child-component: updating completed
gunicorn stderr | [2022-08-30 16:57:17,604: INFO/298] project/root-component: updating completed
gunicorn stderr | [2022-08-30 16:57:17,609: INFO/298] project/root-component: scheduling push
celery-celery stderr | [2022-08-30 16:57:17,614: INFO/MainProcess] Task weblate.trans.tasks.perform_push[7a31582c-63af-4f66-973d-aa7d168d69b0] received
gunicorn stderr | [2022-08-30 16:57:17,653: INFO/298] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:57:17,657: INFO/ForkPoolWorker-1] Got lock for 'lock:lock:repo:1'.
celery-celery stderr | [2022-08-30 16:57:17,663: INFO/322] project/root-component: updating repository
celery-celery stderr | [2022-08-30 16:57:17,663: INFO/ForkPoolWorker-1] project/root-component: updating repository
celery-celery stderr | [2022-08-30 16:57:17,667: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[89f1e3a4-1348-4898-8f8b-a5cbf8f0e81c] received
celery-celery stderr | [2022-08-30 16:57:17,687: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[ed3a8f66-7f71-44ea-b6f8-647022175fa6] received
gunicorn stderr | [2022-08-30 16:57:17,688: INFO/298] project/root-component: updating stats caches
celery-celery stderr | [2022-08-30 16:57:17,719: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[f1363a3c-8fdd-4d94-bbe0-8ff39f43dfe2] received
celery-celery stderr | [2022-08-30 16:57:17,731: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[cbb0434e-608e-492e-84ed-b0947973e35f] received
celery-celery stderr | [2022-08-30 16:57:19,940: INFO/322] project/root-component: update took 2.27 seconds
celery-celery stderr | [2022-08-30 16:57:19,940: INFO/ForkPoolWorker-1] project/root-component: update took 2.27 seconds
celery-celery stderr | [2022-08-30 16:57:19,949: INFO/322] project/root-component: repository up to date at 1553f770aeb582612539f63f341befcaa38c9171
celery-celery stderr | [2022-08-30 16:57:19,949: INFO/ForkPoolWorker-1] project/root-component: repository up to date at 1553f770aeb582612539f63f341befcaa38c9171
celery-celery stderr | [2022-08-30 16:57:20,043: INFO/ForkPoolWorker-1] Got lock for 'lock:lock:repo:1'.
celery-celery stderr | [2022-08-30 16:57:20,044: INFO/322] project/root-component: pushing to remote repo
celery-celery stderr | [2022-08-30 16:57:20,044: INFO/ForkPoolWorker-1] project/root-component: pushing to remote repo
celery-notify stderr | [2022-08-30 16:57:21,917: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[6f31c228-5db7-4843-9c6a-b8e61212de3e] received
gunicorn stderr | [2022-08-30 16:57:30,235: INFO/299] received Azure notification on repository Repository, URL https://remote-repos//_git/Repository, branch weblate, 0 matching components, 0 to process, 0 linked
celery-notify stderr | [2022-08-30 16:57:40,208: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[e94a51ec-58de-4455-8bcd-e0f8a400cde7] received
celery-memory stderr | [2022-08-30 16:57:40,222: INFO/MainProcess] Task weblate.memory.tasks.handle_unit_translation_change[71539556-fa51-4138-b812-255c90b84c87] received
celery-notify stderr | [2022-08-30 16:57:57,689: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[2a2206df-a633-4725-ad81-4b36e44d650f] received
celery-memory stderr | [2022-08-30 16:57:57,705: INFO/MainProcess] Task weblate.memory.tasks.handle_unit_translation_change[137a871b-87cc-47ff-a12d-c12415777bce] received
celery-celery stderr | [2022-08-30 16:59:04,791: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[03526cb0-6d1c-4597-8063-6da2e65c9803] received
gunicorn stderr | [2022-08-30 16:59:04,801: INFO/299] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:04,820: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[576cc18a-410b-4819-9e36-9775c09e8bd0] received
celery-notify stderr | [2022-08-30 16:59:04,834: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[0953808c-34e6-4fec-93de-6340eccd9818] received
gunicorn stderr | [2022-08-30 16:59:07,799: INFO/298] project/child-component/de_CH: committing pending changes (pending unit)
gunicorn stderr | [2022-08-30 16:59:07,847: DEBUG/298] project/root-component: running post_commit add-on: weblate.git.squash
gunicorn stderr | [2022-08-30 16:59:07,895: INFO/298] project/root-component: checking Repository/RootComponent/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:59:07,905: INFO/298] project/root-component: checking Repository/RootComponent/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:59:07,912: INFO/298] project/root-component: checking Repository/RootComponent/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:59:07,919: INFO/298] project/root-component: checking triggered alerts
gunicorn stderr | [2022-08-30 16:59:07,923: INFO/298] project/root-component: updating linked project project/child-component-3 [1/4]
gunicorn stderr | [2022-08-30 16:59:07,925: INFO/298] project/child-component-3: checking Repository/child-component-3/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:59:07,934: INFO/298] project/child-component-3: checking Repository/child-component-3/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:59:07,940: INFO/298] project/child-component-3: checking Repository/child-component-3/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:59:07,947: INFO/298] project/child-component-3: checking triggered alerts
gunicorn stderr | [2022-08-30 16:59:07,953: INFO/298] project/child-component-3: updating completed
gunicorn stderr | [2022-08-30 16:59:07,954: INFO/298] project/root-component: updating linked project project/child-component-2 [2/4]
gunicorn stderr | [2022-08-30 16:59:07,956: INFO/298] project/child-component-2: checking Repository/child-component-2/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:59:07,964: INFO/298] project/child-component-2: checking Repository/child-component-2/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:59:07,971: INFO/298] project/child-component-2: checking Repository/child-component-2/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:59:07,979: INFO/298] project/child-component-2: checking triggered alerts
gunicorn stderr | [2022-08-30 16:59:07,984: INFO/298] project/child-component-2: updating completed
gunicorn stderr | [2022-08-30 16:59:07,985: INFO/298] project/root-component: updating linked project project/child-component-4 [3/4]
gunicorn stderr | [2022-08-30 16:59:07,987: INFO/298] project/child-component-4: checking Repository/child-component-4/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:59:08,000: INFO/298] project/child-component-4: checking Repository/child-component-4/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:59:08,007: INFO/298] project/child-component-4: checking Repository/child-component-4/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:59:08,015: INFO/298] project/child-component-4: checking triggered alerts
gunicorn stderr | [2022-08-30 16:59:08,020: INFO/298] project/child-component-4: updating completed
gunicorn stderr | [2022-08-30 16:59:08,021: INFO/298] project/root-component: updating linked project project/child-component [4/4]
gunicorn stderr | [2022-08-30 16:59:08,024: INFO/298] project/child-component: checking Repository/ChildComponent/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 16:59:08,040: INFO/298] project/child-component/de_CH: processing Repository/ChildComponent/Resources.resx, content changed, 163 strings
gunicorn stderr | [2022-08-30 16:59:08,108: INFO/298] project/child-component/de_CH: updating completed
gunicorn stderr | [2022-08-30 16:59:08,109: INFO/298] project/child-component: checking Repository/ChildComponent/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 16:59:08,124: INFO/298] project/child-component/fr_CH: processing Repository/ChildComponent/Resources.fr-ch.resx, content changed, 163 strings
gunicorn stderr | [2022-08-30 16:59:08,183: INFO/298] project/child-component/fr_CH: updating completed
gunicorn stderr | [2022-08-30 16:59:08,185: INFO/298] project/child-component: checking Repository/ChildComponent/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 16:59:08,196: INFO/298] project/child-component/it_CH: processing Repository/ChildComponent/Resources.it-ch.resx, content changed, 163 strings
gunicorn stderr | [2022-08-30 16:59:08,252: INFO/298] project/child-component/it_CH: updating completed
gunicorn stderr | [2022-08-30 16:59:08,254: INFO/298] project/child-component: checking triggered alerts
gunicorn stderr | [2022-08-30 16:59:08,259: INFO/298] project/child-component: running source checks for 1 strings
gunicorn stderr | [2022-08-30 16:59:08,265: INFO/298] project/child-component: updating completed
gunicorn stderr | [2022-08-30 16:59:08,266: INFO/298] project/root-component: updating completed
gunicorn stderr | [2022-08-30 16:59:08,271: INFO/298] project/root-component: scheduling push
celery-celery stderr | [2022-08-30 16:59:08,274: INFO/MainProcess] Task weblate.trans.tasks.perform_push[296b93a5-dd4b-4081-b4fa-5fded85e3e0d] received
gunicorn stderr | [2022-08-30 16:59:08,315: INFO/298] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:08,317: INFO/ForkPoolWorker-1] Got lock for 'lock:lock:repo:1'.
celery-celery stderr | [2022-08-30 16:59:08,326: INFO/322] project/root-component: updating repository
celery-celery stderr | [2022-08-30 16:59:08,326: INFO/ForkPoolWorker-1] project/root-component: updating repository
celery-celery stderr | [2022-08-30 16:59:08,331: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[e34d178a-c8f1-4966-acc2-4539b3d124f6] received
celery-notify stderr | [2022-08-30 16:59:08,342: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[35420a52-43ec-47c2-b62d-2e31fca4c61b] received
celery-notify stderr | [2022-08-30 16:59:08,355: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[bb58f124-bc3f-4035-8b3e-a8fcb5a0ee80] received
celery-celery stderr | [2022-08-30 16:59:08,377: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[cafb0070-2f82-4d80-9cbd-fc76b8dc4f23] received
celery-celery stderr | [2022-08-30 16:59:08,392: INFO/MainProcess] Task weblate.checks.tasks.batch_update_checks[900c6169-9cf4-4676-a7f6-771d49d5e7a7] received
gunicorn stderr | [2022-08-30 16:59:08,394: INFO/298] project/root-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:08,446: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[363741f4-e7f8-4fa5-b689-5917e22bd893] received
celery-celery stderr | [2022-08-30 16:59:08,462: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[a093a17d-9597-4f47-8598-f18ec811f6a0] received
celery-celery stderr | [2022-08-30 16:59:09,042: INFO/327] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:09,042: INFO/ForkPoolWorker-2] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:09,057: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[fccc9293-fe5f-481f-afa3-28bdcca49ab2] received
celery-celery stderr | [2022-08-30 16:59:09,104: INFO/327] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:09,104: INFO/ForkPoolWorker-2] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 16:59:09,115: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[1d026d71-f271-4147-bdaf-f714b23cda6a] received
celery-celery stderr | [2022-08-30 16:59:11,080: INFO/322] project/root-component: update took 2.75 seconds
celery-celery stderr | [2022-08-30 16:59:11,080: INFO/ForkPoolWorker-1] project/root-component: update took 2.75 seconds
celery-celery stderr | [2022-08-30 16:59:11,085: INFO/322] project/root-component: repository up to date at 1553f770aeb582612539f63f341befcaa38c9171
celery-celery stderr | [2022-08-30 16:59:11,085: INFO/ForkPoolWorker-1] project/root-component: repository up to date at 1553f770aeb582612539f63f341befcaa38c9171
celery-celery stderr | [2022-08-30 16:59:11,148: INFO/ForkPoolWorker-1] Got lock for 'lock:lock:repo:1'.
celery-celery stderr | [2022-08-30 16:59:11,149: INFO/322] project/root-component: pushing to remote repo
celery-celery stderr | [2022-08-30 16:59:11,149: INFO/ForkPoolWorker-1] project/root-component: pushing to remote repo
celery-notify stderr | [2022-08-30 16:59:12,667: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[80a95c96-83cc-452f-9409-4dbe38f20591] received
gunicorn stderr | [2022-08-30 16:59:20,501: INFO/299] received Azure notification on repository Repository, URL https://remote-repos//_git/Repository, branch weblate, 0 matching components, 0 to process, 0 linked
celery-notify stderr | [2022-08-30 16:59:22,270: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[cb86d3de-8dd0-4116-bee2-56cc480397f2] received
celery-memory stderr | [2022-08-30 16:59:22,284: INFO/MainProcess] Task weblate.memory.tasks.handle_unit_translation_change[623fe77b-0ab1-49ad-8d96-3c5c639f3344] received
celery-notify stderr | [2022-08-30 16:59:37,668: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[4f12efde-9ed6-430c-99e5-d65eb0d78866] received
celery-memory stderr | [2022-08-30 16:59:37,681: INFO/MainProcess] Task weblate.memory.tasks.handle_unit_translation_change[e1e5572e-376d-4a91-aeb7-7b1bd2be9c70] received
celery-celery stderr | [2022-08-30 17:00:02,146: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[2e10e29f-fa9b-4e38-8df5-3d8151ac714b] received
gunicorn stderr | [2022-08-30 17:00:02,156: INFO/300] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 17:00:02,166: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[aa6daee7-077c-4e83-bb78-221032073f6e] received
celery-notify stderr | [2022-08-30 17:00:02,186: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[1895f228-c6b2-475e-8f5d-c7aa8a7638a9] received
gunicorn stderr | [2022-08-30 17:00:11,633: INFO/300] project/child-component/de_CH: committing pending changes (pending unit)
gunicorn stderr | [2022-08-30 17:00:11,678: DEBUG/300] project/root-component: running post_commit add-on: weblate.git.squash
gunicorn stderr | [2022-08-30 17:00:11,727: INFO/300] project/root-component: checking Repository/RootComponent/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 17:00:11,737: INFO/300] project/root-component: checking Repository/RootComponent/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 17:00:11,745: INFO/300] project/root-component: checking Repository/RootComponent/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 17:00:11,755: INFO/300] project/root-component: checking triggered alerts
gunicorn stderr | [2022-08-30 17:00:11,759: INFO/300] project/root-component: updating linked project project/child-component-3 [1/4]
gunicorn stderr | [2022-08-30 17:00:11,761: INFO/300] project/child-component-3: checking Repository/child-component-3/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 17:00:11,770: INFO/300] project/child-component-3: checking Repository/child-component-3/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 17:00:11,777: INFO/300] project/child-component-3: checking Repository/child-component-3/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 17:00:11,785: INFO/300] project/child-component-3: checking triggered alerts
gunicorn stderr | [2022-08-30 17:00:11,790: INFO/300] project/child-component-3: updating completed
gunicorn stderr | [2022-08-30 17:00:11,792: INFO/300] project/root-component: updating linked project project/child-component-2 [2/4]
gunicorn stderr | [2022-08-30 17:00:11,795: INFO/300] project/child-component-2: checking Repository/child-component-2/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 17:00:11,803: INFO/300] project/child-component-2: checking Repository/child-component-2/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 17:00:11,810: INFO/300] project/child-component-2: checking Repository/child-component-2/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 17:00:11,818: INFO/300] project/child-component-2: checking triggered alerts
gunicorn stderr | [2022-08-30 17:00:11,823: INFO/300] project/child-component-2: updating completed
gunicorn stderr | [2022-08-30 17:00:11,825: INFO/300] project/root-component: updating linked project project/child-component-4 [3/4]
gunicorn stderr | [2022-08-30 17:00:11,827: INFO/300] project/child-component-4: checking Repository/child-component-4/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 17:00:11,840: INFO/300] project/child-component-4: checking Repository/child-component-4/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 17:00:11,847: INFO/300] project/child-component-4: checking Repository/child-component-4/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 17:00:11,855: INFO/300] project/child-component-4: checking triggered alerts
gunicorn stderr | [2022-08-30 17:00:11,860: INFO/300] project/child-component-4: updating completed
gunicorn stderr | [2022-08-30 17:00:11,862: INFO/300] project/root-component: updating linked project project/child-component [4/4]
gunicorn stderr | [2022-08-30 17:00:11,864: INFO/300] project/child-component: checking Repository/ChildComponent/Resources.resx (de_CH) [1/3]
gunicorn stderr | [2022-08-30 17:00:11,883: INFO/300] project/child-component/de_CH: processing Repository/ChildComponent/Resources.resx, content changed, 164 strings
gunicorn stderr | [2022-08-30 17:00:11,944: INFO/300] project/child-component/de_CH: updating completed
gunicorn stderr | [2022-08-30 17:00:11,945: INFO/300] project/child-component: checking Repository/ChildComponent/Resources.fr-ch.resx (fr-ch) [2/3]
gunicorn stderr | [2022-08-30 17:00:11,960: INFO/300] project/child-component/fr_CH: processing Repository/ChildComponent/Resources.fr-ch.resx, content changed, 164 strings
gunicorn stderr | [2022-08-30 17:00:12,019: INFO/300] project/child-component/fr_CH: updating completed
gunicorn stderr | [2022-08-30 17:00:12,021: INFO/300] project/child-component: checking Repository/ChildComponent/Resources.it-ch.resx (it-ch) [3/3]
gunicorn stderr | [2022-08-30 17:00:12,032: INFO/300] project/child-component/it_CH: processing Repository/ChildComponent/Resources.it-ch.resx, content changed, 164 strings
gunicorn stderr | [2022-08-30 17:00:12,096: INFO/300] project/child-component/it_CH: updating completed
gunicorn stderr | [2022-08-30 17:00:12,098: INFO/300] project/child-component: checking triggered alerts
gunicorn stderr | [2022-08-30 17:00:12,104: INFO/300] project/child-component: running source checks for 1 strings
gunicorn stderr | [2022-08-30 17:00:12,108: INFO/300] project/child-component: updating completed
gunicorn stderr | [2022-08-30 17:00:12,110: INFO/300] project/root-component: updating completed
gunicorn stderr | [2022-08-30 17:00:12,115: INFO/300] project/root-component: scheduling push
celery-celery stderr | [2022-08-30 17:00:12,118: INFO/MainProcess] Task weblate.trans.tasks.perform_push[42c56307-48aa-47d6-b152-e138b48fa581] received
gunicorn stderr | [2022-08-30 17:00:12,149: INFO/300] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 17:00:12,164: INFO/ForkPoolWorker-1] Got lock for 'lock:lock:repo:1'.
celery-celery stderr | [2022-08-30 17:00:12,169: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[4a2827aa-3b71-475d-a971-b8641fc31315] received
celery-notify stderr | [2022-08-30 17:00:12,178: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[41433c3b-f1f0-4dba-9dad-ac4fed19487f] received
celery-notify stderr | [2022-08-30 17:00:12,195: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[d0c42e68-f236-447c-8990-945041d69bfa] received
celery-celery stderr | [2022-08-30 17:00:12,198: INFO/322] project/root-component: updating repository
celery-celery stderr | [2022-08-30 17:00:12,198: INFO/ForkPoolWorker-1] project/root-component: updating repository
celery-celery stderr | [2022-08-30 17:00:12,219: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[d04dcf17-e107-4dec-baf9-2d2ec921aead] received
celery-celery stderr | [2022-08-30 17:00:12,227: INFO/MainProcess] Task weblate.checks.tasks.batch_update_checks[220a2e01-67e5-47f4-9ee5-f3f69d6f2f3c] received
gunicorn stderr | [2022-08-30 17:00:12,233: INFO/300] project/root-component: updating stats caches
celery-celery stderr | [2022-08-30 17:00:12,273: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[df6ddae2-ffb8-4ca1-8fb6-9bb45d7db206] received
celery-celery stderr | [2022-08-30 17:00:12,298: INFO/MainProcess] Task weblate.glossary.tasks.sync_glossary_languages[e8993590-2c68-4731-9783-8f30165e7389] received
celery-celery stderr | [2022-08-30 17:00:13,006: INFO/327] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 17:00:13,006: INFO/ForkPoolWorker-2] project/child-component: updating stats caches
celery-celery stderr | [2022-08-30 17:00:13,024: INFO/MainProcess] Task weblate.trans.tasks.update_component_stats[13a60297-8f08-471a-b8ef-7c78a0dd188a] received
celery-celery stderr | [2022-08-30 17:00:14,505: INFO/322] project/root-component: update took 2.31 seconds
celery-celery stderr | [2022-08-30 17:00:14,505: INFO/ForkPoolWorker-1] project/root-component: update took 2.31 seconds
celery-celery stderr | [2022-08-30 17:00:14,510: INFO/322] project/root-component: repository up to date at 1553f770aeb582612539f63f341befcaa38c9171
celery-celery stderr | [2022-08-30 17:00:14,510: INFO/ForkPoolWorker-1] project/root-component: repository up to date at 1553f770aeb582612539f63f341befcaa38c9171
celery-celery stderr | [2022-08-30 17:00:14,573: INFO/ForkPoolWorker-1] Got lock for 'lock:lock:repo:1'.
celery-celery stderr | [2022-08-30 17:00:14,575: INFO/322] project/root-component: pushing to remote repo
celery-celery stderr | [2022-08-30 17:00:14,575: INFO/ForkPoolWorker-1] project/root-component: pushing to remote repo
celery-notify stderr | [2022-08-30 17:00:15,970: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[7f4fbea4-571d-4171-9c7c-46c62e98fc65] received
celery-notify stderr | [2022-08-30 17:00:16,595: INFO/MainProcess] Task weblate.accounts.tasks.notify_change[40f5e345-8a06-47dc-83c1-432eee449b29] received
celery-memory stderr | [2022-08-30 17:00:16,610: INFO/MainProcess] Task weblate.memory.tasks.handle_unit_translation_change[8e83bf10-0c7b-442a-b7e5-21bcd7fb8096] received

I already tried

  • I've read and searched the documentation.
  • I've searched for similar issues in this repository.

Steps to reproduce the behavior

This is the steps to re-create our setup - because it's occurring all the time, but not really reproducible on-demand:

  1. Create a new Git repository that contains multiple folders, each folder containing a Resources.resx, Resources.fr-ch.resx, and Resources.it-ch.resx.
  2. Set up a component using this remote Git repository, using de-CH as base language. Set it up to use one of the folders as base. It should fetch the master-branch, and push its changes to a weblate branch. The issue occurred with or without the Git squash add-on; with or without the "Update RESX files" add-on.
  3. Create one (or multiple) linked components that use the repository from the first component, but a different folder.
  4. Add a new string over the REST API or Web interface for de-CH.
  5. Add translations for fr-CH and it-CH for the newly created string immediately after creating it.
  6. Wait anything from five minutes to a few hours.
  7. The fr-CH and it-CH translations have been overwritten by Weblate with empty strings.

Expected behavior

Weblate does not reset translations by itself.

Screenshots

image

Exception traceback

No response

How do you run Weblate?

Docker container

Weblate versions

  • Weblate: 4.14
  • Django: 4.1
  • siphashc: 2.1
  • translate-toolkit: 3.7.2
  • lxml: 4.9.1
  • Pillow: 9.2.0
  • bleach: 5.0.1
  • python-dateutil: 2.8.2
  • social-auth-core: 4.3.0
  • social-auth-app-django: 5.0.0
  • django-crispy-forms: 1.14.0
  • oauthlib: 3.2.0
  • django-compressor: 4.1
  • djangorestframework: 3.13.1
  • django-filter: 22.1
  • django-appconf: 1.0.5
  • user-agents: 2.2.0
  • filelock: 3.8.0
  • rapidfuzz: 2.6.0
  • openpyxl: 3.0.10
  • celery: 5.2.7
  • kombu: 5.2.4
  • translation-finder: 2.14
  • weblate-language-data: 2022.7
  • html2text: 2020.1.16
  • pycairo: 1.21.0
  • pygobject: 3.42.2
  • diff-match-patch: 20200713
  • requests: 2.28.1
  • django-redis: 5.2.0
  • hiredis: 2.0.0
  • sentry_sdk: 1.9.5
  • Cython: 0.29.32
  • misaka: 2.1.1
  • GitPython: 3.1.27
  • borgbackup: 1.2.2
  • pyparsing: 3.0.9
  • pyahocorasick: 1.4.4
  • python-redis-lock: 3.7.0
  • charset-normalizer: 2.1.1
  • Python: 3.10.6
  • Git: 2.30.2
  • psycopg2: 2.9.3
  • psycopg2-binary: 2.9.3
  • phply: 1.2.5
  • ruamel.yaml: 0.17.21
  • tesserocr: 2.5.2
  • boto3: 1.24.57
  • zeep: 4.1.0
  • aeidon: 1.11
  • iniparse: 0.5
  • mysqlclient: 2.1.1
  • Mercurial: 6.2.1
  • git-svn: 2.30.2
  • git-review: 2.3.1
  • Redis server: 6.2.7
  • PostgreSQL server: 14.2
  • Database backends: django.db.backends.postgresql
  • Cache backends: default:RedisCache, avatar:FileBasedCache
  • Email setup: django.core.mail.backends.smtp.EmailBackend: 127.0.0.1
  • OS encoding: filesystem=utf-8, default=utf-8
  • Celery: redis://redis:6379/1, redis://redis:6379/1, regular
  • Platform: Linux 5.15.0-1014-azure (x86_64)

Weblate deploy checks

No response

Additional context

No response

@nijel
Copy link
Member

nijel commented Sep 5, 2022

Have you checked whether the translation is written to the file?

@EndOfTheSpline
Copy link
Author

I can't remember if we ever had a look between "translations are created" and "entries have been reset", but after the entires have been reset, only the main file (Resources.resx) contains the new strings. Resources.fr-ch.resx and Resources-it.ch.resx do not contain the new strings.

It's how we found out about the issue, mostly - by having PRs that weren't "balanced" (i.e. had multiple new strings in the main language, but none in the translations, which given our workflow shouldn't happen).

@nijel
Copy link
Member

nijel commented Sep 6, 2022

The string should be added to the translation files once it is translated.

Going again through the original report, are you confident that it happens without the resx update add-on? To me, this looks the most likely way of introducing such bug (though, I still didn't manage to reproduce it).

nijel added a commit that referenced this issue Sep 6, 2022
@EndOfTheSpline
Copy link
Author

I've added the resx add-on as a failed attempt to fix the issue. I've added the squash add-on at some point (although I believe the issue happened before already - just in two separate commits, one adding it, one removing it?), and the resx later on.

I'll happily try whatever you suggest. I've tried to reproduce it with my local installation, but couldn't trigger it, no matter what I tried - even with the same docker config. Even on the server itself, it's not guaranteed to revert. I'll remove the resx add-on for the time being.

I don't know how weblate internals work, but if it's a timing issue (i.e. around the time some daily task runs), it could be possible that the usual work hours just collide really well with that. I suppose I could take a look at the recent changes and see when a previous populated field is wiped without an user and see if a pattern emerges.

@nijel
Copy link
Member

nijel commented Sep 6, 2022

Any update on the repository is performed with a lock held, so it should not be a concurrency issue. It could be an ordering issue - the new string is added to the source, it triggers commit, what might trigger resx add-on (it should not, it should happen once all pending changes are committed), which adds empty string. This could explain overriding translated pending string. Still, I could not find reproducer for that...

@EndOfTheSpline
Copy link
Author

Yes, it seems hard to reproduce it. It's not guaranteed to happen even on the prod system where it does seemingly all the time.

However, it just happened again, and I had the resx addon uninstalled. This is the latest one:

2022-09-07T07:15:54.660356+00:00  New string added                  de-CH
2022-09-07T07:15:54.660476+00:00  New string added                  it-CH
2022-09-07T07:15:54.660535+00:00  New string added                  fr-CH
2022-09-07T07:16:34.119509+00:00  String updated in the repository  de-CH   de_TEXT
2022-09-07T07:16:45.409690+00:00  New translation                   it-CH   it_TEXT
2022-09-07T07:16:51.900675+00:00  New translation                   fr-CH   fr-TEXT

2022-09-07T07:18:13.146930+00:00  New string added                  de-CH   (This is another, new string)
2022-09-07T07:18:16.094245+00:00  String updated in the repository  de-CH   (The new string from before is updated)

2022-09-07T07:18:16.094316+00:00  Resource update                   de-CH    
2022-09-07T07:18:16.170792+00:00  String updated in the repository  fr-CH   (text is set to empty)
2022-09-07T07:18:16.170863+00:00  Resource update                   fr-CH
2022-09-07T07:18:16.246254+00:00  String updated in the repository  it-CH   (text is set to empty)
2022-09-07T07:18:16.246321+00:00  Resource update                   it-CH
2022-09-07T07:18:16.258126+00:00  New string to translate           fr-CH
2022-09-07T07:18:16.260995+00:00  New string to translate           it-CH
2022-09-07T07:18:20.410251+00:00  Pushed changes                    (root component)

Looking at it like this, it seems like adding another string could trigger this behaviour?

Roughly:

  • T+0: The new string is added (TextA)
  • T+20s: Translations have been added (TextA)
  • T+120s: Another string is added (TextB)
  • T+123s: Another string is updated (TextB)
  • T+123s: Resource Update
  • T+123s: TextA is wiped

I'm sorry if this is total nonsense because I don't know how weblate works, internally, but could it be that adding another string triggers an update of the main file/language, which then resets the other translations?

@botanegg
Copy link

botanegg commented Sep 13, 2022

Got similar problem with "Cleanup translation files" plugin

I use selfhosted weblate 4.14 and gitlab integration via ssh

  1. translate a lot of strings (stored in simple JSON)
  2. push some commits to gitlab repo with development changes (wo lang files changed, but trigger webhook in gitlab)
  3. weblate runs Cleanup translation files", and push commit with small diff (removes 2 strings from big (1700 strings) file)
  4. weblate make a lot of single string changees "String updated in the repository," and rewrite all penging changes back

Should I create new issue?
Can anyone help me to debug this?

@nijel
Copy link
Member

nijel commented Sep 14, 2022

Can you reproduce the issue without the add-on? It seems we might have issue with add-on execution, as that would explain such behavior.

@nijel nijel added this to the 4.14.1 milestone Sep 14, 2022
@nijel nijel added the bug Something is broken. label Sep 14, 2022
@botanegg
Copy link

i cannot reproduce w/o addon
It seems like "cleanup" addon works first, and overwrite changes from weblate "pending to commit"

anyway i'll try to describe reproduction steps on next weekend

@EndOfTheSpline
Copy link
Author

The only add-on we're still using is GIt squash to keep the git history somewhat short-ish. I've disabled it temporarily to see if that helps (or at least makes it more obvious what is happening).

@nijel
Copy link
Member

nijel commented Sep 14, 2022

#8113 might address this issue, I would appreciate if somebody could test whether it really does (I didn't yet manage to reproduce the issue).

@nijel
Copy link
Member

nijel commented Sep 15, 2022

Closing, this with assumption that #8113 will address this in the 4.14.1 release. In case it does not, please let me know.

@nijel nijel closed this as completed Sep 15, 2022
@nijel nijel self-assigned this Sep 15, 2022
@github-actions
Copy link

Thank you for your report; the issue you have reported has just been fixed.

  • In case you see a problem with the fix, please comment on this issue.
  • In case you see a similar problem, please open a separate issue.
  • If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is broken.
Projects
None yet
Development

No branches or pull requests

3 participants