Change not committed if user full name has accent

[cedk]

Describe the bug
If the full name contains an accent, weblate shows modification on the repository but does not commit them.

Server configuration and status

• Weblate 3.7.1
• Python 3.7.3
• Django 2.2.3
• Celery 4.3.0
• celery-batches 0.2
• six 1.12.0
• social-auth-core 3.2.0
• social-auth-app-django 3.1.0
• django-appconf 1.0.3
• translate-toolkit 2.4.0
• translation-finder 1.6
• Whoosh 2.7.4
• defusedxml 0.6.0
• Git 2.20.1
• Pillow 5.4.1
• python-dateutil 2.8.0
• lxml 4.3.2
• django-crispy-forms 1.7.2
• django_compressor 2.3
• djangorestframework 3.9.4
• user-agents 2.0
• jellyfish 0.7.2
• diff-match-patch 20121119
• pytz 2019.1
• pyuca 1.2
• ruamel.yaml 0.15.97
• tesserocr 2.4.0
• Mercurial 4.8.2
• git-svn 2.20.1
• Database backends: django.db.backends.postgresql
• Cache backends: avatar:FileBasedCache, default:RedisCache
• Email setup: django.core.mail.backends.smtp.EmailBackend: smtp.tryton.org
• Celery: redis://cache:6379/1, redis://cache:6379/1, regular
• Platform: Linux 4.19.52-gentoo (x86_64)

Additional context
The repository is using mercurial.

[nijel]

Do you get any errors? Most likely it will be in celery logs.

[cedk]

When I force a commit, I see no error in the logs.

[nijel]

Is celery worker running? Can you see the commit task in their logs?

[cedk]

I guess as I see logs like "Received task" and changes from other users (without accent, only ASCII) are committed.

[nijel]

It can hardly disappear without any trace.

In the main celery log there should be something like:

[2019-06-30 20:02:14,978: INFO/MainProcess] Received task: weblate.trans.tasks.perform_commit[1c2d7c1b-2131-4751-8221-bcaa2dc7b8c9] 

An then followed in the worker log:

[2019-06-30 11:24:25,573: INFO/ForkPoolWorker-11] osmand/main/de: committing pending changes (update)
[2019-06-30 11:24:25,573: INFO/ForkPoolWorker-11] Lock 140548480471112 acquired on /home/www-data/weblate-hosted/vcs/osmand/main.lock
[2019-06-30 11:24:26,026: INFO/ForkPoolWorker-11] Lock 140548480471112 acquired on /home/www-data/weblate-hosted/vcs/osmand/main.lock
[2019-06-30 11:24:26,027: INFO/ForkPoolWorker-11] Lock 140548480471112 acquired on /home/www-data/weblate-hosted/vcs/osmand/main.lock
[2019-06-30 11:24:26,058: INFO/ForkPoolWorker-11] osmand/main/de: committing ['/home/www-data/weblate-hosted/vcs/osmand/main/OsmAnd/res/values-de/strings.xml'] as Hakuchi <....>

[cedk]

I got such logging for commit from other users. But for my changes, there is no trace.
More over, the commit button is disabled while still showing modification in the repository detail. See:

[nijel]

There should be no uncommitted changes in the repo, so in past something has probably failed. Can you check what changes are there? It seems that in some corner cases flushing changes is not properly executed in a transaction (see 8188c4b), so there was probably some bug in the past where it failed. It really doesn't have to be related to accents because this should work fine (tested on myself).

[cedk]

They are only changes I made with accented full name and one that I made today.
When I remove the accent and made 1 change, it was committed directly but any of the previous.

But if it is not related to the full name, what should I do to commit those changes?

[nijel]

Without finding the actual exception in the logs, it's only guessing what has happened. We do have unit tests for commit with unicode name and I'm usually the one to hit such problems, so it has definitely received some coverage. On the other side Mercurial backend is way less used than Git, so it's possible it's some issue with that.

[nijel]

Hmm, it turns out that the error would not be logged because of Celery nog logging such errors. Can you please retry after applying 4170919?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[cedk]

I did not have time yet to test the patch because I have only a docker setup in production and not any development environment. So it would be a pity to close the issue without a fix.

[nijel]

True, the bot just kicks in when there is nothing new for two weeks.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nijel]

Today, when testing new CI platform, I got test failure on the commit with Unicode name, see https://cloud.drone.io/WeblateOrg/weblate/7/1/3

It turns out that Mercurial chokes on that when the locale is not present.

[nijel]

Thank you for your report, the issue you have reported has just been fixed.

• In case you see problem with the fix, please comment on this issue.
• In case you see similar problem, please open separate issue.
• If you are happy with the outcome, consider supporting Weblate by donating.