Repository navigation
Android resources support - #2
Conversation
|
Thanks for the work on this! Is it possible to make some simple tests similar to the other storage classes? You can inherit a lot of tests (like most of the storage classes do) and then maybe add a few to test the parsing and serialisation (especially since you are doing some work to keep specific formatting by setting the .tail attribute). |
|
I've extended the support for proper escaping and added testcase for the class. |
|
Thanks for this! I'm sure this must have taken quite a bit of time to do. I'll try to review this soon. Thanks again! |
|
Seems it is still buggy, and the problem I reported before is still there. Also it seems, that the script now removes end of lines within the string. |
|
Or probably this is new bug: global name 'name' is not defined Request Method: POST global name 'name' is not defined Exception Location: /usr/lib/python2.7/dist-packages/translate/storage/aresource.py in unescape, line 205 ['/var/www/weblate', Server time: st, 29 Srp 2012 11:09:05 +0200 |
|
The exception you've seen is now fixed (but you will get another as this was in error path). According to docs I've found newlines should be written as \n in the XML file, real newlines are handled as any other whitespace. If it actually works otherwise then the parser needs to be adjusted. |
|
Actually it was caused by \N in your translation, what is IMHO wrong (not sure if it actually works on Android). I've adjusted the parser to cope with this now. |
|
No, it is not about using the newlines in the result. Is about using the newlines to better format the message for translator. Like, when the source string is something like this Than, you would probably like also the translation to by like this: For better reading and comparision. But currently, it seems to collapse to this result (or similar), which is sometimes hard to read. |
|
Ah now I see, I've tried to add something like that, but currently I don't see way how to preserve this formatting in all cases. |
|
Latest error: Environment: Request Method: GET Django Version: 1.4 Traceback:
Exception Type: IndexError at /projects/osmand/osmand-main/sk/translate/ |
|
Again should be fixed. |
|
So far it works fine. Thanx a lot. (We have lot of merge conflicts because of this missing processing) |
|
I get the following error when I try to save any translation: ValueError at /projects/osmand/osmand-main/af/translate/ removing unsupported escape sequence "\ " Request Method: GET Request URL: http://translate.osmand.net/projects/osmand/osmand-main/af/translate/?type=untranslated&pos=909&accept=108 Django Version: 1.4 Exception Type: ValueError Exception Value: removing unsupported escape sequence "\ " Exception Location: /usr/lib/python2.7/dist-packages/translate/storage/aresource.py in unescape, line 185 Python Executable: /usr/bin/python Python Version: 2.7.2 Python Path: ['/var/www/weblate', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/local/lib/python2.7/dist-packages/git/ext/gitdb', '/usr/local/lib/python2.7/dist-packages/gitdb/ext/async', '/usr/local/lib/python2.7/dist-packages/gitdb/ext/smmap'] Server time: Fri, 31 Aug 2012 20:29:52 +0200 |
|
And another one: TypeError at /projects/osmand/osmand-main/cs/translate/ 'NoneType' object is not iterable Request Method: GET 'NoneType' object is not iterable Exception Location: /var/www/weblate/weblate/trans/models.py in update_unit, line 1421 ['/var/www/weblate', Server time: so, 1 Zář 2012 13:28:24 +0200 Request information u'suggestions' pos u'929' accept u'140' POST No POST data No FILES data '0' csrftoken 'TECp7sYOV0cit4hCgp7d1kDdftHW0MY8' sessionid 'b1c87e59bc9f6b75b3eadc36e6cf2827' ui-tabs-2 '0' translate-tab '0' META '80' HTTP_COOKIE 'ui-tabs-1=0; ui-tabs-2=0; csrftoken=TECp7sYOV0cit4hCgp7d1kDdftHW0MY8; translate-tab=0; sessionid=b1c87e59bc9f6b75b3eadc36e6cf2827' mod_wsgi.listener_host '' SERVER_SOFTWARE 'Apache/2.2.20 (Ubuntu)' SCRIPT_NAME u'' mod_wsgi.handler_script '' SERVER_SIGNATURE ' Apache/2.2.20 (Ubuntu) Server at translate.osmand.net Port 80\n'REQUEST_METHOD 'GET' PATH_INFO u'/projects/osmand/osmand-main/cs/translate/' SERVER_PROTOCOL 'HTTP/1.1' QUERY_STRING 'type=suggestions&pos=929&accept=140' HTTP_USER_AGENT 'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0' HTTP_CONNECTION 'keep-alive' HTTP_REFERER 'http://translate.osmand.net/projects/osmand/osmand-main/cs/translate/?type=suggestions&pos=135' SERVER_NAME 'translate.osmand.net' REMOTE_ADDR '84.73.206.125' mod_wsgi.request_handler 'wsgi-script' wsgi.url_scheme 'http' PATH_TRANSLATED '/var/www/weblate/weblate/wsgi.py/projects/osmand/osmand-main/cs/translate/' SERVER_PORT '80' wsgi.multiprocess True mod_wsgi.input_chunked '0' SERVER_ADDR '78.46.73.132' DOCUMENT_ROOT '/var/www/weblate/weblate/media/' mod_wsgi.process_group '' SCRIPT_FILENAME '/var/www/weblate/weblate/wsgi.py' SERVER_ADMIN 'pavol.zibrita+weblateadmin@gmail.com' wsgi.input <mod_wsgi.Input object at 0x7f4ea0fe1070> HTTP_HOST 'translate.osmand.net' wsgi.multithread False mod_wsgi.callable_object 'application' REQUEST_URI '/projects/osmand/osmand-main/cs/translate/?type=suggestions&pos=929&accept=140' HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8' wsgi.version (1, 1) GATEWAY_INTERFACE 'CGI/1.1' wsgi.run_once False wsgi.errors <mod_wsgi.Log object at 0x7f4ea0fe1c70> REMOTE_PORT '53775' HTTP_ACCEPT_LANGUAGE 'en-us,en;q=0.5' mod_wsgi.version (3, 3) mod_wsgi.application_group 'translate.osmand.net|' mod_wsgi.script_reloading '1' wsgi.file_wrapper '' CSRF_COOKIE 'TECp7sYOV0cit4hCgp7d1kDdftHW0MY8' HTTP_ACCEPT_ENCODING 'gzip, deflate' Settings 'F Y' NEARBY_MESSAGES 5 USE_L10N True USE_THOUSAND_SEPARATOR False CSRF_COOKIE_SECURE False LANGUAGE_CODE 'en-us' ROOT_URLCONF 'weblate.urls' MANAGERS (('Pavol Zibrita', 'pavol.zibrita+weblate@gmail.com'),) ACCOUNT_ACTIVATION_DAYS 7 STATIC_ROOT '' MESSAGE_STORAGE 'django.contrib.messages.storage.fallback.FallbackStorage' EMAIL_SUBJECT_PREFIX '[Weblate for OsmAnd] ' SEND_BROKEN_LINK_EMAILS False URL_VALIDATOR_USER_AGENT 'Django/1.4 (https://www.djangoproject.com)' STATICFILES_FINDERS ('django.contrib.staticfiles.finders.FileSystemFinder', SESSION_COOKIE_DOMAIN None SESSION_COOKIE_NAME 'sessionid' DEFAULT_LOG 'console' ADMIN_FOR () TIME_INPUT_FORMATS ('%H:%M:%S', '%H:%M') DATABASES {'default': {'ENGINE': 'django.db.backends.mysql', DEFAULT_CONTENT_TYPE 'text/html' FILE_UPLOAD_PERMISSIONS None FILE_UPLOAD_HANDLERS ('django.core.files.uploadhandler.MemoryFileUploadHandler', MT_APERTIUM_KEY u'********************' ENABLE_HOOKS True APPEND_SLASH True FIRST_DAY_OF_WEEK 0 DATABASE_ROUTERS [] WHOOSH_INDEX '/var/www/weblate/weblate/whoosh-index' STATICFILES_STORAGE 'django.contrib.staticfiles.storage.StaticFilesStorage' CACHES {'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache', SERVER_EMAIL 'pavol.zibrita+weblateadmin@gmail.com' SESSION_COOKIE_PATH '/' USE_X_FORWARDED_HOST False MIDDLEWARE_CLASSES ('django.middleware.common.CommonMiddleware', USE_I18N True THOUSAND_SEPARATOR ',' SECRET_KEY u'********************' LANGUAGE_COOKIE_NAME 'django_language' LAZY_COMMITS True LOCK_TIME 900 DEFAULT_INDEX_TABLESPACE '' GIT_ROOT '/var/www/weblate/weblate/repos/' TRANSACTIONS_MANAGED False LOGGING_CONFIG 'django.utils.log.dictConfig' SIGNING_BACKEND 'django.core.signing.TimestampSigner' SIMILAR_MESSAGES 5 TEMPLATE_LOADERS ('django.template.loaders.filesystem.Loader', WSGI_APPLICATION None TEMPLATE_DEBUG True X_FRAME_OPTIONS 'SAMEORIGIN' AUTHENTICATION_BACKENDS ('django.contrib.auth.backends.ModelBackend',) FORCE_SCRIPT_NAME None CACHE_BACKEND 'locmem://' DEFAULT_CHARSET 'utf-8' SESSION_COOKIE_SECURE False CSRF_COOKIE_DOMAIN None FILE_CHARSET 'utf-8' DEBUG True SESSION_FILE_PATH None OFFLOAD_INDEXING True INSTALLED_APPS ('django.contrib.auth', LANGUAGES (('cs', u'\u010cesky'), COMMENTS_ALLOW_PROFANITIES False STATICFILES_DIRS () PREPEND_WWW False SECURE_PROXY_SSL_HEADER None PASSWORD_RESET_TIMEOUT_DAYS u'********************' SESSION_COOKIE_HTTPONLY True DEBUG_PROPAGATE_EXCEPTIONS False SITE_TITLE 'Weblate for OsmAnd' MONTH_DAY_FORMAT 'F j' LOGIN_URL '/accounts/login/' SESSION_EXPIRE_AT_BROWSER_CLOSE False TIME_FORMAT 'P' DATE_INPUT_FORMATS ('%Y-%m-%d', CSRF_COOKIE_NAME 'csrftoken' EMAIL_HOST_PASSWORD u'********************' AUTH_PROFILE_MODULE 'accounts.Profile' CACHE_MIDDLEWARE_ALIAS 'default' SESSION_SAVE_EVERY_REQUEST False ADMIN_MEDIA_PREFIX '/static/admin/' NUMBER_GROUPING 0 SESSION_ENGINE 'django.contrib.sessions.backends.db' DEFAULT_FILE_STORAGE 'django.core.files.storage.FileSystemStorage' CSRF_FAILURE_VIEW 'django.views.csrf.csrf_failure' CSRF_COOKIE_PATH '/' LOGIN_REDIRECT_URL '/' LOGGING {'disable_existing_loggers': False, IGNORABLE_404_URLS () LOCALE_PATHS () TEMPLATE_STRING_IF_INVALID '' LOGOUT_URL '/accounts/logout/' EMAIL_USE_TLS False EMAIL_SEND_HTML False FIXTURE_DIRS () EMAIL_HOST 'localhost' DATE_FORMAT 'N j, Y' MEDIA_ROOT '/var/www/weblate/weblate/media/' DEFAULT_EXCEPTION_REPORTER_FILTER 'django.views.debug.SafeExceptionReporterFilter' ADMINS (('Pavol Zibrita', 'pavol.zibrita+weblate@gmail.com'),) FORMAT_MODULE_PATH None DEFAULT_FROM_EMAIL 'pavol.zibrita+weblateadmin@gmail.com' MT_MICROSOFT_KEY u'********************' AUTO_LOCK True MEDIA_URL '/media/' DATETIME_FORMAT 'N j, Y, P' TEMPLATE_DIRS ('/var/www/weblate/weblate/html/',) SITE_ID 1 DISALLOWED_USER_AGENTS () ALLOWED_INCLUDE_ROOTS () DECIMAL_SEPARATOR '.' SHORT_DATE_FORMAT 'm/d/Y' TEST_RUNNER 'django.test.simple.DjangoTestSuiteRunner' CACHE_MIDDLEWARE_KEY_PREFIX u'********************' TIME_ZONE 'Europe/Prague' FILE_UPLOAD_MAX_MEMORY_SIZE 2621440 WEB_ROOT '/var/www/weblate/weblate' EMAIL_BACKEND 'django.core.mail.backends.smtp.EmailBackend' DEFAULT_TABLESPACE '' TEMPLATE_CONTEXT_PROCESSORS ('django.contrib.auth.context_processors.auth', SESSION_COOKIE_AGE 1209600 SETTINGS_MODULE 'weblate.settings' USE_ETAGS False LANGUAGES_BIDI ('he', 'ar', 'fa') CHECK_LIST ('weblate.trans.checks.SameCheck', FILE_UPLOAD_TEMP_DIR None INTERNAL_IPS () STATIC_URL '/static/' EMAIL_PORT 25 USE_TZ False SHORT_DATETIME_FORMAT 'm/d/Y P' PASSWORD_HASHERS u'********************' ABSOLUTE_URL_OVERRIDES {} CACHE_MIDDLEWARE_SECONDS 600 DATETIME_INPUT_FORMATS ('%Y-%m-%d %H:%M:%S', EMAIL_HOST_USER '' PROFANITIES_LIST |
|
No, still errors. Very strange. Not using the escape/unescape in aresource does not make the problems, so the problem lies somewhere in there... I was not able to track it down, python is still a mistery for me... |
|
I just realized, the 'NoneType' object is not iterable is other bug in weblate (sync the template and translation), and probably it has nothing to do with this. However, it occurs more frequently in weblate if this escape/unescape is in place. |
|
I believe that all mentioned bugs are rather issues in Weblate than in this patch. |
|
Hm, this error now: |
|
Already fixed in f2abb1a :-) |
|
Hm, I still get the latest error, also with the last change. |
|
no, works fine, probably it was not recompiled before. Thanx! |
This is needed to have not translated strings and also original english messages.
Moving elements around seems to cause more troubles than benefits.
This should be rather handled on application level (same as with other storages).
- Tried to conform source, target and name properties handling to other monolingual storages (like for java properties), in order to not break Pootle handling (check his "hackish" fix_monolingual method) - Handled standalone "<" character escaping when setting the target text
|
I've just rebased my branch on top of current translate-toolkit, any chance to get this merged or reviewed? |
|
Hey nijel, Thanks for the rebase. I've been trying to schedule time for fwolff to complete his review. He doesn't have much time at the moment. Sorry its taking so long but I do want to land this before we do another toolkit release. |
|
Thanks for all the work and testing! I hope to have a proper look soon. Here are a few notes from a quick look:
Thanks for the tests and comments. I think there are very few good sources of documentation on this format, but if you have any, let's add that at the top of the file as well. I'll try to verify tests on my side soon and integrate. Would it be ok if I squash all of this in one commit? (not that I feel strongly about that) Thanks again. |
|
Indeed pdb is not needed, it slipped in one merge request and I somehow forgot to delete it. I have no problems with squashing my commits, but there are few contributions not from me and I think it would be fair to keep the correct attribution there. |
|
Thanks a lot for this work. I merged and pushed it to master. I ended up not squashing, so your fork should still share the history, but in future, keep in mind that we might prefer to rebase your changes on master, in which case your fork would have to be recreated. In other words, doing this on a branch might be better, as I understand it. This is a nasty file format. Thanks a lot for the contribution! |
Hi
I've hacked simple support for Android resources. As I had troubles finding what all methods need to be implemented, I've just implemented all I needed to make Weblate work with these files.
The most non standard method is probably set_base_resource, which is used for setting original resource which contains all messages (compared to translations which do contain only translated strings) and their original strings. Without this, it is still possible to work with the file, but you won't get non translated strings.
Michal