Skip to content

Android resources support - #2

Merged
friedelwolff merged 43 commits into
translate:masterfrom
nijel:master
Dec 4, 2012
Merged

friedelwolff merged 43 commits into
translate:masterfrom
nijel:master

Conversation

@nijel

@nijel nijel commented Apr 20, 2012

Copy link
Copy Markdown
Member

Hi

I've hacked simple support for Android resources. As I had troubles finding what all methods need to be implemented, I've just implemented all I needed to make Weblate work with these files.

The most non standard method is probably set_base_resource, which is used for setting original resource which contains all messages (compared to translations which do contain only translated strings) and their original strings. Without this, it is still possible to work with the file, but you won't get non translated strings.

Michal

@friedelwolff

Copy link
Copy Markdown
Member

Thanks for the work on this! Is it possible to make some simple tests similar to the other storage classes? You can inherit a lot of tests (like most of the storage classes do) and then maybe add a few to test the parsing and serialisation (especially since you are doing some work to keep specific formatting by setting the .tail attribute).

@nijel

nijel commented Aug 28, 2012

Copy link
Copy Markdown
Member Author

I've extended the support for proper escaping and added testcase for the class.

@friedelwolff

Copy link
Copy Markdown
Member

Thanks for this! I'm sure this must have taken quite a bit of time to do. I'll try to review this soon. Thanks again!

@pavolzibrita

Copy link
Copy Markdown

Seems it is still buggy, and the problem I reported before is still there. Also it seems, that the script now removes end of lines within the string.

@pavolzibrita

Copy link
Copy Markdown

Or probably this is new bug:
NameError at /projects/osmand/osmand-main/cs/translate/

global name 'name' is not defined

Request Method: POST
Request URL: http://translate.osmand.net/projects/osmand/osmand-main/cs/translate/
Django Version: 1.4
Exception Type: NameError
Exception Value:

global name 'name' is not defined

Exception Location: /usr/lib/python2.7/dist-packages/translate/storage/aresource.py in unescape, line 205
Python Executable: /usr/bin/python
Python Version: 2.7.2
Python Path:

['/var/www/weblate',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/local/lib/python2.7/dist-packages/git/ext/gitdb',
'/usr/local/lib/python2.7/dist-packages/gitdb/ext/async',
'/usr/local/lib/python2.7/dist-packages/gitdb/ext/smmap']

Server time: st, 29 Srp 2012 11:09:05 +0200
Traceback Switch back to interactive view
Environment: Request Method: POST Request URL: http://translate.osmand.net/projects/osmand/osmand-main/cs/translate/ Django Version: 1.4 Python Version: 2.7.2 Installed Applications: ('django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.admin', 'django.contrib.admindocs', 'registration', 'south', 'weblate.trans', 'weblate.lang', 'weblate.accounts', 'weblate') Installed Middleware: ('django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware') Traceback: File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response 111. response = callback(request, _callback_args, *_callback_kwargs) File "/var/www/weblate/weblate/trans/views.py" in translate 785. unit.save_backend(request) File "/var/www/weblate/weblate/trans/models.py" in save_backend 1728. (saved, pounit) = self.translation.update_unit(self, request) File "/var/www/weblate/weblate/trans/models.py" in update_unit 1416. store = self.get_store() File "/var/www/weblate/weblate/trans/models.py" in get_store 1124. store = ttkit(self.get_filename()) File "/var/www/weblate/weblate/trans/models.py" in ttkit 76. return factory.getobject(storefile) File "/usr/lib/python2.7/dist-packages/translate/storage/factory.py" in getobject 189. store = storeclass.parsefile(storefile) File "/usr/lib/python2.7/dist-packages/translate/storage/base.py" in parsefile 813. newstore = cls.parsestring(storestring) File "/usr/lib/python2.7/dist-packages/translate/storage/base.py" in parsestring 717. newstore.parse(storestring) File "/usr/lib/python2.7/dist-packages/translate/storage/lisa.py" in parse 345. term = self.UnitClass.createfromxmlElement(entry) File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in createfromxmlElement 247. term._parse() File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in _parse 44. self.target = self.gettarget() File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in gettarget 237. return self.unescape(target) File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in unescape 205. name, "".join(text[i-1 : i+1]))) Exception Type: NameError at /projects/osmand/osmand-main/cs/translate/ Exception Value: global name 'name' is not defined

@nijel

nijel commented Aug 31, 2012

Copy link
Copy Markdown
Member Author

The exception you've seen is now fixed (but you will get another as this was in error path).

According to docs I've found newlines should be written as \n in the XML file, real newlines are handled as any other whitespace. If it actually works otherwise then the parser needs to be adjusted.

@nijel

nijel commented Aug 31, 2012

Copy link
Copy Markdown
Member Author

Actually it was caused by \N in your translation, what is IMHO wrong (not sure if it actually works on Android).

I've adjusted the parser to cope with this now.

@pavolzibrita

Copy link
Copy Markdown

No, it is not about using the newlines in the result. Is about using the newlines to better format the message for translator.

Like, when the source string is something like this

  This are changes:
   \n\t my first change
   \n\t my second change

Than, you would probably like also the translation to by like this:

  This are changes:
   \n\t my first change
   \n\t my second change

For better reading and comparision. But currently, it seems to collapse to this result (or similar), which is sometimes hard to read.

  This are changes: \n\t my first change \n\t my second change

@nijel

nijel commented Aug 31, 2012

Copy link
Copy Markdown
Member Author

Ah now I see, I've tried to add something like that, but currently I don't see way how to preserve this formatting in all cases.

@pavolzibrita

Copy link
Copy Markdown

Latest error:

Environment:

Request Method: GET
Request URL: http://translate.osmand.net/projects/osmand/osmand-main/sk/translate/?type=suggestions&pos=933&accept=104

Django Version: 1.4
Python Version: 2.7.2
Installed Applications:
('django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'registration',
'south',
'weblate.trans',
'weblate.lang',
'weblate.accounts',
'weblate')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')

Traceback:
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response

  1.                     response = callback(request, _callback_args, *_callback_kwargs)
    
    File "/var/www/weblate/weblate/trans/views.py" in translate
  2.             suggestion.accept(request)
    
    File "/var/www/weblate/weblate/trans/models.py" in accept
  3.         unit.save_backend(request, False)
    
    File "/var/www/weblate/weblate/trans/models.py" in save_backend
  4.     (saved, pounit) = self.translation.update_unit(self, request)
    
    File "/var/www/weblate/weblate/trans/models.py" in update_unit
  5.         store = self.get_store()
    
    File "/var/www/weblate/weblate/trans/models.py" in get_store
  6.         store.set_base_resource(template)
    
    File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in set_base_resource
  7.             newunit.target = ''
    
    File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in settarget
  8.     self.xmlelement.text = self.escape(target)
    
    File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in escape
  9.     if text[0] in WHITESPACE or text[-1] in WHITESPACE or len(MULTIWHITESPACE.findall(text)) > 0:
    

Exception Type: IndexError at /projects/osmand/osmand-main/sk/translate/
Exception Value: string index out of range

@nijel

nijel commented Aug 31, 2012

Copy link
Copy Markdown
Member Author

Again should be fixed.

@pavolzibrita

Copy link
Copy Markdown

So far it works fine. Thanx a lot. (We have lot of merge conflicts because of this missing processing)

@pavolzibrita

Copy link
Copy Markdown

I get the following error when I try to save any translation:

ValueError at /projects/osmand/osmand-main/af/translate/

removing unsupported escape sequence "\ "

Request Method: GET Request URL: http://translate.osmand.net/projects/osmand/osmand-main/af/translate/?type=untranslated&pos=909&accept=108 Django Version: 1.4 Exception Type: ValueError Exception Value:

removing unsupported escape sequence "\ "

Exception Location: /usr/lib/python2.7/dist-packages/translate/storage/aresource.py in unescape, line 185 Python Executable: /usr/bin/python Python Version: 2.7.2 Python Path:

['/var/www/weblate', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/local/lib/python2.7/dist-packages/git/ext/gitdb', '/usr/local/lib/python2.7/dist-packages/gitdb/ext/async', '/usr/local/lib/python2.7/dist-packages/gitdb/ext/smmap']

Server time: Fri, 31 Aug 2012 20:29:52 +0200

@pavolzibrita

Copy link
Copy Markdown

And another one:

TypeError at /projects/osmand/osmand-main/cs/translate/

'NoneType' object is not iterable

Request Method: GET
Request URL: http://translate.osmand.net/projects/osmand/osmand-main/cs/translate/?type=suggestions&pos=929&accept=140
Django Version: 1.4
Exception Type: TypeError
Exception Value:

'NoneType' object is not iterable

Exception Location: /var/www/weblate/weblate/trans/models.py in update_unit, line 1421
Python Executable: /usr/bin/python
Python Version: 2.7.2
Python Path:

['/var/www/weblate',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/local/lib/python2.7/dist-packages/git/ext/gitdb',
'/usr/local/lib/python2.7/dist-packages/gitdb/ext/async',
'/usr/local/lib/python2.7/dist-packages/gitdb/ext/smmap']

Server time: so, 1 Zář 2012 13:28:24 +0200
Traceback Switch to copy-and-paste view

/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py in get_response

                            response = callback(request, *callback_args, **callback_kwargs)

    ...
▶ Local vars
/var/www/weblate/weblate/trans/views.py in translate

                    suggestion.accept(request)

    ...
▶ Local vars
/var/www/weblate/weblate/trans/models.py in accept

                unit.save_backend(request, False)

    ...
▶ Local vars
/var/www/weblate/weblate/trans/models.py in save_backend

            (saved, pounit) = self.translation.update_unit(self, request)

    ...
▶ Local vars
/var/www/weblate/weblate/trans/models.py in update_unit

                for pounit in store.findunits(src):

    ...
▶ Local vars

Request information
GET
Variable Value
type

u'suggestions'

pos

u'929'

accept

u'140'

POST

No POST data
FILES

No FILES data
COOKIES
Variable Value
ui-tabs-1

'0'

csrftoken

'TECp7sYOV0cit4hCgp7d1kDdftHW0MY8'

sessionid

'b1c87e59bc9f6b75b3eadc36e6cf2827'

ui-tabs-2

'0'

translate-tab

'0'

META
Variable Value
mod_wsgi.listener_port

'80'

HTTP_COOKIE

'ui-tabs-1=0; ui-tabs-2=0; csrftoken=TECp7sYOV0cit4hCgp7d1kDdftHW0MY8; translate-tab=0; sessionid=b1c87e59bc9f6b75b3eadc36e6cf2827'

mod_wsgi.listener_host

''

SERVER_SOFTWARE

'Apache/2.2.20 (Ubuntu)'

SCRIPT_NAME

u''

mod_wsgi.handler_script

''

SERVER_SIGNATURE

'

Apache/2.2.20 (Ubuntu) Server at translate.osmand.net Port 80\n'

REQUEST_METHOD

'GET'

PATH_INFO

u'/projects/osmand/osmand-main/cs/translate/'

SERVER_PROTOCOL

'HTTP/1.1'

QUERY_STRING

'type=suggestions&pos=929&accept=140'

HTTP_USER_AGENT

'Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0'

HTTP_CONNECTION

'keep-alive'

HTTP_REFERER

'http://translate.osmand.net/projects/osmand/osmand-main/cs/translate/?type=suggestions&pos=135'

SERVER_NAME

'translate.osmand.net'

REMOTE_ADDR

'84.73.206.125'

mod_wsgi.request_handler

'wsgi-script'

wsgi.url_scheme

'http'

PATH_TRANSLATED

'/var/www/weblate/weblate/wsgi.py/projects/osmand/osmand-main/cs/translate/'

SERVER_PORT

'80'

wsgi.multiprocess

True

mod_wsgi.input_chunked

'0'

SERVER_ADDR

'78.46.73.132'

DOCUMENT_ROOT

'/var/www/weblate/weblate/media/'

mod_wsgi.process_group

''

SCRIPT_FILENAME

'/var/www/weblate/weblate/wsgi.py'

SERVER_ADMIN

'pavol.zibrita+weblateadmin@gmail.com'

wsgi.input

<mod_wsgi.Input object at 0x7f4ea0fe1070>

HTTP_HOST

'translate.osmand.net'

wsgi.multithread

False

mod_wsgi.callable_object

'application'

REQUEST_URI

'/projects/osmand/osmand-main/cs/translate/?type=suggestions&pos=929&accept=140'

HTTP_ACCEPT

'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8'

wsgi.version

(1, 1)

GATEWAY_INTERFACE

'CGI/1.1'

wsgi.run_once

False

wsgi.errors

<mod_wsgi.Log object at 0x7f4ea0fe1c70>

REMOTE_PORT

'53775'

HTTP_ACCEPT_LANGUAGE

'en-us,en;q=0.5'

mod_wsgi.version

(3, 3)

mod_wsgi.application_group

'translate.osmand.net|'

mod_wsgi.script_reloading

'1'

wsgi.file_wrapper

''

CSRF_COOKIE

'TECp7sYOV0cit4hCgp7d1kDdftHW0MY8'

HTTP_ACCEPT_ENCODING

'gzip, deflate'

Settings
Using settings module weblate.settings
Setting Value
YEAR_MONTH_FORMAT

'F Y'

NEARBY_MESSAGES

5

USE_L10N

True

USE_THOUSAND_SEPARATOR

False

CSRF_COOKIE_SECURE

False

LANGUAGE_CODE

'en-us'

ROOT_URLCONF

'weblate.urls'

MANAGERS

(('Pavol Zibrita', 'pavol.zibrita+weblate@gmail.com'),)

ACCOUNT_ACTIVATION_DAYS

7

STATIC_ROOT

''

MESSAGE_STORAGE

'django.contrib.messages.storage.fallback.FallbackStorage'

EMAIL_SUBJECT_PREFIX

'[Weblate for OsmAnd] '

SEND_BROKEN_LINK_EMAILS

False

URL_VALIDATOR_USER_AGENT

'Django/1.4 (https://www.djangoproject.com)'

STATICFILES_FINDERS

('django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder')

SESSION_COOKIE_DOMAIN

None

SESSION_COOKIE_NAME

'sessionid'

DEFAULT_LOG

'console'

ADMIN_FOR

()

TIME_INPUT_FORMATS

('%H:%M:%S', '%H:%M')

DATABASES

{'default': {'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1',
'NAME': 'weblate',
'OPTIONS': {},
'PASSWORD': u'********************',
'PORT': '',
'TEST_CHARSET': None,
'TEST_COLLATION': None,
'TEST_MIRROR': None,
'TEST_NAME': None,
'TIME_ZONE': 'Europe/Prague',
'USER': 'weblate'}}

DEFAULT_CONTENT_TYPE

'text/html'

FILE_UPLOAD_PERMISSIONS

None

FILE_UPLOAD_HANDLERS

('django.core.files.uploadhandler.MemoryFileUploadHandler',
'django.core.files.uploadhandler.TemporaryFileUploadHandler')

MT_APERTIUM_KEY

u'********************'

ENABLE_HOOKS

True

APPEND_SLASH

True

FIRST_DAY_OF_WEEK

0

DATABASE_ROUTERS

[]

WHOOSH_INDEX

'/var/www/weblate/weblate/whoosh-index'

STATICFILES_STORAGE

'django.contrib.staticfiles.storage.StaticFilesStorage'

CACHES

{'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
'LOCATION': ''}}

SERVER_EMAIL

'pavol.zibrita+weblateadmin@gmail.com'

SESSION_COOKIE_PATH

'/'

USE_X_FORWARDED_HOST

False

MIDDLEWARE_CLASSES

('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')

USE_I18N

True

THOUSAND_SEPARATOR

','

SECRET_KEY

u'********************'

LANGUAGE_COOKIE_NAME

'django_language'

LAZY_COMMITS

True

LOCK_TIME

900

DEFAULT_INDEX_TABLESPACE

''

GIT_ROOT

'/var/www/weblate/weblate/repos/'

TRANSACTIONS_MANAGED

False

LOGGING_CONFIG

'django.utils.log.dictConfig'

SIGNING_BACKEND

'django.core.signing.TimestampSigner'

SIMILAR_MESSAGES

5

TEMPLATE_LOADERS

('django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader')

WSGI_APPLICATION

None

TEMPLATE_DEBUG

True

X_FRAME_OPTIONS

'SAMEORIGIN'

AUTHENTICATION_BACKENDS

('django.contrib.auth.backends.ModelBackend',)

FORCE_SCRIPT_NAME

None

CACHE_BACKEND

'locmem://'

DEFAULT_CHARSET

'utf-8'

SESSION_COOKIE_SECURE

False

CSRF_COOKIE_DOMAIN

None

FILE_CHARSET

'utf-8'

DEBUG

True

SESSION_FILE_PATH

None

OFFLOAD_INDEXING

True

INSTALLED_APPS

('django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'registration',
'south',
'weblate.trans',
'weblate.lang',
'weblate.accounts',
'weblate')

LANGUAGES

(('cs', u'\u010cesky'),
('da', 'Dansk'),
('de', 'Deutsch'),
('en', 'English'),
('el', '\xce\x95\xce\xbb\xce\xbb\xce\xb7\xce\xbd\xce\xb9\xce\xba\xce\xac'),
('es', u'Espa\xf1ol'),
('fi', 'Suomi'),
('fr', u'Fran\xe7ais'),
('gl', 'Galego'),
('id', 'Indonesia'),
('ja', u'\u65e5\u672c\u8a9e'),
('nl', 'Nederlands'),
('pl', 'Polski'),
('pt_BR', u'Portugu\xeas brasileiro'),
('sl', u'Sloven\u0161\u010dina'),
('sv', u'Svenska'),
('tr', u'T\xfcrk\xe7e'),
('zh_CN', u'\u4e2d\u6587'))

COMMENTS_ALLOW_PROFANITIES

False

STATICFILES_DIRS

()

PREPEND_WWW

False

SECURE_PROXY_SSL_HEADER

None

PASSWORD_RESET_TIMEOUT_DAYS

u'********************'

SESSION_COOKIE_HTTPONLY

True

DEBUG_PROPAGATE_EXCEPTIONS

False

SITE_TITLE

'Weblate for OsmAnd'

MONTH_DAY_FORMAT

'F j'

LOGIN_URL

'/accounts/login/'

SESSION_EXPIRE_AT_BROWSER_CLOSE

False

TIME_FORMAT

'P'

DATE_INPUT_FORMATS

('%Y-%m-%d',
'%m/%d/%Y',
'%m/%d/%y',
'%b %d %Y',
'%b %d, %Y',
'%d %b %Y',
'%d %b, %Y',
'%B %d %Y',
'%B %d, %Y',
'%d %B %Y',
'%d %B, %Y')

CSRF_COOKIE_NAME

'csrftoken'

EMAIL_HOST_PASSWORD

u'********************'

AUTH_PROFILE_MODULE

'accounts.Profile'

CACHE_MIDDLEWARE_ALIAS

'default'

SESSION_SAVE_EVERY_REQUEST

False

ADMIN_MEDIA_PREFIX

'/static/admin/'

NUMBER_GROUPING

0

SESSION_ENGINE

'django.contrib.sessions.backends.db'

DEFAULT_FILE_STORAGE

'django.core.files.storage.FileSystemStorage'

CSRF_FAILURE_VIEW

'django.views.csrf.csrf_failure'

CSRF_COOKIE_PATH

'/'

LOGIN_REDIRECT_URL

'/'

LOGGING

{'disable_existing_loggers': False,
'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}},
'formatters': {'simple': {'format': '%(levelname)s %(message)s'},
'syslog': {'format': 'weblate[%(process)d]: %(levelname)s %(message)s'},
'verbose': {'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'}},
'handlers': {'console': {'class': 'logging.StreamHandler',
'formatter': 'simple',
'level': 'DEBUG'},
'mail_admins': {'class': 'django.utils.log.AdminEmailHandler',
'filters': ['require_debug_false'],
'level': 'ERROR'},
'syslog': {'address': '/dev/log',
'class': 'logging.handlers.SysLogHandler',
'facility': 18,
'formatter': 'syslog',
'level': 'DEBUG'}},
'loggers': {'django.request': {'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True},
'weblate': {'handlers': ['console'], 'level': 'DEBUG'}},
'version': 1}

IGNORABLE_404_URLS

()

LOCALE_PATHS

()

TEMPLATE_STRING_IF_INVALID

''

LOGOUT_URL

'/accounts/logout/'

EMAIL_USE_TLS

False

EMAIL_SEND_HTML

False

FIXTURE_DIRS

()

EMAIL_HOST

'localhost'

DATE_FORMAT

'N j, Y'

MEDIA_ROOT

'/var/www/weblate/weblate/media/'

DEFAULT_EXCEPTION_REPORTER_FILTER

'django.views.debug.SafeExceptionReporterFilter'

ADMINS

(('Pavol Zibrita', 'pavol.zibrita+weblate@gmail.com'),)

FORMAT_MODULE_PATH

None

DEFAULT_FROM_EMAIL

'pavol.zibrita+weblateadmin@gmail.com'

MT_MICROSOFT_KEY

u'********************'

AUTO_LOCK

True

MEDIA_URL

'/media/'

DATETIME_FORMAT

'N j, Y, P'

TEMPLATE_DIRS

('/var/www/weblate/weblate/html/',)

SITE_ID

1

DISALLOWED_USER_AGENTS

()

ALLOWED_INCLUDE_ROOTS

()

DECIMAL_SEPARATOR

'.'

SHORT_DATE_FORMAT

'm/d/Y'

TEST_RUNNER

'django.test.simple.DjangoTestSuiteRunner'

CACHE_MIDDLEWARE_KEY_PREFIX

u'********************'

TIME_ZONE

'Europe/Prague'

FILE_UPLOAD_MAX_MEMORY_SIZE

2621440

WEB_ROOT

'/var/www/weblate/weblate'

EMAIL_BACKEND

'django.core.mail.backends.smtp.EmailBackend'

DEFAULT_TABLESPACE

''

TEMPLATE_CONTEXT_PROCESSORS

('django.contrib.auth.context_processors.auth',
'django.core.context_processors.debug',
'django.core.context_processors.i18n',
'django.core.context_processors.media',
'django.core.context_processors.request',
'django.core.context_processors.csrf',
'django.contrib.messages.context_processors.messages',
'weblate.trans.context_processors.version',
'weblate.trans.context_processors.title',
'weblate.trans.context_processors.date',
'weblate.trans.context_processors.url',
'weblate.trans.context_processors.mt')

SESSION_COOKIE_AGE

1209600

SETTINGS_MODULE

'weblate.settings'

USE_ETAGS

False

LANGUAGES_BIDI

('he', 'ar', 'fa')

CHECK_LIST

('weblate.trans.checks.SameCheck',
'weblate.trans.checks.BeginNewlineCheck',
'weblate.trans.checks.EndNewlineCheck',
'weblate.trans.checks.EndSpaceCheck',
'weblate.trans.checks.EndStopCheck',
'weblate.trans.checks.EndColonCheck',
'weblate.trans.checks.EndQuestionCheck',
'weblate.trans.checks.EndExclamationCheck',
'weblate.trans.checks.PythonFormatCheck',
'weblate.trans.checks.PHPFormatCheck',
'weblate.trans.checks.CFormatCheck',
'weblate.trans.checks.PluralsCheck',
'weblate.trans.checks.ConsistencyCheck',
'weblate.trans.checks.DirectionCheck')

FILE_UPLOAD_TEMP_DIR

None

INTERNAL_IPS

()

STATIC_URL

'/static/'

EMAIL_PORT

25

USE_TZ

False

SHORT_DATETIME_FORMAT

'm/d/Y P'

PASSWORD_HASHERS

u'********************'

ABSOLUTE_URL_OVERRIDES

{}

CACHE_MIDDLEWARE_SECONDS

600

DATETIME_INPUT_FORMATS

('%Y-%m-%d %H:%M:%S',
'%Y-%m-%d %H:%M:%S.%f',
'%Y-%m-%d %H:%M',
'%Y-%m-%d',
'%m/%d/%Y %H:%M:%S',
'%m/%d/%Y %H:%M:%S.%f',
'%m/%d/%Y %H:%M',
'%m/%d/%Y',
'%m/%d/%y %H:%M:%S',
'%m/%d/%y %H:%M:%S.%f',
'%m/%d/%y %H:%M',
'%m/%d/%y')

EMAIL_HOST_USER

''

PROFANITIES_LIST

@pavolzibrita

Copy link
Copy Markdown

No, still errors. Very strange. Not using the escape/unescape in aresource does not make the problems, so the problem lies somewhere in there... I was not able to track it down, python is still a mistery for me...
The last error somehow is without correct stack trace if I understand it. Is there something I could do to get a better strack trace for you?

@pavolzibrita

Copy link
Copy Markdown

I just realized, the 'NoneType' object is not iterable is other bug in weblate (sync the template and translation), and probably it has nothing to do with this. However, it occurs more frequently in weblate if this escape/unescape is in place.

@nijel

nijel commented Sep 19, 2012

Copy link
Copy Markdown
Member Author

I believe that all mentioned bugs are rather issues in Weblate than in this patch.

@pavolzibrita

Copy link
Copy Markdown

Hm, this error now:


Environment:


Request Method: POST
Request URL: http://translate.osmand.net/admin/trans/subproject/add/

Django Version: 1.4
Python Version: 2.7.2
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'registration',
 'south',
 'weblate.trans',
 'weblate.lang',
 'weblate.accounts',
 'weblate')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Traceback:
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in wrapper
  366.                 return self.admin_site.admin_view(view)(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
  91.                     response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/views/decorators/cache.py" in _wrapped_view_func
  89.         response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/sites.py" in inner
  196.             return view(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapper
  25.             return bound_func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in _wrapped_view
  91.                     response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/utils/decorators.py" in bound_func
  21.                 return func(self, *args2, **kwargs2)
File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py" in inner
  209.                 return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in add_view
  955.                 self.save_model(request, new_object, form, False)
File "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/options.py" in save_model
  709.         obj.save()
File "/var/www/weblate/weblate/trans/models.py" in save
  1011.             self.create_translations()
File "/var/www/weblate/weblate/trans/models.py" in create_translations
  888.             translation = Translation.objects.update_from_blob(self, code, path, force)
File "/var/www/weblate/weblate/trans/managers.py" in update_from_blob
  106.         translation.update_from_blob(force)
File "/var/www/weblate/weblate/trans/models.py" in update_from_blob
  1362.                 newunit, is_new = Unit.objects.update_from_unit(self, unit, pos, template = template_unit)
File "/var/www/weblate/weblate/trans/managers.py" in update_from_unit
  152.         dbunit.update_from_unit(unit, pos, force, template)
File "/var/www/weblate/weblate/trans/models.py" in update_from_unit
  1866.         target = get_target(unit)
File "/var/www/weblate/weblate/trans/util.py" in get_target
  83.         if hasattr(unit.target, 'strings'):
File "/usr/lib/python2.7/dist-packages/translate/storage/aresource.py" in gettarget
  236.         target += ''.join([etree.tostring(child, encoding = 'utf-8') for child in self.xmlelement.iterchildren()])

Exception Type: UnicodeDecodeError at /admin/trans/subproject/add/
Exception Value: 'ascii' codec can't decode byte 0xe1 in position 19: ordinal not in range(128)
The string that could not be encoded/decoded was: /u> v���i �

@nijel

nijel commented Sep 20, 2012

Copy link
Copy Markdown
Member Author

Already fixed in f2abb1a :-)

@pavolzibrita

Copy link
Copy Markdown

Hm, I still get the latest error, also with the last change.

@pavolzibrita

Copy link
Copy Markdown

no, works fine, probably it was not recompiled before. Thanx!

@nijel

nijel commented Nov 14, 2012

Copy link
Copy Markdown
Member Author

I've just rebased my branch on top of current translate-toolkit, any chance to get this merged or reviewed?

@dwaynebailey

Copy link
Copy Markdown
Member

Hey nijel,

Thanks for the rebase. I've been trying to schedule time for fwolff to complete his review. He doesn't have much time at the moment. Sorry its taking so long but I do want to land this before we do another toolkit release.

@friedelwolff

Copy link
Copy Markdown
Member

Thanks for all the work and testing!

I hope to have a proper look soon. Here are a few notes from a quick look:

  • I guess pdb is unused
  • We don't currently put monolingual formats in the storage factory (the .xml extension will catch far too much anyway :-/ ) Applications can always add them programmatically.

Thanks for the tests and comments. I think there are very few good sources of documentation on this format, but if you have any, let's add that at the top of the file as well.

I'll try to verify tests on my side soon and integrate. Would it be ok if I squash all of this in one commit? (not that I feel strongly about that)

Thanks again.

@nijel

nijel commented Nov 27, 2012

Copy link
Copy Markdown
Member Author

Indeed pdb is not needed, it slipped in one merge request and I somehow forgot to delete it.

I have no problems with squashing my commits, but there are few contributions not from me and I think it would be fair to keep the correct attribution there.

@friedelwolff
friedelwolff merged commit 1b84744 into translate:master Dec 4, 2012
@friedelwolff

Copy link
Copy Markdown
Member

Thanks a lot for this work.

I merged and pushed it to master. I ended up not squashing, so your fork should still share the history, but in future, keep in mind that we might prefer to rebase your changes on master, in which case your fork would have to be recreated. In other words, doing this on a branch might be better, as I understand it.

This is a nasty file format. Thanks a lot for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants