Couple issues with Web2py converter

[pgcd]

I just tried using the Web2py converter and it seems to have a couple of issues:

• At first it wouldn't even run because of an unexpected keyword argument (duplicates)
• After changing that to accept **kwargs, I found that it choked on utf-8 strings (in my case, with Umlauts).

In any case, I have made a couple changes and now it works, at least for me - but since I haven't done any proper testing, I don't feel like creating an actual pull request. On the other hand, maybe somebody else can take advantage of this, so here's a gist with the changes I made to web2py2po.py
https://gist.github.com/pgcd/52338762bdcb4728cd39

[petterreinholdtsen]

I see something similar using python-translate version 2.0.0-1 in Debian stable.

When trying to use web2py2po and po2web2py with the translations in https://github.com/OneZoom/OZtree , the tools fail to work.

First, trying to create a po file fail like this:

  % web2py2po  languages/it.py languages/it-new.po processing 1 files...
  web2py2po: WARNING: Error processing: input languages/it.py, output \
    languages/it-new.po, template None: convertpy() got an unexpected keyword \
    argument 'duplicatestyle'
  [###########################################] 100%
  %

Next, after using my own tool to convert from .py to .py, converting
back to py also fail like this:

  % po2web2py languages/it.po languages/it.py 
  processing 1 files...
  po2web2py: WARNING: Error processing: input languages/it.po, output languages/it.py, \
    template None: 'ascii' codec can't encode character u'\xf9' in position 34: \
    ordinal not in range(128)
  [###########################################] 100%
  %

Is the web2py handling completely broken?

[petterreinholdtsen]

Looking at the gist, this seem to be the relevant change:

--- /usr/lib/python2.7/dist-packages/translate/convert/web2py2po.py     2017-01-27 20:51:52.000000000 +0100
+++ web2py2po.py        2018-12-18 08:44:51.392384401 +0100
@@ -46,7 +46,7 @@
         targetheader.addnote("extracted from web2py", "developer")
 
         for source_str in mydict.keys():
-            target_str = mydict[source_str]
+            target_str = mydict[source_str].decode('utf-8')
             if target_str == source_str:
                 # a convention with new (untranslated) web2py files
                 target_str = u''
@@ -59,7 +59,7 @@
         return self.mypofile
 
 
-def convertpy(inputfile, outputfile, encoding="UTF-8"):
+def convertpy(inputfile, outputfile, encoding="UTF-8", **kwargs):
 
     new_pofile = po.pofile()
     convertor = web2py2po(new_pofile)

[petterreinholdtsen]

My issue is also reported to Debian as https://bugs.debian.org/916745 .

Note, my testing show that the patch help. I also patched the "mydict = eval(inputfile.read())" line to "mydict = eval(inputfile.read().decode('utf-8'))" to avoid a python warning about not being able to compare two strings.

[petterreinholdtsen]

The #3838 pull request also try to fix this issue.

[petterreinholdtsen]

Any hope to have this problem fixed in master?

[petterreinholdtsen]

I am happy to discover that the issue is fixed in Debian (in version 2.3.1-3), at least. I tested the new version there today, and it seem to work well. I hope it will get fixed up stream too, soon.

[petterreinholdtsen]

Is this code no longer mantained. There seem to be no response from any maintainer for four years?

[unho]

Fixed with code from PR #3838 landed in master.