Python3 port

[ericrector]

First thanks for the time and effort with this project!

Which python version does this run under? I am using, Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:19:30) [MSC v.1600 64 bit (AMD64)] on win32

When I run the scripts I get modules that are not found:
ImportError: No module named 'builtin'

I changed:

import builtin
to
import Builtins

(http://docs.pythonsprints.com/python3_porting/py-porting.html#name-changes)

then I get:
Traceback (most recent call last):
File "C:\xxxxxxxxxxxxxxxxx\bCNC.py", line 27, in
builtin._ = gettext.translation('bCNC', 'locale', fallback=True).ugettext
AttributeError: 'NullTranslations' object has no attribute 'ugettext'

also httplib is now urllib????

Do you have any advice?

[HomineLudens]

Any 2.7 version is working. Efforts are made to work with 3.3 but not high priority for now. Give a try and report any feedback.

[ericrector]

I'll see how I can help.

[vlachoudis]

@ericrector most of the functionality is already there with python3 (not fully tested yet). The main problem comes from the core sender routine, where the serial interface expects a "bytes" type in python3 and the strings are unicode, while in python2 expects a "str".
I didn't any good way of making it work for both versions with the same code, without making an extensive use of try: except:

[voneiden]

@vlachoudis I'm not in the loop on the details of the code in question, but I'll just throw this here.

Python 3.5.1 (default, Feb  2 2016, 15:28:26)
>>> type("foo".encode("utf8"))
<class 'bytes'>


Python 2.7.5 (default, May 29 2013, 21:11:16)
>>> type("foo".encode("utf8"))
<type 'str'>

Encoding a python 3 normal unicode string (str) results in bytes. Encoding a python 2 regular string (str) results in str. Encoding a python 2 unicode string (unicode) results in str. I have a hunch that's the method you've been looking for.

[vlachoudis]

Thanks @voneiden this could do the conversion. However there are a lot of checks in the code for "unicode" type that doesn't exist in python3 which raise NameError exceptions.
Maybe a dirty hack would be to create a builtin type unicode equal to str.
There are several places we have to place conditions for either python2 or python3.
All these will end up with a messed up code.
Personally I believe that the best option would be one day to move completely to python3 and stop supporting python2.

[voneiden]

Alright. I'll try to find some time to sit down and get familiar with the codebase, maybe I can come up with something. I agree on moving towards python3. Legacy python2 support would be nice to keep, but it should have minimal impact on the main code.

[vlachoudis]

I was a bit afraid of doing the jump right now. I wanted to get operational all the features I wanted the program to have, and then move to python3.

[voneiden]

OK, I'll keep an eye on the situation.

[ericrector]

Hi guys,

I didn't mean to open a can of worms. I can try to help. I have a strong
back ground in c, cpp and java. I am pretty new to python. I don't think
it will be too difficult to learn, but finding time, as you all know, is an
issue. Maybe someone has a suggestion on how we can work together to help
create a Python 3 version.

But if Vasilis wishes are to wait, I can respect that.

On Thu, Feb 11, 2016 at 11:07 AM, Matti Eiden notifications@github.com
wrote:

OK, I'll keep an eye on the situation.

—
Reply to this email directly or view it on GitHub
#228 (comment).

[vlachoudis]

@ericrector thanks for your offer to help. I just want to avoid having a code which is a patchwork in order to support both python versions while I am still some features are not fully implemented. I would like at some point to make the jump to python3 and stop the support of python2.

I would say the first step would be to modify all no-cricital parts to be python2 and python3 compatible. I am trying already to do this, but extra help is welcome. Then start working on the critical parts of the code and convert them only to python3 (especially the sender)

[DasWookie]

Launch on python v2.6.6 is throwing an error with latest code:
$ ./bCNC.py
Traceback (most recent call last):
File "./bCNC.py", line 2374, in
if sys.version_info.major != 2:
AttributeError: 'tuple' object has no attribute 'major'
$ python --version
Python 2.6.6

---

On Python version 2.7.9 the 'major' tuple object resolves, so no problem, and bCNC runs without any issues with the version check code.

[vlachoudis]

@DasWookie thanks for reporting. I've modified the code to ask the sys.version_info[0] instead of the .major. Now it should work also on 2.6

[atrueresistance]

Is support for 3 working now?

[vlachoudis]

Nope, a python 3 will require quite a lot of work and testing since the string now is unicode (2bytes), while on the controller it has to communicate with byte streams.

[atrueresistance]

@vlachoudis I've started a 100 days of code challenge and once I'm more familiar with the language in general I'll be looking into this.

I'm wondering if setting tosend in Sender.py def serialIO(self) to a byte type would be enough?

I use bCNC almost daily, so should be able to test on another laptop of mine.

[colinluthier]

Right now I can’t see any code that requires Python 3. Waiting seems to come without cost.

[Harvie]

Just for my internal record: https://www.root.cz/clanky/domena-cz-slavi-25-let-zacalo-to-souborem-hosts-txt/

find -name '*.py' -type f | while read i; do sed -i 's/\.\(de\|en\)code("utf8")//g' "$i"; done
python2 -m modernize -x libmodernize.fixes.fix_import -w -n --no-six .
python2 -m modernize -x libmodernize.fixes.fix_import -w -n --no-six  --future-unicode .
futurize -j 4 -w -n -1  .
futurize -j 4 -w -n -2  .
futurize -j 4 -w -n -0 -u  .
2to3-2.7 -j 4 -p -w -n -x import -x unicode .

var.encode()  #unicode to bytes
var.decode()  #bytes to unicode

>>> max(None, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>' not supported between instances of 'int' and 'NoneType'

[atrueresistance]

@Harvie I see you made a branch V3. Is this supposed to be the location to develop/test against?

owner@DESKTOP-75RC9P5 MINGW64 /d/bCNC/bCNC (v3)
$ python bCNC.py
Traceback (most recent call last):
  File "bCNC.py", line 1, in <module>
    __main__.py
NameError: name '__main__' is not defined

owner@DESKTOP-75RC9P5 MINGW64 /d/bCNC/bCNC (v3)
$ git branch
  master
* v3

owner@DESKTOP-75RC9P5 MINGW64 /d/bCNC/bCNC (v3)
$ python --version
Python 3.6.1

[Harvie]

@atrueresistance it's not really up to date. also we never got it to work properly. i've managed main window to show, but almost everything was broken.

[Harvie]

Good news everyone! I think i've fixed most of the problems with bCNC on python3. Now i need you all to heavily test latest git master version on both python2 and python3, so we can be sure it's ready for release!

[MARIOBASZ]

I'm glad, congratulations!

[Harvie]

OK guys. I've figured it out. Now serial connection should work well in both python2 and python3.
I am waiting for you to search for more issues (both py2 and py3) before i release this!

Problem is the spy:// pyserial mode is broken in ?raw mode. So until they fix it in pyserial/pyserial#467 , you have to use it without ?raw modifier on python3

[onekk]

This is working at least for python3

   def serial_write(self, data):
        """Serial write"""
        #print("W "+str(type(data))+" : "+str(data))

        if OCV.IS_PY3 is True:
            c_type = type(data)

            if OCV.COM_DEBUG is True:
                print("serial to send > ", data)
                print("serial to send type> ", c_type)

            if isinstance(data, bytes):
                ret = self.serial.write(data)
            elif isinstance(data, str):
                ret = self.serial.write(data.encode("ascii"))
        else:
            ret = self.serial.write(str(data))

        return ret

The logic is inverted respect to bCNC

as the problems arise only in python3 with the different str() treatment, the routine check for the data type and act accordingly, encoding only if needed.

the print statement pollutes the output but make clear when the problem arise during testing.

In some part bCNC send b"" strings that in python2 are treated as str() while in python3 are different data, but

• with this serial_write, maybe all the b"" occurencies in bCNC have to be reverted to normal str() and converted only if needed here?

• maybe even a serial_read would be written that permits to convert the data in only one place, as checking for str() and b"" is always false in python3?

Regards

Carlo D.

[Harvie]

@onekk Don't worry. I've got it working. Also i don't think i need custom serial debuging code, because pyserial has the spy:// feature...

Now it looks like this and i think it works nice:

        def serial_write(self, data):
                #print("W "+str(type(data))+" : "+str(data))

                #if sys.version_info[0] == 2:
                #        ret = self.serial.write(str(data))
                if isinstance(data, bytes):
                        ret = self.serial.write(data)
                else:
                        ret = self.serial.write(data.encode())

                return ret

[Harvie]

maybe all the b"" occurencies in bCNC have to be reverted to normal str()

@onekk i think bytes is preffered data type for serial communication. so don't really. if you specify the commands as bytes, you can be sure that they don't get messed up by encoding. Feel free to try latest git master. I think it works well.

[onekk]

maybe all the b"" occurencies in bCNC have to be reverted to normal str()

@onekk i think bytes is preffered data type for serial communication. so don't really. if you specify the commands as bytes, you can be sure that they don't get messed up by encoding. Feel free to try latest git master. I think it works well.

If you check with the print statements enabled, you will not that sometimes data are passed as str() and sometimes as b"" and this raise the problem, most of the time.

I've noted in the process of error checking that in python3 you could specify some parameters to the open() routine to do encodings there, maybe it is good to converted directly on input.

b"" data are not very friendly with the "".format() syntax, used in many places (bCNC uses the 'old' % syntax, but I presume it is working in the same manner).
so you can't write

b"something {0} {1:.4f}".format(vara, varb)

or even

b"G{0} X{1:.{4}f Y{2:.{4}f Z{3:{4}f}".format(cmd, X, Y, Z, CNC["digits"])

and is difficult to process all the "string" in bCNC as b"" data.

if you are sure that all data that reach serial_write are str() the python3 check for the type is useful.

Regards

Carlo D.

[Harvie]

So i've been trying the python3 version on real machine and it seems to work, however it produced following message:

Traceback (most recent call last):
  File "/home/harvie/Temp/bCNC/bCNC/__main__.py", line 2501, in monitorSerial
    self._monitorSerial()
  File "/home/harvie/Temp/bCNC/bCNC/__main__.py", line 2350, in _monitorSerial
    line = line.rstrip("\n")
TypeError: a bytes-like object is required, not 'str'

so i will have to fix it

update: i've fixed this

[Harvie]

So i've just published python3 compatible version 0.9.14.300 at pypi:
https://pypi.org/project/bCNC/

[Harvie]

Ok, python3 support seems to work, there might be some problems, but that is to be handled in separate issues, so i will close this one.