Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Python 3.3 #6

Closed
graingert opened this issue Jan 4, 2013 · 35 comments

Comments

@graingert
Copy link
Contributor

@graingert graingert commented Jan 4, 2013

Currently eventlet seems to support 2.4 - 2.7 however a lot of projects are moving to support Python 3 and 2 with a single code base using tools such as six.

I'd recommend dropping support for 2.4 - 2.6, and supporting both 2.7 and 3.3 using the six library.

@graingert

This comment has been minimized.

Copy link
Contributor Author

@graingert graingert commented Jan 4, 2013

There's probably some issues in particular regarding the changes between 2 and 3 in http://docs.python.org/2/library/urllib2.html

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Jan 4, 2013

Right, Python3 support would be really awesome. But as i said here https://plus.google.com/109869205442495270563/posts/BhRip2sG128 personally, i have no time for that.

If you are willing to take this issue, that would be super awesome.

Although, i must warn that we're not dropping 2.6 easily. And i'm still considering 2.5. While i hate old unsupported versions with passion, sad reality is, there are millions of installations of Debian, Ubuntu and what else with old Python packages. And we should create less obstacles for users of those.

Also, in my opinion, Python3 version must be done on separate code base, such way both versions are kept clean and reap benefits from their syntax, libraries, etc.

@graingert

This comment has been minimized.

Copy link
Contributor Author

@graingert graingert commented Jan 23, 2013

I think the first step to investigating py3k support is porting the 2.5-2.7 code-base to use python six and demonstrate tests working when converted to 3.3 with 2to3

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Jan 23, 2013

Yeah, that's a reasonable strategy also. Although, i think the differences in except clause syntax may be impossible to overcome. Good luck, anyway. Ask what you need, i'll try to help.

@ewindisch

This comment has been minimized.

Copy link
Contributor

@ewindisch ewindisch commented May 22, 2013

Dropping Python 2.5 support will make forward-compatibility a lot easier. There aren't that many changes between 2.6 and 2.7 to make it worth dropping 2.6, from the perspective of adding Python3 compatibility.

FYI - We're adding Python3 support to the OpenStack code base and eventlet is (presently) a critical dependency for us.

@edwardgeorge

This comment has been minimized.

Copy link
Member

@edwardgeorge edwardgeorge commented May 22, 2013

Yeah, that's a reasonable strategy also. Although, i think the differences in except clause syntax may be
impossible to overcome. Good luck, anyway. Ask what you need, i'll try to help.

just seen this comment because of the new one; it's ugly but the way you can overcome the except clauses differences is replacing:

except Exception, e:

with:

except Exception:
    e = sys.exc_info()[1]

if keeping 2.5 is important. not that i'm advocating that ugliness over dropping 2.5 support ;)

@creiht

This comment has been minimized.

Copy link
Contributor

@creiht creiht commented May 22, 2013

I think the argument for keeping the older python support was because the Second Life folks still needed it. The last time it came up was quite a while ago. If any of them are still around, it would be great to hear from them. Personally, I would love to drop the older 2.5 and below.

@pauloppenheim-liveloop

This comment has been minimized.

Copy link
Contributor

@pauloppenheim-liveloop pauloppenheim-liveloop commented May 23, 2013

Many web hosting companies are still at 2.5, namely Dreamhost:
http://wiki.dreamhost.com/Python

Debian 5.0 (Lenny) defaulted to 2.5, and was oldstable until earlier this month - http://wiki.debian.org/Python
CentOS5 / RHEL5 default to python 2.4 - https://www.centos.org/docs/5/

@jstasiak

This comment has been minimized.

Copy link
Contributor

@jstasiak jstasiak commented Aug 1, 2013

I haven't seen/used any live system supporting only 2.5 or lower for a while now (I used CentOS 5 but it supports 2.6 just fine), I feel that 2.5 support has to go in order to move forward.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Aug 5, 2013

My idea was to fix as many problems as possible before dropping any previously supported environments. And I think that is accomplished by now. With some exceptions, many issues are fixed.

So, yeah, let's move on. I will make announcement and then we can start working on code adjustments in branch.

@vikt0rs

This comment has been minimized.

Copy link
Contributor

@vikt0rs vikt0rs commented Nov 4, 2013

At the moment we are use MySQL-python==1.2.4 as mysql connector. But there is no python3 support in MySQL-python (see https://pypi.python.org/pypi/MySQL-python/1.2.4 ). What should we use instead of this module in python3 environment?

@graingert

This comment has been minimized.

Copy link
Contributor Author

@graingert graingert commented Nov 4, 2013

@achanda

This comment has been minimized.

Copy link

@achanda achanda commented Dec 4, 2013

@temoto I'd love to contribute to this. Let me know if there is any specific area I should focus at.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Dec 5, 2013

Great, thank you. Please, create a branch python3 and make all necessary
changes to make all tests pass in as many small commits as feasible. I will
merge the least intrusive changes first.
On Dec 4, 2013 10:55 PM, "Abhishek Chanda" notifications@github.com wrote:

@temoto https://github.com/temoto I'd love to contribute to this. Let
me know if there is any specific area I should focus at.


Reply to this email directly or view it on GitHubhttps://github.com//issues/6#issuecomment-29833500
.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Dec 5, 2013

@achanda you may wish to cooperate with @vikt0rs
he had put a lot of effort into Python3 support here #59

@vikt0rs

This comment has been minimized.

Copy link
Contributor

@vikt0rs vikt0rs commented Dec 10, 2013

@temoto I have a little experience for fork with github community :)
I guess, that you, as an author, should coordinate it anyway.

@achanda you can, for example, remove popen2 module from eventlet code - this module was removed in python3 - we should use subprocess (IMO) module instead. This also will work for python 2.5

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Dec 23, 2013

@vikt0rs of course it's my job to coordinate. I proposed him to join forces with your work instead of doing it all separately (which he has already started).

Suggested course: I will slowly pick easiest commits into eventlet master and you both can rebase your work on top.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Mar 25, 2014

Thanks to @vikt0rs @achanda @jstasiak we have great progress in Python3 support.

Everyone interested, please try this branch https://github.com/eventlet/eventlet/tree/python3 in both Python2 and Python3, because it must be compatible.
pip install https://github.com/eventlet/eventlet/archive/python3.zip

Comment about any issues. With positive feedback, this branch may land into next release, 0.15, which will make further progress a lot easier.

@vikt0rs

This comment has been minimized.

Copy link
Contributor

@vikt0rs vikt0rs commented Mar 26, 2014

@temoto - my 2 cents are:

    • I'm pretty sure, that this branch doesn't work on PY3 properly.
    • If somebody want to help with porting - I have a few question about socket._fileobject() implementation in python 3 :)
@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Mar 26, 2014

I want at least confirmations that it does not break Python2. There are lots of changes (like new except syntax) which should be merged into master as soon as possible.

@jstasiak

This comment has been minimized.

Copy link
Contributor

@jstasiak jstasiak commented Mar 26, 2014

Looks good to me on Python 2, I'll be able to test it with larger codebase tomorrow. When you merge this please consider merging 60ae403 in as well.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Mar 27, 2014

Thank you, @jstasiak . Tox/Travis changes are in python3 branch. Looking forward your feedback.

@jstasiak

This comment has been minimized.

Copy link
Contributor

@jstasiak jstasiak commented Mar 27, 2014

@temoto No problem. I did more testing and the branch still worked fine with Python 2.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Mar 28, 2014

Thank you. All current python3 work has been merged into master.

Everyone, please run

git fetch
for b in branch1 branch2; do
  git checkout $b
  git rebase origin/master

on all your branches to import Python3 compatibility changes (lots of small changes in almost all code base).

Now we can move forward more easily. Tests indicate some obvious and some obscure errors. Hopefully we can have Eventlet working on Python3 in next few months.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Mar 28, 2014

@ewindisch please share it with OpenStack team that Eventlet has some momentum and real progress into Python3 compatibility.

@chmouel

This comment has been minimized.

Copy link

@chmouel chmouel commented Mar 28, 2014

@temoto thanks for workign on that

@temoto temoto added the python3 label Jul 11, 2014
jstasiak added a commit that referenced this issue Oct 10, 2014
This includes changes to WSGI, websocket, bytes/str/unicode handling,
SSL, backdoor, greenio and tests.

Some comments and conditionals (PY2/PY3) were added for clarity

GH issues:

Closes #106
Closes #111
Closes #118
Closes #141

Incidentally should also close #135 (reopen if didn't)

cc #6
@jstasiak

This comment has been minimized.

Copy link
Contributor

@jstasiak jstasiak commented Oct 12, 2014

The number of incompatibilities is significantly down now, unfortunately I won't be able to spend more time on improving it further in the near future.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Oct 12, 2014

Thank you for great boost.

@jstasiak

This comment has been minimized.

Copy link
Contributor

@jstasiak jstasiak commented Oct 12, 2014

You're welcome. If that helps I think about half of remaining failures (other than teardown methods complaining about nonempty hub) is caused by test code rather than library code.

One of them that I looked at is WSGI test_close_before_finished test - apparently the hub is not empty after the test finishes (Python 3, it is empty on Python 2), if I add sleep(0) after this line the test passes but I didn't want to put it there not knowing the details.

fd.sendall(req.encode())

@jstasiak

This comment has been minimized.

Copy link
Contributor

@jstasiak jstasiak commented Oct 13, 2014

Oops, I forgot to push my last changes to master, I'll do it now

@kennell

This comment has been minimized.

Copy link
Contributor

@kennell kennell commented Oct 21, 2014

Progress is looking great. Thank you all for your effort so far.

jstasiak added a commit that referenced this issue Feb 12, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * increasing (!) sleep time in test_close_before_finished from 0 to
    0.01s, I suspect failures somehow specific to Python 3 were caused
    by the server thread not having the chance to do anything before the
    sleep(0) finished and all got killed in the tearDown cleanup
    procedure
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
jstasiak added a commit that referenced this issue Feb 13, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
jstasiak added a commit that referenced this issue Feb 13, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
jstasiak added a commit that referenced this issue Feb 13, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
jstasiak added a commit that referenced this issue Feb 13, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
jstasiak added a commit that referenced this issue Feb 13, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
jstasiak added a commit that referenced this issue Feb 13, 2015
This patch consists of the following changes:

* Splitting eventlet.greenio into base, py2 and py3 parts
  (eventlet.greenio should be exporing the same public objects). This
  change is motivated by the size and the number of conditions present
  in the current greenio code
* Connected to the first point: implementing almost completely new
  GreenPipe callable utilizing parts of old GreenPipe code but dropping
  _fileobject/SocketIO inheritance in favour of io.FileIO and making use
  of patched _pyio.open function which wraps raw file-like object in
  various readers and writers (they take care of the buffering,
  encoding/decoding etc.)
* Implementing (from scratch or updating existing versions)
  green versions of the following modules:

  * http.* (needed by Python 3's urllib)
  * selectors (Python >= 3.4, used in subprocess module)
  * urllib.* (needed by various tests and we were already exposing green
    urllib)

* Modifying some tests to make tests pass, which includes:

  * unicode/bytestring issues
  * modifying wsgi_test_conntimeout.py to not pass bufsize and close
    arguments to ExplodingSocketFile - on Python 3 it inherits from
    SocketIO, which doesn't deal with buffering at all as far as I can
    see

* Random cleaning up and reorganizing
* Requiring Python 3.x tests to pass for the whole build to pass

Known issues:

* code repetition
* naming inconsistencies
* possibly breaking some external code using private eventlet.greenio attributes

Closes #108

Affects #6 (I'd call it an
experimental support)

Should help for #145
Should help for #157
@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Feb 13, 2015

Thanks to huge amount of work by @jstasiak status of Eventlet Python3 support has changed from "something in development" straight to Beta. As in please try it in your real world applications because all tests pass. It will be part of v0.17 release this month.

If you find some particular issues, please don't hesitate to report them.

@temoto temoto closed this Feb 13, 2015
@temoto temoto removed the need-test label Feb 13, 2015
@vikt0rs

This comment has been minimized.

Copy link
Contributor

@vikt0rs vikt0rs commented Feb 23, 2015

@temoto should we also add python 3 to the classifiers list in setup.py to announce py3 support in pypi obviously?

@thedrow

This comment has been minimized.

Copy link
Contributor

@thedrow thedrow commented Feb 23, 2015

@temoto It's always a good idea.

@temoto

This comment has been minimized.

Copy link
Member

@temoto temoto commented Feb 25, 2015

@vikt0rs @thedrow done in v0.17.1 (on PyPi). Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.