New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3 support #16

Closed
goertzenator opened this Issue Feb 20, 2011 · 122 comments

Comments

Projects
None yet
@goertzenator

goertzenator commented Feb 20, 2011

I have Paramiko running on Python 3 at https://github.com/goertzenator/paramiko

All unit tests, except for sftp, pass. Needs documentation updates for the many str -> bytes changes.

Looking for advice on what to do with this. Adding a python 3 branch to paramiko would have a lot of complications...

@robey

This comment has been minimized.

robey commented May 22, 2011

there's no way to get code that will run on both python 2 and 3, is there? :(

@goertzenator

This comment has been minimized.

goertzenator commented May 22, 2011

Paramiko is much too large and io heavy to make that realistic. You could get the two codebases (paramiko py2 and py3) very similar if you started with a py3 version and downported to 2.6 or 2.7. But the current paramiko works well on 2.6 and older, so I don't see a lot of value in that.

@ramon-garcia

This comment has been minimized.

ramon-garcia commented Feb 8, 2012

Hello,

You can have a single codebase for python 2 and python 3. In the setup.py add the conversion 2to3. This is the recommended way. Add the following code to the setup.py

try:
from distutils.command.build_py import build_py_2to3 as build_py
except ImportError:
# 2.x
from distutils.command.build_py import build_py
...
setup(...
cmdclass = {'build_py':build_py}
)

This is the recommended way: http://wiki.python.org/moin/PortingPythonToPy3k

@goertzenator

This comment has been minimized.

goertzenator commented Feb 8, 2012

Using 2to3 was the first step I took for the port and it did convert a lot of things, but it didn't do anything for the massive str/bytes separation that had to take place.

Now I don't know a lot about it, but 3to2 might have more potential for paramiko.

@ramon-garcia

This comment has been minimized.

ramon-garcia commented Feb 8, 2012

I am working with it now. Since paramiko development seems to be stopped (no reply from the author for months), I am using the ssh fork https://github.com/bitprophet/ssh

@goertzenator

This comment has been minimized.

goertzenator commented Feb 8, 2012

Cool, let me know how you make out or if you have any questions.

@sciunto

This comment has been minimized.

sciunto commented Aug 19, 2012

Hi,

Is there any progress on that? Is https://github.com/bitprophet/ssh the best implementation in python3?

Thank you!

@bitprophet

This comment has been minimized.

Member

bitprophet commented Aug 20, 2012

@sciunto That 'ssh' fork will merge back into this one shortly, for what it's worth. It's also not Python3 compat yet, though there are patches.

Speaking personally, Python 3 support is a medium priority -- my hope is for Fabric 2.x to be Python 2.6=>3.x compatible, and that will require doing the same for some Paramiko version.

For Fabric and my other projects, I'm hoping to go the "one codebase that works on both Python versions" approach. As mentioned above, I'm not sure if that is feasible for Paramiko due to it being bigger, more complex and more IO oriented. I also haven't experienced the joy/sadness of 2to3/3to2 yet -- if they work and don't make maintenance a huge pain, awesome. If not, hard to say what will happen.

@sciunto

This comment has been minimized.

sciunto commented Aug 21, 2012

Alright. I'm going to limit my codes to some popen calls until things evolve.

Thanks a lot for your reply and your work

@bitprophet

This comment has been minimized.

Member

bitprophet commented Sep 26, 2012

For the record, the python 'ssh' lib has now merged back into Paramiko (as of tonight).

Nothing else has changed yet re: investigating the least painful Py3k route for Paramiko itself -- but as I start to work on Fabric 2.x, this will become increasingly important to me, so look for updates in the nearish future.

@takluyver takluyver referenced this issue Dec 21, 2012

Closed

python 3 #123

@timdawborn

This comment has been minimized.

timdawborn commented Jan 22, 2013

Any updates or an ETA on a py3k version?

@hamilton-z

This comment has been minimized.

hamilton-z commented Jan 30, 2013

any news about py3k version?

@bitprophet

This comment has been minimized.

Member

bitprophet commented Feb 3, 2013

No py3k news yet, it is still definitely planned (see my above comment), but other priorities have to come first :(

@jlegewie

This comment has been minimized.

jlegewie commented Feb 4, 2013

I will just add my encouragement here. paramiko is great and I would really like to see it for Python 3.

@aspineux

This comment has been minimized.

aspineux commented Mar 4, 2013

I wrote a python library pyzmail that work for both python 2 and 3.
I wrote it for 2.x then ran py2to3 on it. I made some changes to make it work for 2.x and be convertible by py2to3.
Then I have updated my setup.py to make it compatible with py2to3 ( py2to3 is run automatically when installing on 3.X, then no need to have two source tree )

The most difficult part is about unicode string and unicode literal. pyzmail use a lot of both. At 2 or 3 place I had no other choice than test check the python version and write specific code for both version.
The second one was to write doctest that work for both, because py2to3 don't convert comments.

paramiko don't make an heavy use of unicode string then it should be possible to use the same way.

@mvanderkolff

This comment has been minimized.

Contributor

mvanderkolff commented Mar 4, 2013

Not quite: Particularly in the agent stuff, it should be using
bytestrings.

Cheers,

Michael

On Mon, Mar 4, 2013 at 1:26 PM, Alain Spineux notifications@github.comwrote:

I wrote a python library pyzmail that work for both python 2 and 3.
I wrote it for 2.x then ran py2to3 on it. I made some changes to make it
work for 2.x and be convertible by py2to3.
Then I have updated my setup.py to make it compatible with py2to3 ( py2to3
is run automatically when installing on 3.X, then no need to have two
source tree )

The most difficult part is about unicode string and unicode literal.
pyzmail use a lot of both. At 2 or 3 place I had no other choice than test
check the python version and write specific code for both version.
The second one was to write doctest that work for both, because py2to3
don't convert comments.

paramiko don't make an heavy use of unicode string then it should be
possible to use the same way.


Reply to this email directly or view it on GitHubhttps://github.com//issues/16#issuecomment-14361382
.

@aspineux

This comment has been minimized.

aspineux commented Mar 4, 2013

By "using unicode", I mean having bytestring to convert to 'human' readable strings and vice versa.
98% of the string in paramiko are bytestrings; hostname, login and password are from the user and en then must be converted appropriately. SSH Keys must be loaded and saved the right way.
Thats all. This is not a complete rewrite.

@geertj

This comment has been minimized.

geertj commented Mar 6, 2013

I'd like to voice my support for this issue as well.

@bitprophet You should definitely go for the single code base approach. If you only need to support 2.6, 2.7 and 3.3+ this is very feasible.

@e0ne

This comment has been minimized.

e0ne commented Mar 9, 2013

It looks like need to support two different code bases: for python 2.x and 3.7. I'm trying to port paramiko to 3.3 but it really hard to suport 2.x and 3.x in one codebase.

@aspineux

This comment has been minimized.

aspineux commented Mar 9, 2013

What is your main problem ? Do you have some recurring piece of code that require some changes ? Does this make the code difficult to read ? Are you using 2to3 in POST-process (after you have made the original code for 2.x , 3.x ready )?

@e0ne

This comment has been minimized.

e0ne commented Mar 9, 2013

Yes, I'm using 2to3. Main problem is new types: no long and unicode more. I'm pretty sure that that it will broke 2.x code or code will be difficult to read with a lot of try-except blocks

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 14, 2013

I've been actively porting my (much smaller & newer) projects Spec and Invoke to a shared 2.6+3.3 style codebase using six, an approach that not only everybody who's already done this has recommended, but which does seem to work quite well in practice. The transition's been thorny but only because I hadn't done any work with Python 3.x prior so I'm both porting + learning.

I expect any final Paramiko Python 3.x support will end up using six and similar approaches, even if an initial 2to3 pass is used to help highlight trouble spots.

@FirefighterBlu3

This comment has been minimized.

FirefighterBlu3 commented Mar 15, 2013

@e0ne it's not that hard. instead of making crazy amounts of try/except blocks, make a function that has that in it. i maintain a daily use set of software that is approaching 10K lines and has to work on everything from 2.4 to 3.3

@FirefighterBlu3

This comment has been minimized.

FirefighterBlu3 commented Mar 15, 2013

also, i find it much easier to write for py3 and then handle legacy breakage as exceptions to the norm. it makes dealing with issues a lot easier.

@e0ne

This comment has been minimized.

e0ne commented Mar 15, 2013

@FirefighterBlu3 thanks for advise. I'll try it. I'm newbie in porting to python 3.x. I'm working on Lettuce and wont to get fabric/paramiko on python 3.3 in my project.
I don't like six because it makes python not so clear. Have a two branches for 2.x and 3.x is harder but code is better for me.

@FirefighterBlu3

This comment has been minimized.

FirefighterBlu3 commented Mar 15, 2013

no problem. i'd love to see a py3 capable paramiko. i'd port it but i'm really really pressed for time. i prefer a single branch

@tshepang

This comment has been minimized.

tshepang commented Feb 3, 2014

thanks for sayin' something @bitprophet

@sybrenstuvel

This comment has been minimized.

sybrenstuvel commented Mar 4, 2014

Please update the Fabric documentation; it now reads "Fabric is a Python (2.5 or higher) library", which implies 3.x support. Only after installing with pip and seeing syntax errors fly by, did I get suspicious about the 3.x support...

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 5, 2014

@sybrenstuvel Most folks realize Python 3 is a big break from Python 2...but clearly not everybody. Fixed: fabric/fabric@a722a7c - thanks!

I just completed #256 so it is now officially PYTHON 3 TIME and I will dive into the above PRs next time I have OSS time.

@asmeurer

This comment has been minimized.

asmeurer commented Mar 5, 2014

When you start actually using Python 3 you realize that it's not actually that big of a break.

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 5, 2014

Looks like #236 is the most recent attack on this so I will start by updating a copy of it, w/ latest master (main pain point is likely to be the docs changes from #256).

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 5, 2014

@asmeurer Sure, it's no Perl 6 ;) but it's enough of a barrier that I would never expect a project to default to supporting 3 if it just says "Python 2.5+". (Especially if it says 2.5+, given supporting 2.5 and 3 at the same time is even harder ;))

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 6, 2014

Mostly done applying #236's changes on top of latest master, taking notes as well. Plan is to present a new, clean PR with that changeset; discuss my notes on it (probably in that PR, just because it makes line comments easier); then iterate until we're satisfied.

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 7, 2014

#276 is that PR. Please put any interested eyes on that copy of the code, and I will be posting progress there as I hack on things.

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 8, 2014

We're basically done with #276, got test suite parity + did some cleanup + a little remaining that is being handled now.

What's left, IMO:

  • Changelog entry
  • Modify README/website/etc as required to note that Py3 support is now implemented
  • Possibly wait for a few more invested parties (that would be: you) to report that yes, the post-merge master branch works well for them.
  • Release Paramiko 1.13 (now that's an auspicious version number)
  • Handle any bugs-in-the-wild as they come up, in their own tickets.
@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 10, 2014

Was randomly poking https://caniusepython3.com/ and realized it's asserting ecdsa is not Py3 compatible. Should check that out; will be a hard blocker if it cannot be installed and/or imported (if it's only broken when actually used, then we can just deal for now as ECDSA isn't always required.)

@yegle

This comment has been minimized.

yegle commented Mar 10, 2014

https://github.com/warner/python-ecdsa
It says it support 3.2 and 3.3.

@tshepang

This comment has been minimized.

tshepang commented Mar 10, 2014

The latest released version (0.10) has no trove classifiers indicating which Python versions are supported, but there is a commit claiming support for both 2.x and 3.x.

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 13, 2014

Well that's good to hear then, thanks @yegle @tshepang!

Back on OSS today (it's...been a long week), hopefully will have this out on PyPI by EOD if I'm lucky.

bitprophet added a commit that referenced this issue Mar 13, 2014

@bitprophet

This comment has been minimized.

Member

bitprophet commented Mar 14, 2014

Merged into master!

Can't see a great reason to hold off on a 1.13 release; if it breaks, it breaks, and we bugfix.

@bitprophet bitprophet closed this Mar 14, 2014

@sybrenstuvel

This comment has been minimized.

sybrenstuvel commented Mar 14, 2014

Nice work guys!

The reason I posted a comment about the "2.5+" thing, is that I'm trying to get "X and newer" to actually mean "X and newer", and not "X and newer except fo other new stuff" ;-)

@jaraco

This comment has been minimized.

Contributor

jaraco commented Mar 14, 2014

Smart move to release and move forward. Too many projects get stymied by fear of issues.

@travis-bear

This comment has been minimized.

travis-bear commented Mar 20, 2014

Congrats. And thank you all.

@tomoh1r

This comment has been minimized.

tomoh1r commented Aug 1, 2014

👍 💯

@leafonsword

This comment has been minimized.

leafonsword commented Dec 3, 2014

Hope py3 supporting will come soon!

@tshepang

This comment has been minimized.

tshepang commented Dec 3, 2014

@leafonsword there is, since 1.13.0.

@leafonsword

This comment has been minimized.

leafonsword commented Dec 3, 2014

@tshepang
haha,Thx~

@ghost

This comment has been minimized.

ghost commented Jun 5, 2015

Error from python 3.4.3:
from Crypto.Util import _counter
ImportError: Cannot load specified object

@Russell-Jones

This comment has been minimized.

Russell-Jones commented Jun 6, 2015

@TeeBSD could you
a) provide more context. e.g. Why is an error from Crypto to do with paramiko? Which OS and version of paramiko? How was it installed?
b) open a new ticket for this issue if there isn't one already (I couldn't find one)

@Lazik

This comment has been minimized.

Lazik commented Jun 6, 2015

Paramiko uses pyCrypto.
It's more about how/which version of pyCrypto you installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment