Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Version 1.12.2 does not work with Python 3(.3) because of 2to3 issues #143

Closed
ptwobrussell opened this Issue · 13 comments

3 participants

@ptwobrussell

In trying to pip install PyGithub with Python 3.3 just now, I see the following in the console:

Downloading/unpacking PyGithub
Downloading PyGithub-1.12.2.tar.gz (1.6MB): 1.6MB downloaded
Running setup.py egg_info for package PyGithub

Installing collected packages: PyGithub
Running setup.py install for PyGithub
Fixing build/lib/github/init.py build/lib/github/AuthenticatedUser.py

...

  File "/Users/matthew/virtual-environments/mtsw2e/lib/python3.3/site-packages/github/AuthenticatedUser.py", line 16
    from . import github.GithubObject
                        ^
SyntaxError: invalid syntax

.... more of the same errors ...

In looking at the source on GitHub for this version (such as https://github.com/jacquev6/PyGithub/blob/v1.12.2/github/UserKey.py), it appears that you are just doing straight imports as "import github.X" which is causing 2to3 to produce invalid imports when it rewrites.

@bilderbuchi

I installed it a couple of days ago, for python 3.2, worked without problems (ubuntu 12.10 64bit, using pip-3.2)
I believe the import syntax change is made automatically by python 2to3 - maybe this is malfunctioning on your end somehow?
I just checked, and in that file, I have import github.something-type import statements in both 3.2 and 2.7 versions.

edit: I just realized that I'm on the machine where I only have 1.12.1 installed, so my info will not be as relevant. I can check on the other machine tomorrow.

@ptwobrussell

I've done a little more digging, and discovered a couple of things that may be helpful. First, running 2to3 on a file such as UserKey.py produces a file that is simply not Python 3 compatible, so that confirms the source of what was happening above once I figured out that you were using 2to3 in setup.py.

I tried fiddling around with some of the 2to3 options such as "-x import" to see if this would result in fixes, but I couldn't see any silver bullets because of the way you use references throughout your file(s) to the github package in fairly liberal ways. There may be some other kind of fix, but all I could come up with was to make a few fundamental changes to the way you import and reference modules in packages so that it works with Python 2 and doesn't cause 2to3 to try to "help". Referencing the UserKey.py file as an example, there are just a few changes that would make be required to work with Python 2 as-is and still work with Python 3.3 after running 2to3. Basically, just stop referencing "github" in your imports and references:

$ diff UserKey.py.original UserKey.py.modified
16c16
< import github.GithubObject
---
> import GithubObject
19c19
< class UserKey(github.GithubObject.GithubObject):
---
> class UserKey(GithubObject.GithubObject):
76c76
<     def edit(self, title=github.GithubObject.NotSet, key=github.GithubObject.NotSet):
---
>     def edit(self, title=GithubObject.NotSet, key=GithubObject.NotSet):

I've omitted the few other changes that would have followed that 2to3 made for brevity and tested that the resulting changes work with Python 2 and 3.

I'm no expert on best practices for managing Python package references, so there might be better ways, but I thought I'd pass this on since it might be helpful and save you some time in diagnostics. If this is indeed the "best" approach, the good news is that it looks like it's a fairly mechanical set of changes. The bad news is that it is systemic and you'll have to touch a lot of files.

@jacquev6 jacquev6 was assigned
@bilderbuchi

just to clarify, I'm not the dev of this package - @jacquev6 is. ;-)
2to3 is a hunch of mine from what I saw on installing it - and afaik using 2to3 is one kind of standard practice to maintain python packages which are compatible with Py2 and Py3 from one codebase.

@jacquev6
Owner

This is very strange, because I had to refactor the imports so that 2to3 works... See commits 9a03610, which was exactly the opposite of what you propose...

And it works on Travis: https://travis-ci.org/jacquev6/PyGithub/jobs/5206619, and on Ubuntu as stated by @bilderbuchi...

But I acknowledge your issue and I will try to understand it. I have just tried Cygwin's 2to3 and python3, and I have the same error. I don't have access to Linux right now, but I will check soon.

@bilderbuchi

note that my working Ubuntu version is on Py3.2.3, and the OP is on 3.3 - maybe that makes a difference?

@ptwobrussell
@bilderbuchi

Could it possibly be this bug? Although I don't see why this should only be exposed in 3.3...

@ptwobrussell

It certainly looks like the same issue, though, like you said, the 3.2 vs 3.3 difference doesn't make sense. Then again, I haven't tested it with 3.2, so the same issue could be the case on OS X perhaps. If it's helpful to know, I installed Python 3 with homebrew if that makes any difference.

@jacquev6
Owner

Wow... I think I've understood! 2to3 does not behave the same on case-insensitive filesystems, because I have a package named 'github' and a module 'Github.py'. That's why 2to3 generates bad imports on my Cygwin install, but works on Linux.

@ptwobrussell Do you confirm your Mac OS filesystem is case-insensitive?

@ptwobrussell

Wow. Good catch. I've never really thought about it but it does appear that the file system on Mac is case insenstive:

Goblin:tmp matthew$ touch foo
Goblin:tmp matthew$ ls -la foo
-rw-r--r--  1 matthew  wheel  0 Mar 11 17:18 foo
Goblin:tmp matthew$ ls -la Foo
-rw-r--r--  1 matthew  wheel  0 Mar 11 17:18 Foo
Goblin:tmp matthew$ 

However, there is no file Foo in that directory -- only a foo file exists.

@bilderbuchi

wow, that's really surprising - I was not aware of that. I wonder though, how does 2to3 conversion work on Windows? Or does it not?

edit: apparently the macOS file system can be case-sensitive or not, depending on user choice. there's also a way to find out which your file system is mentioned.

@jacquev6
Owner

@bilderbuchi I don't have a native Python3 on the Windows PC I'm using right now, but I'll check soon.

@jacquev6
Owner

I've just checked, 2to3 on Windows has the same issue. That's coherent.

@jacquev6 jacquev6 closed this
@ptwobrussell ptwobrussell added the v1 label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.