Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

# Is there a duck-typing way to know we can't compile extensions? #80

Closed
nedbat opened this issue Aug 10, 2010 · 16 comments
Closed

# Is there a duck-typing way to know we can't compile extensions? #80

nedbat opened this issue Aug 10, 2010 · 16 comments
Labels
enhancement New feature or request install

Comments

@nedbat
Copy link
Owner

nedbat commented Aug 10, 2010

Originally reported by Zooko O'Whielacronx (Bitbucket: zooko, GitHub: zooko)


Is there a duck-typing way to know we can't compile extensions?

Yes! You try to build the extension and if it fails then you automatically fall back to using your pure-Python alternative. This way install will also succeed on a system without a C compiler, for example.

The example code that I crib from whenever I'm going to use this hack is simplejson:

http://simplejson.googlecode.com/svn/trunk/setup.py


@nedbat
Copy link
Owner Author

nedbat commented Feb 13, 2011

I'm experimenting with this code:

try:
    setup(**setup_args)
except:
    del setup_args['ext_modules']
    setup(**setup_args)

It works, but might be a bit over-zealous in catching exceptions. The simplejson code catches BuildFailed, but I find on Windows that when I can't install the extension, it fails with ValueError:

...
  File "c:\python26\Lib\distutils\msvc9compiler.py", line 458, in compile
    self.initialize()
  File "c:\python26\Lib\distutils\msvc9compiler.py", line 368, in initialize
    vc_env = query_vcvarsall(VERSION, plat_spec)
  File "c:\python26\Lib\distutils\msvc9compiler.py", line 284, in query_vcvarsall
    raise ValueError(str(list(result.keys())))
ValueError: [u'path']

@nedbat
Copy link
Owner Author

nedbat commented Feb 14, 2011

Oops, sorry, just read further into the simplejson code, it throws its own BuildFailed error. I'll look into using its technique.

@nedbat
Copy link
Owner Author

nedbat commented Feb 14, 2011

Fixed in <<changeset 740b68eb8fc5 (bb)>>.

@nedbat
Copy link
Owner Author

nedbat commented Nov 20, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


This still fails if no gcc is present: http://dpaste.org/A1WwH/

@nedbat
Copy link
Owner Author

nedbat commented Nov 20, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


This is under zc.buildout.

@nedbat
Copy link
Owner Author

nedbat commented Nov 20, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


Probably needs to use https://bitbucket.org/zzzeek/sqlalchemy/src/b91c1cc4c893741443534a112b1d2d74321fce64/setup.py?at=default#cl-115

@nedbat
Copy link
Owner Author

nedbat commented Nov 20, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


I'm afraid zc.buildout doesn't pass "install" to setup.py, taking the route sqlalchemy does would stop such issues.

@nedbat
Copy link
Owner Author

nedbat commented Nov 21, 2012

Domen, thanks for the extra information. I'm afraid I've never used zc.buildout, can you provide very detailed instructions on how to reproduce this? You mention zc.buildout doesn't pass "install" to setup.py, how can that be right? Maybe I don't understand what this is all meant to accomplish. I don't see how the sqlalchemy code gets around not having a verb passed to it.

@nedbat
Copy link
Owner Author

nedbat commented Nov 21, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


Sure:

  • git clone git://gist.github.com/4122350.git
  • cd 4122350 (bb)
  • python bootstrap.py
  • bin/buildout

(make sure you don't have gcc installed)

@nedbat
Copy link
Owner Author

nedbat commented Nov 21, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


So what SQLAlchemy does different that is more bullet-proof is the following: https://bitbucket.org/zzzeek/sqlalchemy/src/b91c1cc4c893741443534a112b1d2d74321fce64/setup.py?at=default#cl-61

@nedbat
Copy link
Owner Author

nedbat commented Nov 24, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


Would you accept pull-request to fix this?

@nedbat
Copy link
Owner Author

nedbat commented Nov 26, 2012

I would accept a pull request, provided I could understand it, and it supported Py2.3 - Py3.3, and worked on Windows, Linux, and Mac. This part of the setup.py doesn't have automated tests, but some guidance on how to make sure it keeps working would be good too.

@nedbat
Copy link
Owner Author

nedbat commented Nov 26, 2012

Original comment by Zooko O'Whielacronx (Bitbucket: zooko, GitHub: zooko)


One way to reduce the risks of undetected breakage on different platforms would be to hew as closely as possible to what other packages do.

Simplejson is very widely used and actively supported, so it's setup.py probably works:

https://github.com/simplejson/simplejson/blob/master/setup.py

By the way, please add PyPy to your list of platforms that you don't want to break!

@nedbat
Copy link
Owner Author

nedbat commented Nov 26, 2012

Original comment by Domen Kožar (Bitbucket: iElectric, GitHub: iElectric)


(Reply via do...@dev.si):

simplejson is doing the same thing: catching exception early and raising
BuildFaild that is caught later on.

On Mon, Nov 26, 2012 at 4:46 PM, Zooko O'Whielacronx <
issues-reply@bitbucket.org> wrote:

@nedbat
Copy link
Owner Author

nedbat commented Dec 15, 2012

Hmm, I have a complaint that 3.5.3 fails where 3.5.2 worked. Maybe simplejson's way is the way to go.

@nedbat
Copy link
Owner Author

nedbat commented Dec 17, 2012

Handled in the way SQLAlchemy and simplejson do (they use the same way!) in <<changeset fd9b29a3383188ae87692fdd0a030f4f2ceca858 (bb)>>.

@nedbat nedbat closed this as completed Dec 17, 2012
@nedbat nedbat added major enhancement New feature or request install labels Jun 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request install
Projects
None yet
Development

No branches or pull requests

1 participant