--compiler build option on windows #18

Closed
vbabiy opened this Issue Mar 15, 2011 · 21 comments

Comments

Projects
None yet
@vbabiy
Contributor

vbabiy commented Mar 15, 2011

On Windows while installing from source code several packages (psycopg2, PIL, ...) emit an error:

error: Unable to find vcvarsall.bat

The issue is being fixed by installing MinGW and using mingw32 compiler like this:

setup.py install build --compiler=mingw32

As far as I see, it would be very good for PIP to have "--build-option" like "--install-option", so that one could use PIP with mingw32 compiler:

pip install --build-option="--compiler=mingw32" psycopg2

@vbabiy

This comment has been minimized.

Show comment
Hide comment
@vbabiy

vbabiy Mar 15, 2011

Contributor

I could solve the problem adding:

[build]

compiler=mingw32

to the file distutils.cfg in C:\Python27\Lib\distutils\

compilers-on-windows

config-files


Original Comment By: http://docs.python.org/install/index.html#using-non-microsoft-compilers-on-windows
Contributor

vbabiy commented Mar 15, 2011

I could solve the problem adding:

[build]

compiler=mingw32

to the file distutils.cfg in C:\Python27\Lib\distutils\

compilers-on-windows

config-files


Original Comment By: http://docs.python.org/install/index.html#using-non-microsoft-compilers-on-windows
@vbabiy

This comment has been minimized.

Show comment
Hide comment
@vbabiy

vbabiy Mar 15, 2011

Contributor

I would also like to see pip support a --build-option for the same reason. I
recently ran into the issue when using pip to install MarkupSafe and ended up
using a %USERPROFILE%\pydistutils.cfg file with the same content as the OP.


Original Comment By: Anonymous
Contributor

vbabiy commented Mar 15, 2011

I would also like to see pip support a --build-option for the same reason. I
recently ran into the issue when using pip to install MarkupSafe and ended up
using a %USERPROFILE%\pydistutils.cfg file with the same content as the OP.


Original Comment By: Anonymous
@vbabiy

This comment has been minimized.

Show comment
Hide comment
@vbabiy

vbabiy Mar 15, 2011

Contributor

I wonder does '--install-option' meet your requirements? I have not tried it,
but it looks like it will work. Note, it's "hidden" in the help output of the
install command.

# pip install --help

....


--install-option=INSTALL_OPTIONS

                        Extra arguments to be supplied to the setup.py

install

                        command (use like --install-option="--install-

                        scripts=/usr/local/bin").  Use multiple --install-

                        option options to pass multiple options to

setup.py

                        install.  If you are using an option with a

directory

                        path, be sure to use absolute path.

Original Comment By: Kelsey Hightower
Contributor

vbabiy commented Mar 15, 2011

I wonder does '--install-option' meet your requirements? I have not tried it,
but it looks like it will work. Note, it's "hidden" in the help output of the
install command.

# pip install --help

....


--install-option=INSTALL_OPTIONS

                        Extra arguments to be supplied to the setup.py

install

                        command (use like --install-option="--install-

                        scripts=/usr/local/bin").  Use multiple --install-

                        option options to pass multiple options to

setup.py

                        install.  If you are using an option with a

directory

                        path, be sure to use absolute path.

Original Comment By: Kelsey Hightower
@vbabiy

This comment has been minimized.

Show comment
Hide comment
@vbabiy

vbabiy Mar 15, 2011

Contributor

Using --install-option in the following manner doesn't currently work. I still
need to do something like http://jonforums.github.com/python.html as a
workaround.

C:\tools>pip --version pip 0.8.2 from c:\python27\lib\site-
packages\pip-0.8.2-py2.7.egg (python 2.7)

C:\tools>pip install --install-option="--compiler=mingw32" markupsafe
Downloading/unpacking markupsafe Running setup.py egg_info for package
markupsafe

Installing collected packages: markupsafe Running setup.py install for
markupsafe usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or:
-c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help

error: option --compiler not recognized


Original Comment By: http://jonforums.github.com/python.html
Contributor

vbabiy commented Mar 15, 2011

Using --install-option in the following manner doesn't currently work. I still
need to do something like http://jonforums.github.com/python.html as a
workaround.

C:\tools>pip --version pip 0.8.2 from c:\python27\lib\site-
packages\pip-0.8.2-py2.7.egg (python 2.7)

C:\tools>pip install --install-option="--compiler=mingw32" markupsafe
Downloading/unpacking markupsafe Running setup.py egg_info for package
markupsafe

Installing collected packages: markupsafe Running setup.py install for
markupsafe usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or:
-c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help

error: option --compiler not recognized


Original Comment By: http://jonforums.github.com/python.html
@pnasrat

This comment has been minimized.

Show comment
Hide comment
@pnasrat

pnasrat May 1, 2011

Contributor

See also issue #167

Contributor

pnasrat commented May 1, 2011

See also issue #167

@hickford

This comment has been minimized.

Show comment
Hide comment
@hickford

hickford Feb 2, 2012

Thanks vbabiy for the workaround. I too would like to see this option.

hickford commented Feb 2, 2012

Thanks vbabiy for the workaround. I too would like to see this option.

@gimmi

This comment has been minimized.

Show comment
Hide comment

gimmi commented Sep 10, 2012

+1

@felciano

This comment has been minimized.

Show comment
Hide comment

+1

@grigoryvp

This comment has been minimized.

Show comment
Hide comment
@grigoryvp

grigoryvp Oct 13, 2012

+1
Need some easy-to-use command-line way to install mercurial via pip on windows. Since Microsoft (c) (r) (tm) removed compiler from Windows 8 (c) (r) (tm) SDK, it's more important now.

+1
Need some easy-to-use command-line way to install mercurial via pip on windows. Since Microsoft (c) (r) (tm) removed compiler from Windows 8 (c) (r) (tm) SDK, it's more important now.

@wchorolque

This comment has been minimized.

Show comment
Hide comment

+1

@piotr-dobrogost

This comment has been minimized.

Show comment
Hide comment
@piotr-dobrogost

piotr-dobrogost Apr 1, 2013

Could somebody from core devs please comment on what's needed to move this forward? Is addition of --build-option option and passing its value to setup.py build the way to go?

Could somebody from core devs please comment on what's needed to move this forward? Is addition of --build-option option and passing its value to setup.py build the way to go?

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Apr 1, 2013

Contributor

pip doesn't currently call setup.py build directly to be able to pass the options.
more likely in the short term I think that we add build option support to "pip wheel", which is due out in pip-1.4?
not sure, see #881, @dholth , @pfmoore ?

Contributor

qwcode commented Apr 1, 2013

pip doesn't currently call setup.py build directly to be able to pass the options.
more likely in the short term I think that we add build option support to "pip wheel", which is due out in pip-1.4?
not sure, see #881, @dholth , @pfmoore ?

@dholth

This comment has been minimized.

Show comment
Hide comment
@dholth

dholth Apr 1, 2013

Member

Do you need to give the same options to every build? Can we ask you to do it manually for those non-default builds?

Bdist_wheel certainly may not be sophisticated enough to pass the arguments anyway......

Marcus Smith notifications@github.com wrote:

pip doesn't currently call setup.py build directly to be able to pass the options.
more likely in the short term I think that we add build option support to "pip wheel", which is due out in pip-1.4?
not sure, see #881, @dholth , @pfmoore ?


Reply to this email directly or view it on GitHub.

Member

dholth commented Apr 1, 2013

Do you need to give the same options to every build? Can we ask you to do it manually for those non-default builds?

Bdist_wheel certainly may not be sophisticated enough to pass the arguments anyway......

Marcus Smith notifications@github.com wrote:

pip doesn't currently call setup.py build directly to be able to pass the options.
more likely in the short term I think that we add build option support to "pip wheel", which is due out in pip-1.4?
not sure, see #881, @dholth , @pfmoore ?


Reply to this email directly or view it on GitHub.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Apr 1, 2013

Contributor

Can we ask you to do it manually for those non-default builds

what are you suggesting? how manually?

I guess somebody can do this?

pip install --no-install SomePackage
cd <venv>/build/SomePackage
python setup.py build --compiler=special-compiler
pip install --no-download SomePackage
Contributor

qwcode commented Apr 1, 2013

Can we ask you to do it manually for those non-default builds

what are you suggesting? how manually?

I guess somebody can do this?

pip install --no-install SomePackage
cd <venv>/build/SomePackage
python setup.py build --compiler=special-compiler
pip install --no-download SomePackage
@steveoh

This comment has been minimized.

Show comment
Hide comment

steveoh commented Dec 18, 2014

👍

@ax3l

This comment has been minimized.

Show comment
Hide comment
@ax3l

ax3l Feb 10, 2015

is introduction --build-option still on the table? I could need this for h5p's --mpi flag that can not be addressed via the environment either.

ax3l commented Feb 10, 2015

is introduction --build-option still on the table? I could need this for h5p's --mpi flag that can not be addressed via the environment either.

@pfmoore

This comment has been minimized.

Show comment
Hide comment
@pfmoore

pfmoore Feb 10, 2015

Member

It's a terrible hack, but you can abuse --global-option for this. Because --global-option values are added to the setup invocation before the install command, you can do things like

pip install --global-option build_ext --global-option --compiler=mingw32 foo

This constructs a command line that looks like

python setup.py build_ext --compiler=mingw32 install

(but if you say I told you to do this, I'll deny everything :-))

I suspect there won't be much change to what's currently available - the long-term goal is to make the build process independent of the install process, and in the meantime, the recommendation is really to build wheels. Either get the project to supply (Windows and/or OSX) wheels on PyPI, or if that's not practical, build wheels yourself locally (setup.py bdist_wheel should always be possible) and serve them from a local directory or a devpi instance.

Member

pfmoore commented Feb 10, 2015

It's a terrible hack, but you can abuse --global-option for this. Because --global-option values are added to the setup invocation before the install command, you can do things like

pip install --global-option build_ext --global-option --compiler=mingw32 foo

This constructs a command line that looks like

python setup.py build_ext --compiler=mingw32 install

(but if you say I told you to do this, I'll deny everything :-))

I suspect there won't be much change to what's currently available - the long-term goal is to make the build process independent of the install process, and in the meantime, the recommendation is really to build wheels. Either get the project to supply (Windows and/or OSX) wheels on PyPI, or if that's not practical, build wheels yourself locally (setup.py bdist_wheel should always be possible) and serve them from a local directory or a devpi instance.

@bgilbert bgilbert referenced this issue in openslide/openslide-python May 1, 2015

Closed

Cannot install on Python 2.7.9 (x64) + Windows 7 x64 #13

@fgregg fgregg referenced this issue in dirko/pyhacrf Sep 14, 2015

Closed

build error under conda on Windows 7 #22

@pradyunsg

This comment has been minimized.

Show comment
Hide comment
@pradyunsg

pradyunsg May 18, 2017

Member

@pfmoore So, the current recommended workaround is to download the package, build wheels manually and use the wheels?

Member

pradyunsg commented May 18, 2017

@pfmoore So, the current recommended workaround is to download the package, build wheels manually and use the wheels?

@pfmoore

This comment has been minimized.

Show comment
Hide comment
Member

pfmoore commented May 18, 2017

@pradyunsg

This comment has been minimized.

Show comment
Hide comment
@pradyunsg

pradyunsg May 18, 2017

Member

We have a strong reason not to do this (won't add a setuptools specific command to pip) and a workaround for this issue as well.

Can this issue be closed as a "won't fix", mentioning both of them in a closing comment?

Member

pradyunsg commented May 18, 2017

We have a strong reason not to do this (won't add a setuptools specific command to pip) and a workaround for this issue as well.

Can this issue be closed as a "won't fix", mentioning both of them in a closing comment?

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft May 18, 2017

Member

Seems reasonable to me, so I'm going to go ahead and close this. If someone wants it I suggest the best path forward is when we work on the abstract build system on distutils-sig, to chime in about wanting a way to control the compiler being used.

Member

dstufft commented May 18, 2017

Seems reasonable to me, so I'm going to go ahead and close this. If someone wants it I suggest the best path forward is when we work on the abstract build system on distutils-sig, to chime in about wanting a way to control the compiler being used.

@dstufft dstufft closed this May 18, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment