New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The --build-dir options doesn't work for local archive installs #804

Closed
hetmankp opened this Issue Feb 11, 2013 · 12 comments

Comments

Projects
None yet
6 participants
@hetmankp

hetmankp commented Feb 11, 2013

When installing from a local archive, the --build-dir option is ignored and instead a temporary directory is always used.

As far as I can tell this is decided in req.py, in InstallRequirement.build_location(), where the value supplied by --build-dir is only taken into consideration if self.req is not None. I'm not really sure what self.req is meant to represent so I can't submit a patch at this time (or decide if this is expected behaviour in some way).

Thanks for your feedback.

UPDATE: #1831 fixed this for wheel archives. the problem remains for local sdist installs.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Feb 11, 2013

Contributor

the parsing currently doesn't try to determine a "project name" just from a archive file alone, so therefore doesn't create a build dir using the name it didn't parse, but rather uses a tmp dir.

in most cases it should/could parse something I think. btw, the new project distlib has a good function for this: https://bitbucket.org/vinay.sajip/distlib/src/a586fab287abc1f34d1d5e251ef20ae19d1ad51e/distlib/util.py?at=default#cl-521 (@vsajip, what's the origins of this function and supported scope?)

for now though, a workaround is to install from the local archive using ---find-links like so:
(assuming it's using standard naming)

pip install --no-index --find-links=file://path/to/archive/dir/ name==version

Contributor

qwcode commented Feb 11, 2013

the parsing currently doesn't try to determine a "project name" just from a archive file alone, so therefore doesn't create a build dir using the name it didn't parse, but rather uses a tmp dir.

in most cases it should/could parse something I think. btw, the new project distlib has a good function for this: https://bitbucket.org/vinay.sajip/distlib/src/a586fab287abc1f34d1d5e251ef20ae19d1ad51e/distlib/util.py?at=default#cl-521 (@vsajip, what's the origins of this function and supported scope?)

for now though, a workaround is to install from the local archive using ---find-links like so:
(assuming it's using standard naming)

pip install --no-index --find-links=file://path/to/archive/dir/ name==version

@vsajip

This comment has been minimized.

Show comment
Hide comment
@vsajip

vsajip Feb 11, 2013

Contributor
Contributor

vsajip commented Feb 11, 2013

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Feb 11, 2013

Contributor

there's an equivalent that assumes you have a project name to use in parsing, but nothing that just parses in the abstract. btw, I'm using your function in a project at work. could pip just grab this w/o trying to use distlib for now?

Contributor

qwcode commented Feb 11, 2013

there's an equivalent that assumes you have a project name to use in parsing, but nothing that just parses in the abstract. btw, I'm using your function in a project at work. could pip just grab this w/o trying to use distlib for now?

@vsajip

This comment has been minimized.

Show comment
Hide comment
@vsajip

vsajip Feb 11, 2013

Contributor

could pip just grab this w/o trying to use distlib for now?

Sure, as long as you pass back any improvements you make ;-) Just kidding, but it would be nice if you did.

Contributor

vsajip commented Feb 11, 2013

could pip just grab this w/o trying to use distlib for now?

Sure, as long as you pass back any improvements you make ;-) Just kidding, but it would be nice if you did.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Feb 11, 2013

Contributor

my concern is to be clear what version schemes it's supposed to work for or not work for.... corner cases that fail, or work, but shouldn't etc.... I certainly pass back clarity/fixes related to that.

Contributor

qwcode commented Feb 11, 2013

my concern is to be clear what version schemes it's supposed to work for or not work for.... corner cases that fail, or work, but shouldn't etc.... I certainly pass back clarity/fixes related to that.

@vsajip

This comment has been minimized.

Show comment
Hide comment
@vsajip

vsajip Feb 11, 2013

Contributor

It's meant to be version-scheme agnostic. The version part will match any sequence of alphanumerics, underscores, period, plus and minus, which should cover most sane version numbers. I have been thinking about changing the regex so that project name components can't have '-' - at the moment it allows 'my.project.name' or 'my-project-name', but that won't work in the case where version is e.g. 'latest-0.9' and the composite is 'my-project-name-latest-0.9', unless a project name is passed in.

Although setuptools/distribute convert '-' in project names to '_' in file names of corresponding distributions, I don't think distutils does, so there are ambiguous filenames out there AFAIK.

Contributor

vsajip commented Feb 11, 2013

It's meant to be version-scheme agnostic. The version part will match any sequence of alphanumerics, underscores, period, plus and minus, which should cover most sane version numbers. I have been thinking about changing the regex so that project name components can't have '-' - at the moment it allows 'my.project.name' or 'my-project-name', but that won't work in the case where version is e.g. 'latest-0.9' and the composite is 'my-project-name-latest-0.9', unless a project name is passed in.

Although setuptools/distribute convert '-' in project names to '_' in file names of corresponding distributions, I don't think distutils does, so there are ambiguous filenames out there AFAIK.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Feb 26, 2013

Contributor

leaving open; we should update the --build help text to explain this.

Contributor

qwcode commented Feb 26, 2013

leaving open; we should update the --build help text to explain this.

@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Sep 25, 2014

Contributor

adding an important comment from #2060 (#2060 (comment) )

"even if it uses the temp build directory, there's no rationale for --no-clean to be ignored"

Contributor

qwcode commented Sep 25, 2014

adding an important comment from #2060 (#2060 (comment) )

"even if it uses the temp build directory, there's no rationale for --no-clean to be ignored"

@qwcode qwcode changed the title from The --build-dir options doesn't work for local archive installs to The --build-dir options doesn't work for local sdist archive installs Sep 25, 2014

@qwcode qwcode changed the title from The --build-dir options doesn't work for local sdist archive installs to The --build-dir options doesn't work for local archive installs Sep 25, 2014

@rgommers

This comment has been minimized.

Show comment
Hide comment
@rgommers

rgommers Nov 5, 2015

The cause of this issue as well as related ones like gh-3171 seems to be that the options dict is simply ignored when calling WheelBuilder here: https://github.com/pypa/pip/blob/develop/pip/commands/install.py#L288

rgommers commented Nov 5, 2015

The cause of this issue as well as related ones like gh-3171 seems to be that the options dict is simply ignored when calling WheelBuilder here: https://github.com/pypa/pip/blob/develop/pip/commands/install.py#L288

vbraun added a commit to vbraun/sage that referenced this issue Sep 24, 2016

Trac #21480: Make sagelib setup.py self-contained and independent of …
…SAGE_ROOT

This ticket changes the build process of sagelib in the following way:
 - `src/Makefile` delegates ALL building to `src/setup.py`
 - `src/setup.py` no longer depends on environment variables
`$SAGE_ROOT`, `$SAGE_SRC`, `$SAGE_DOC_SRC` etc. (to demonstrate this,
`Makefile` poisons these environment variables). It still depends on
`$SAGE_LOCAL` and environment variables that point below it.

This ticket is meant as:
 - preparation for VPATH builds of sage-the-distribution (#21469)
 - working towards the goal of making `sagelib` pip-installable -- see
#21507 for the eventual goal of having sagelib on PyPI
 - making the flow of directory information at build time clearer for
developers

 More specifically, the goal of this ticket is that only SAGE_LOCAL
needs to be set when the user does 'pip install' of the sagelib. (This
ticket almost achieves this, except it also needs SAGE_PKGS and
SAGE_CYTHONIZED to be set. The hope is that #20382 and other future
tickets will develop better mechanisms to communicate package and
directory information to the build.)

. . . . . . .

Some possibly useful information:
 - Documentation on distutils (https://docs.python.org/2/install/),
describing use of `--build-base` to do VPATH builds.
 - `pip install` keeps the source directory clean, building instead in a
temporary directory, by copying the sources.
 `pip install` also offers options `--build` to select a build
directory, but there are some pip issues:
[pypa/pip#2060 2060],
[pypa/pip#2053 2053],
[pypa/pip#804 804] that affect this
 - #14807 has some tricks to making VPATH builds work without copying
all python source files. But it uses automake instead of setup.py; we
will not do this in our ticket.

URL: https://trac.sagemath.org/21480
Reported by: mkoeppe
Ticket author(s): Matthias Koeppe
Reviewer(s): Jeroen Demeyer

vbraun added a commit to vbraun/sage that referenced this issue Oct 11, 2016

Trac #21480: Make sagelib setup.py self-contained and independent of …
…SAGE_ROOT

This ticket changes the build process of sagelib in the following way:
 - `src/Makefile` delegates ALL building to `src/setup.py`
 - `src/setup.py` no longer depends on environment variables
`$SAGE_ROOT`, `$SAGE_SRC`, `$SAGE_DOC_SRC` etc. (to demonstrate this,
`Makefile` poisons these environment variables). It still depends on
`$SAGE_LOCAL` and environment variables that point below it.

This ticket is meant as:
 - preparation for VPATH builds of sage-the-distribution (#21469)
 - working towards the goal of making `sagelib` pip-installable -- see
#21507 for the eventual goal of having sagelib on PyPI
 - making the flow of directory information at build time clearer for
developers

 More specifically, the goal of this ticket is that only SAGE_LOCAL
needs to be set when the user does 'pip install' of the sagelib. (This
ticket almost achieves this, except it also needs SAGE_PKGS and
SAGE_CYTHONIZED to be set. The hope is that #20382 and other future
tickets will develop better mechanisms to communicate package and
directory information to the build.)

. . . . . . .

Some possibly useful information:
 - Documentation on distutils (https://docs.python.org/2/install/),
describing use of `--build-base` to do VPATH builds.
 - `pip install` keeps the source directory clean, building instead in a
temporary directory, by copying the sources.
 `pip install` also offers options `--build` to select a build
directory, but there are some pip issues:
[pypa/pip#2060 2060],
[pypa/pip#2053 2053],
[pypa/pip#804 804] that affect this
 - #14807 has some tricks to making VPATH builds work without copying
all python source files. But it uses automake instead of setup.sh; we
will not do this in our ticket.

'''configure tarball''':
http://sage.ugent.be/www/jdemeyer/sage/configure-185.tar.gz

URL: https://trac.sagemath.org/21480
Reported by: mkoeppe
Ticket author(s): Matthias Koeppe
Reviewer(s): Jeroen Demeyer
@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Mar 24, 2017

Member

Closing this, with #4371 we're going to be deprecating/removing the --build option.

Member

dstufft commented Mar 24, 2017

Closing this, with #4371 we're going to be deprecating/removing the --build option.

@dstufft dstufft closed this Mar 24, 2017

@jdemeyer

This comment has been minimized.

Show comment
Hide comment
@jdemeyer

jdemeyer Aug 22, 2017

Please reopen, this issue still remains.

jdemeyer commented Aug 22, 2017

Please reopen, this issue still remains.

@jdemeyer

This comment has been minimized.

Show comment
Hide comment
@jdemeyer

jdemeyer Aug 22, 2017

I'm not sure what "local archive installs" in the title means here, but at least --build is not supported when installing from a local directory:

$ pip install --verbose --upgrade --build=/tmp/pip .
Processing /home/jdemeyer/math/cylinder/src
  Running setup.py (path:/tmp/pip-hyhaPT-build/setup.py) [...]

jdemeyer commented Aug 22, 2017

I'm not sure what "local archive installs" in the title means here, but at least --build is not supported when installing from a local directory:

$ pip install --verbose --upgrade --build=/tmp/pip .
Processing /home/jdemeyer/math/cylinder/src
  Running setup.py (path:/tmp/pip-hyhaPT-build/setup.py) [...]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment