New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pypi: install packages using --user #834

Merged
merged 7 commits into from Feb 4, 2019

Conversation

Projects
None yet
5 participants
@ssbarnea
Copy link
Contributor

ssbarnea commented Jul 9, 2018

Avoids failure to install due to lack of sudo access.

ssbarnea added some commits Jul 9, 2018

pypi: install packages using --user
Avoids failure to install due to lack of sudo access.

@ssbarnea ssbarnea referenced this pull request Jul 9, 2018

Open

Heads-up about new PyPI #779

pypi: detect virtualenv
Avoids failure on virtualenvs:

Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
Couldn't install pip, setuptools, twine or wheel.

Makes pip download less verbose.

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
@webknjaz

This comment has been minimized.

Copy link
Contributor

webknjaz commented Jul 9, 2018

@ssbarnea you'll need to do an example build with your changes.

@@ -66,7 +66,9 @@ def pypi_wheel_arg
end

def install_deploy_dependencies
unless context.shell "wget -O - https://bootstrap.pypa.io/get-pip.py | python - --no-setuptools --no-wheel && " \
# --user likely fails inside virtualenvs but is needed outside to avoid needing sudo.
unless context.shell "if [ -z ${VIRTUAL_ENV+x} ]; then export PIP_USER=yes; fi && " \

This comment has been minimized.

@webknjaz

webknjaz Jul 9, 2018

Contributor

This creates hard to understand mix of languages. I'd defer env checks to Ruby side.

This comment has been minimized.

@ssbarnea

ssbarnea Jul 9, 2018

Author Contributor

@webknjaz I totally agree with you here. The only problem is that I do not master ruby at all, so if you give me an extra hint on how to do it, I would not mind doing it. Mainly I need to define this new envvar if the other one is defined.

This comment has been minimized.

@webknjaz

webknjaz Jul 9, 2018

Contributor

you can probably check context.env['VIRTUAL_ENV']

This comment has been minimized.

@BanzaiMan

BanzaiMan Oct 18, 2018

Member

What is PIP_USER=yes supposed to do?

This comment has been minimized.

unless context.shell "wget -O - https://bootstrap.pypa.io/get-pip.py | python - --no-setuptools --no-wheel && " \
# --user likely fails inside virtualenvs but is needed outside to avoid needing sudo.
unless context.shell "if [ -z ${VIRTUAL_ENV+x} ]; then export PIP_USER=yes; fi && " \
"wget -nv -O - https://bootstrap.pypa.io/get-pip.py | python - --no-setuptools --no-wheel && " \

This comment has been minimized.

@webknjaz

webknjaz Jul 9, 2018

Contributor

What does -nv do?

This comment has been minimized.

@ssbarnea

ssbarnea Jul 9, 2018

Author Contributor

It disables the progressbar on wget download (cutting ~20 lines of useless console log).

ssbarnea added some commits Jul 9, 2018

Fixed default upload url
The default pypi url was put down after deprecation and now longer works.

New URL is the current production URL.

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
avoid mentioning an url for twine
Allows twine to use its own default server url
instead of hardcoding own on default value for it.

Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Increse dpl version
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
@webknjaz

This comment has been minimized.

Copy link
Contributor

webknjaz commented Jul 11, 2018

@ssbarnea test specs need to be updated.

@@ -1,3 +1,3 @@
module DPL
VERSION = '1.9.7'
VERSION = '1.9.8'

This comment has been minimized.

@BanzaiMan

BanzaiMan Oct 18, 2018

Member

This is not essential to the PR, so please drop this.

This comment has been minimized.

@BanzaiMan

BanzaiMan Feb 4, 2019

Member
Suggested change Beta
VERSION = '1.9.8'
VERSION = '1.9.7'
@@ -66,7 +66,9 @@ def pypi_wheel_arg
end

def install_deploy_dependencies
unless context.shell "wget -O - https://bootstrap.pypa.io/get-pip.py | python - --no-setuptools --no-wheel && " \
# --user likely fails inside virtualenvs but is needed outside to avoid needing sudo.
unless context.shell "if [ -z ${VIRTUAL_ENV+x} ]; then export PIP_USER=yes; fi && " \

This comment has been minimized.

@BanzaiMan

BanzaiMan Oct 18, 2018

Member

What is PIP_USER=yes supposed to do?

@@ -1,7 +1,7 @@
module DPL
class Provider
class PyPI < Provider
DEFAULT_SERVER = 'https://upload.pypi.org/legacy/'
DEFAULT_SERVER = false

This comment has been minimized.

@BanzaiMan

BanzaiMan Oct 18, 2018

Member

What is the reason for not defaulting to this server?

This comment has been minimized.

@ssbarnea

ssbarnea Oct 18, 2018

Author Contributor

I know for sure that the /legacy/ needs to be removed and using defaults is ok pypi. I had something similar to do with twine (pypi python client).

This comment has been minimized.

@BanzaiMan

BanzaiMan Oct 18, 2018

Member

Thanks for the clarification.

@BanzaiMan

This comment has been minimized.

Copy link
Member

BanzaiMan commented Oct 18, 2018

Has this been tested?

@BanzaiMan

This comment has been minimized.

Copy link
Member

BanzaiMan commented Oct 18, 2018

So, let us try testing this on 4 configurations, each combination of 2 languages (Python and Node.js), and two infrastructures (GCE with sudo: required and EC2 with sudo: false).

@BanzaiMan

This comment has been minimized.

Copy link
Member

BanzaiMan commented Oct 29, 2018

If anyone is interested in testing this, use:

deploy:
  provider: pypi
  edge:
    source: ssbarnea/dpl
    branch: master# rest
@kmdouglass

This comment has been minimized.

Copy link

kmdouglass commented Nov 4, 2018

Hi,
I tested the fix described above. It seems to fix the pypi error described in #899 , but I then received a new error:

python: can't open file 'setup.py': [Errno 2] No such file or directory

https://travis-ci.org/kmdouglass/RPi-DeviceAdapters/builds/450472764#L2564

@ssbarnea

This comment has been minimized.

Copy link
Contributor Author

ssbarnea commented Nov 4, 2018

@kmdouglass This is yours to investigate, probably you are missing a setup.py in your repository and building the package fails.

I stopped using Travis module due to lack of support from Travis team which seems to ignore bugs for months, if not years and not even test and merge fixed made by others.

In the end I switched to us of twine directly as seen in https://github.com/pycontribs/jira/blob/master/.travis.yml#L85

I do not have more time to invest on this, feel free to update, create another one.... I will let Travis team close it as they do have the power to update it if they want.

@webknjaz

This comment has been minimized.

Copy link
Contributor

webknjaz commented Nov 4, 2018

@kmdouglass you navigate away from the dir containing setup.py here, in the previous provider: kmdouglass/RPi-DeviceAdapters@a6dc486#diff-354f30a63fb0907d4ad57269548329e3R27
This is why it fails (travis creates a bash script from your config and doesn't care to cd back after steps)

@cyplo

This comment has been minimized.

Copy link

cyplo commented Jan 31, 2019

Heya, any news on this one ?

@BanzaiMan

This comment has been minimized.

Copy link
Member

BanzaiMan commented Jan 31, 2019

@cyplo Have you tested it? If so, please report the results.

@cyplo

This comment has been minimized.

Copy link

cyplo commented Feb 3, 2019

@BanzaiMan - I have just tested it and it works for me - see here.

@BanzaiMan BanzaiMan merged commit e5a1a51 into travis-ci:master Feb 4, 2019

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment