Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run dockerized pip as invoking user #15

Merged
merged 1 commit into from
Apr 6, 2017
Merged

Conversation

suxor42
Copy link
Contributor

@suxor42 suxor42 commented Mar 31, 2017

By default docker executes commands in an container as root. This causes the downloaded dependencies to be owned by root. This messes up cleanup scripts.

Using the correct object which holds the dockerizePip option.
this.serverless.service.custom.dockerizePip

Initialize the this.serverless.service.package.exclude and this.serverless.service.package.include correctly if they are not set.

@dschep dschep self-requested a review March 31, 2017 12:37
Copy link
Contributor

@dschep dschep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Setting UID/GID: yes! great idea! Did you test this? When I tested it, I get permission errors when pip is trying to write to /.cache. And simply adding --disable-cache-dir doesn't help either bc pip doesn't have permissions to upgrade itself.
  • Getting the option from the "right" place. It already is. See https://github.com/UnitedIncome/serverless-python-requirements/pull/15/files#diff-168726dbe96b3ce427e7fedce31bb0bcL119 which sets this.custom to this.serverless.service.custom.pythonRequirements. The dockerizePip option should be in the pythonRequirements object per the docs (this did change in v2, thus the increase in major version number since it's a breaking change)
  • Setting exclude and include correctly. I'm not sure what was wrong about the existing lodash based code (in fact, the code you added wouldn't work if it wasn't working):
$ node
> const _ = require('lodash')
undefined
> this.serverless = {service: {}} // add path that in the plugin is always there
{ service: {} }
> _.has(this.serverless.service, ['package', 'exclude'])
false
> _.set(this.serverless.service, ['package', 'exclude'], []);
{ package: { exclude: [] } }
> this.serverless.service.package.exclude
[]
> this.serverless = {service: {}}
{ service: {} }
> this.serverless.service.package.exclude = []
TypeError: Cannot set property 'exclude' of undefined

@dschep
Copy link
Contributor

dschep commented Mar 31, 2017

For context, I'm updating pip because I need a newer version and I didn't feel like making my own docker image just for this, mostly because I didn't know what was necessary for keeping it up to date despite the Dockerfile I'd write never changing, just the image referenced in FROM would be updating. But it seems docker hub supports triggering a build based off of changes in multiple repos, so I think I could do that route.

@suxor42 suxor42 force-pushed the master branch 5 times, most recently from 92891e4 to b9c6524 Compare April 6, 2017 09:16
By default docker executes commands in an container as root. This causes the downloaded dependencies to be owned by root. This messes up cleanup scripts.

Using the correct object which holds the dockerizePip option.
this.serverless.service.custom.dockerizePip

Initialize the this.serverless.service.package.exclude and this.serverless.service.package.include correctly if they are not set.
@suxor42
Copy link
Contributor Author

suxor42 commented Apr 6, 2017

I removed everything related to the option and the include/exclude parts.

I tested it on our Jenkins slaves:

  • Ubuntu 16.04
  • npm 3.5.2
  • nodejs v4.2.6
  • serverless 1.10.2
  • docker 1.13

My local machine:

  • Antergos Linux
  • npm 4.4.4
  • nodejs v7.7.3
  • serverless 1.10.2

The pip version in the container was already up to date when I tested it.
The message about the cache is "only" a warning. Caching in docker container does not work anyway because a new container is created for every run. We could mount a local directory into the container. I couldn't find the .cache directory in the container.

The directory '/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

If I mount a directory to /.cache (like -v /dev/null:/.cache:rw) the warning is gone.

@dschep dschep merged commit 2035453 into serverless:master Apr 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants