Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't install packages that could mess packaging up #397

Merged
merged 12 commits into from May 31, 2017
12 changes: 12 additions & 0 deletions CHANGELOG.md
@@ -1,5 +1,17 @@
# Python Buildpack Changelog

# 106

Don't install packages that could mess up packaging.

- The Python buildpack will automatically remove `six`, `pyparsing`, `appdirs`,
`setuptools`, and `distribute` from a `requirements.txt` file now, as these
packages are provided by the Python buildpack.

# 105

Improvements to output messaging.

# 104

General improvements.
Expand Down
3 changes: 3 additions & 0 deletions bin/compile
Expand Up @@ -153,6 +153,9 @@ if [ ! -f requirements.txt ] && [ ! -f Pipfile ]; then
echo "-e ." > requirements.txt
fi

# Cleanup requirements.txt
source $BIN_DIR/steps/setuptools

# Fix egg-links.
source $BIN_DIR/steps/eggpath-fix

Expand Down
6 changes: 6 additions & 0 deletions bin/steps/setuptools
@@ -0,0 +1,6 @@
#!/usr/bin/env bash

# Syntax sugar.
source $BIN_DIR/utils

pip-clean requirements.txt
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing EOL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

33 changes: 33 additions & 0 deletions vendor/pip-pop/pip-clean
@@ -0,0 +1,33 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""Usage:
pip-clean <req-file>

Options:
-h --help Show this screen.
"""
from docopt import docopt

BAD_PACKAGES = ['appdirs', 'packaging', 'pyparsing', 'six', 'setuptools', 'distribute']


def good_package(line):
package_name = line.split('=')[0].split('<')[0].split('>')[0].split(' ')[0].split('#')[0].split('\n')[0]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just do .split() with no parameters instead of .split(' ').split('\n') because it deals with all whitespaces. Given the for line in f, \n is not really an issue here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\n is included in each line

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Buggy. c.f. #400

return package_name not in BAD_PACKAGES

def main():
args = docopt(__doc__, version='pip-grep')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version='pip-grep'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

req_file = args['<req-file>']

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be streamlined?

def good_package(line):
	package_name = line.split('=')[0].split('<')[0].split('>')[0]
	return package_name not in BAD_PACKAGES

lines = [line for line in f if good_package(line)]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to do as little parsing as possible, for fear of breaking anything with crazy requirements files.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood... see other package names that begin with but do not end with six for instance https://pypi.python.org/pypi?%3Aaction=search&term=Six&submit=search

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was planning to cross that bridge when I came to it, which I honestly doubt would happen any time soon. I'll see what I can do though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cclauss i updated the code; integrated what you suggested and made a few improvements

with open(req_file, 'r') as f:
# Iterate over every line in the requirements file.
lines = [line for line in f if good_package(line)]

# Write the requirements file to disk.
with open(req_file, 'w') as f:
f.write(''.join(lines))


if __name__ == '__main__':
main()