Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing core-requirements from installed package #1429

merged 5 commits into from Nov 13, 2020


Copy link

Pull Request Description

Fixes #1328

Based on the discussion on #1293, I am printing out all of the installed packages instead of just the core requirements.

This is what the cli output looks like now:

> evalml info
EvalML version: 0.15.0
EvalML installation directory: /Users/freddy.boulton/miniconda3/envs/fresh-env/lib/python3.8/site-packages/evalml

python-bits: 64
OS: Darwin
OS-release: 19.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
# of CPUS: 16
Available memory: 16.2G

zict: 2.0.0
xgboost: 1.2.1
woodwork: 0.0.5
widgetsnbextension: 3.5.1
wheel: 0.35.1
webencodings: 0.5.1
wcwidth: 0.2.5
traitlets: 5.0.5
tqdm: 4.51.0
tornado: 6.1
toolz: 0.11.1
threadpoolctl: 2.1.0
texttable: 1.6.3
testpath: 0.4.4
terminado: 0.9.1
tblib: 1.7.0
sortedcontainers: 2.3.0
slicer: 0.0.3
six: 1.15.0
shap: 0.37.0
setuptools: 50.3.1.post20201107
send2trash: 1.5.0
scipy: 1.5.4
scikit-optimize: 0.8.1
scikit-learn: 0.23.2
retrying: 1.3.3
requirements-parser: 0.2.0
regex: 2020.11.11
pyzmq: 19.0.2
pyyaml: 5.3.1
pytz: 2020.4
python-dateutil: 2.8.1
pyrsistent: 0.17.3
pyparsing: 2.4.7
pygments: 2.7.2
pyflakes: 2.2.0
pycparser: 2.20
pycodestyle: 2.6.0
pyarrow: 2.0.0
pyaml: 20.4.0
ptyprocess: 0.6.0
psutil: 5.7.3
prompt-toolkit: 3.0.8
prometheus-client: 0.8.0
plotly: 4.12.0
pip: 20.2.4
pillow: 8.0.1
pickleshare: 0.7.5
pexpect: 4.8.0
partd: 1.1.0
parso: 0.7.1
pandocfilters: 1.4.3
pandas: 1.1.4
packaging: 20.4
numpy: 1.19.4
numba: 0.51.2
notebook: 6.1.5
nltk: 3.5
nlp-primitives: 1.1.0
nest-asyncio: 1.4.3
nbformat: 5.0.8
nbconvert: 6.0.7
nbclient: 0.5.1
msgpack: 1.0.0
mistune: 0.8.4
mccabe: 0.6.1
matplotlib: 3.3.3
markupsafe: 1.1.1
locket: 0.2.0
llvmlite: 0.34.0
lightgbm: 3.0.0
kiwisolver: 1.3.1
jupyterlab-pygments: 0.1.2
jupyter-core: 4.6.3
jupyter-client: 6.1.7
jsonschema: 3.2.0
joblib: 0.17.0
jinja2: 2.11.2
jedi: 0.17.2
isort: 5.6.4
ipywidgets: 7.5.1
ipython: 7.19.0
ipython-genutils: 0.2.0
ipykernel: 5.3.4
heapdict: 1.0.1
graphviz: 0.14.2
fsspec: 0.8.4
flake8: 3.8.4
featuretools: 0.21.0
evalml: 0.15.0
entrypoints: 0.3
distributed: 2.30.1
defusedxml: 0.6.0
decorator: 4.4.2
dask: 2.30.0
cycler: 0.10.0
colorama: 0.4.4
cloudpickle: 1.6.0
click: 7.1.2
cffi: 1.14.3
certifi: 2020.6.20
catboost: 0.24.2
bleach: 3.2.1
backcall: 0.2.0
attrs: 20.3.0
async-generator: 1.10
argon2-cffi: 20.1.0
appnope: 0.1.0

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

@freddyaboulton freddyaboulton self-assigned this Nov 12, 2020
@freddyaboulton freddyaboulton force-pushed the 1328-remove-core-requirements-from-package branch from 9785ab3 to 276d008 Compare November 12, 2020 18:43
Copy link

codecov bot commented Nov 12, 2020

Codecov Report

Merging #1429 (92ac1f4) into main (42319b5) will decrease coverage by 0.1%.
The diff coverage is 100.0%.

Impacted file tree graph

@@            Coverage Diff            @@
##             main    #1429     +/-   ##
- Coverage   100.0%   100.0%   -0.0%     
  Files         218      218             
  Lines       14438    14433      -5     
- Hits        14431    14426      -5     
  Misses          7        7             
Impacted Files Coverage Δ
evalml/tests/utils_tests/ 100.0% <100.0%> (ø)
evalml/utils/ 100.0% <100.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42319b5...92ac1f4. Read the comment docs.

@freddyaboulton freddyaboulton marked this pull request as ready for review November 12, 2020 19:14
Copy link

@angela97lin angela97lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Makes me wonder how useful this will be, now that we're basically printing out pip freeze and showing a lot of packages that users may have installed that aren't relevant to evalml (I think featuretools used this to help debug any sort of user-filed issues), but am a fan of removing core-requirements 👍

Copy link

@dsherry dsherry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Its funny that the output sorts in reverse alphabetical order haha

I left one comment about test pathing but that's all I got

@angela97lin yep I hear you. I think its good to have more rather than less! We can always filter it down to the deps evalml relies on.

if x in installed_packages:
packages_to_log.append((x, installed_packages[x]))
for package, version in packages_to_log:
for package, version in installed_packages.items():
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah that's fine!

def test_get_core_requirements():
assert len(get_core_requirements()) == 15
def get_core_requirements(current_dir):
reqs_path = os.path.join(current_dir, '../../../core-requirements.txt')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two notes:

  1. This path won't work on windows, right? Interesting that the windows test still passed... Either way, safest to use pathlib: pathlib.Path('..', '..', '..', 'core-requirements.txt')
  2. I thought there was a way to have pytest automagically handle this for you somehow? Although I just spent 2min googling and couldn't find it 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dsherry I spent some time looking at this on the circle-ci windows vm and I don't have a satisfying explanation as to why this works...but it does lol.

Running os.path.join with these inputs returns a string with mixed windows and unix separators - so it should cause a problem:


But then open just handles it like a champ - no idea how it works. It must be doing something to resolve the separators under the hood.


Needless to say, I'll just change this to pathlib lol.

@freddyaboulton freddyaboulton force-pushed the 1328-remove-core-requirements-from-package branch from 1f3f623 to 92ac1f4 Compare November 13, 2020 16:07
@freddyaboulton freddyaboulton merged commit dc6acd5 into main Nov 13, 2020
@freddyaboulton freddyaboulton deleted the 1328-remove-core-requirements-from-package branch November 13, 2020 16:25
christopherbunn pushed a commit that referenced this pull request Nov 19, 2020
* Refactoring cli to print all dependencies.

* Adding PR 1429 to release notes.

* Fixing typo in PR 1429 that was causing the release_notes_updated check to fail.

* Fixing import order in

* Using pathlib for get_core_requirements in
@dsherry dsherry mentioned this pull request Nov 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

Successfully merging this pull request may close these issues.

Remove core-requirements.txt from installed package
3 participants