Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing requirements on windows/linux takes a huge time due to nvidia stuff #285

Closed
julesvanrie opened this issue Apr 5, 2024 · 3 comments · Fixed by #293
Closed

Comments

@julesvanrie
Copy link
Contributor

Since we included transformers, the python requirements install takes even longer than before.

Reasons (double check): transformers needs pytorch, which installs plenty of nvidia stuff too.

Couple of options:

  • Remove transformers stuff: we only use it one day, students could run the notebooks on Colab (like in the previous days of DL).
  • For pytorch to install a cpu-only version. Now we install nvidia cuda stuff, but most students don't have any benefit from it. (Pretty rare to have a (powerful) nvidia gpu in laptops.)
@julesvanrie
Copy link
Contributor Author

On windows / linux 9.4 Gb of which:

  • nvidia 2.8G
  • tangled_up_in_unicode 1.8G
  • torch 1.4G
  • tensorflow 1.4G

On macos 4.3 Gb of which:

  • tangled_up_in_unicode 1.7G
  • torch 0.3G
  • tensorflow 0.9G

Maybe worth trying a cpu only of tensorflow too?

@julesvanrie
Copy link
Contributor Author

tangled-up-in-unicode :

Dependency of pandas-profiling and visions, which in turn is a dependency of pandas-profiling.

So we can drop this if we drop pandas-profiling.

➜ pip install "pipdeptree<2.17.0"
➜ pipdeptree -r -p tangled-up-in-unicode
tangled-up-in-unicode==0.2.0
├── pandas-profiling==3.3.0 [requires: tangled-up-in-unicode==0.2.0]
└── visions==0.7.5 [requires: tangled-up-in-unicode>=0.0.4]
    └── pandas-profiling==3.3.0 [requires: visions==0.7.5]
➜  pipdeptree -r -p visions
visions==0.7.5
└── pandas-profiling==3.3.0 [requires: visions==0.7.5]

@julesvanrie
Copy link
Contributor Author

julesvanrie commented Apr 6, 2024

ydata-profiling does not need tangled-up-in-unicode directly.
visions doesn't need it anymore either in the latest version. But that requires pandas>=2.0.0 and we're still on 1.4.4).

@julesvanrie julesvanrie changed the title Installing requirements takes a huge time due to nvidia stuff Installing requirements on windows/linux takes a huge time due to nvidia stuff Apr 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant