-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No progress bar when training on Google Colab #1112
Comments
Hi! thanks for your contribution!, great first issue! |
we test this very rigorously... look at the docs and the MNIST example then check your code. |
I'm also having issues with the progress bar. Instead of a progress bar, I got
This happened to me only when using tpus and num_tpu_cores=8 (1 tpu core works just as expected). Interestingly, it is just the epoch progress bar, the validation progress bar shows as intended.
|
@williamFalcon I literally took the "MNIST on TPU" from the docs page and ran in on Google colab, and it showed no progress bar or anything. |
I am having the same issue on Sagemaker too. No Progress bar, just the |
@jwallat and @lezwon I'm suspicious the issue happens upstream with tqdm itself. The issue seems to be related with the cancelling (or failing) of a running operation. They suggest the following workaround: We could implement that into Lightning but other issue such as this one make me pessimistic about it working. Any ideas? |
@luiscape I have another notebook (without PL) wherein I use |
check following fix, it should help #1093 |
@Borda I tried it on NextJournal, it still shows |
i don’t really know what this is either. this seems to be a colab thing. |
i see this issue when:
fixes when i reset environment. this is not a lightning issue though... might just be tqdm or colab |
@williamFalcon I have tried this on Sagemaker and Nextjournal. tqdm works fine if I run it myself. When using Lightning though it shows the HBox text. I have tried restarting kernel, upgrading tqdm etc. does not seem to work. |
Even though I'm using |
mind upgrade on 0.9 and try again... :] |
Hi, I am facing the same problem, no progress bar on Jupyter Lab.
I am using pytorch_lightning 1.0.2; |
@czrcbl are you using sagemaker? or running jupyter lab locally? |
I am running the Jupyter Lab in an Amazon ECS instance and connecting to it through ssh port forwarding. |
I think this is because the |
🐛 Bug
To Reproduce
Steps to reproduce the behavior:
Environment
Google Colab, with current github version of pytorch-lightning installed.
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1
OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.12.0
Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: Tesla K80
Nvidia driver version: 418.67
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
Versions of relevant libraries:
[pip3] numpy==1.17.5
[pip3] pytorch-lightning==0.7.1
[pip3] torch==1.4.0
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.3.1
[pip3] torchvision==0.5.0
[conda] Could not collect
The text was updated successfully, but these errors were encountered: