use tqdm.autonotebook

[hugobuddel]

tqdm has special support for Jupyter notebooks in tqdm.notebook. It can also automatically select the right tqdm (the terminal or notebook one) with

from tqdm.autonotebook import tqdm

It looks much better:

For completeness, the same notebook in the terminal:

And the code:

from tqdm import tqdm as tqdm_normal
from tqdm.notebook import tqdm as tqdm_notebook
from time import sleep

maxa = 2
maxb = 3
zzzz = 0.1

print()
print("normal without leave")
for aa in tqdm_normal(range(maxa), desc=" AA1", leave=False):
    for bb in tqdm_normal(range(maxb), desc=" BB1", leave=False):
        sleep(zzzz)

print()
print("normal with leave")
for aa in tqdm_normal(range(maxa), desc=" AA2", leave=True):
    for bb in tqdm_normal(range(maxb), desc=" BB2", leave=True):
        sleep(zzzz)

print()
print("notebook without leave")
for aa in tqdm_notebook(range(maxa), desc=" AA3", leave=False):
    for bb in tqdm_notebook(range(maxb), desc=" BB3", leave=False):
        sleep(zzzz)

print()
print("notebook with leave")
for aa in tqdm_notebook(range(maxa), desc=" AA4", leave=True):
    for bb in tqdm_notebook(range(maxb), desc=" BB4", leave=True):
        sleep(zzzz)

It was not really clear when we used leave=False, or why we specified a position. I think removing those arguments looks fine, but I'm also fine with using leave=False everywhere. The position argument seems to be mostly useful in a multi-threaded environment, which we don't have.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8412602) 75.57% compared to head (86ac017) 75.57%.
Report is 2 commits behind head on dev_master.

Additional details and impacted files

@@             Coverage Diff             @@
##           dev_master     #352   +/-   ##
===========================================
  Coverage       75.57%   75.57%           
===========================================
  Files              59       59           
  Lines            7869     7869           
===========================================
  Hits             5947     5947           
  Misses           1922     1922           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[teutoburg]

I like that the progress bar in the notebooks now looks much nicer. However, I ran a very quick test using from tqdm.autonotebook import tqdm in an IPython console (inside my IDE, which is what I use mostly in general), and the progress bar doesn't print at all, i.e. only the "surroundings", but no bar is filled. Same with the example you provided.

Also on the position and leave arguments: I specifically put them in to deal with nested progress bars. I worked nicely like this on my end, and didn't without IIRC. Mostly an issue when the inner bar takes a while (as in METIS spec modes).

[teutoburg]

Also on the position and leave arguments: I specifically put them in to deal with nested progress bars. I worked nicely like this on my end, and didn't without IIRC. Mostly an issue when the inner bar takes a while (as in METIS spec modes).

Ok, according to tqdm's documentation, the notebook variant deals with nested bars differently, so those arguments might not be needed in that case. But still, it should work from the console as well...

[teutoburg]

FWIW, I'm also fine with switching the progress bars off by default and leaving a config option to switch them on. I mostly added them to debug some spectroscopy simulations, which take forever depending on spectra bin width etc., and I like to see if there's actually something happening within a lifetime 😅

[hugobuddel]

However, I ran a very quick test using from tqdm.autonotebook import tqdm in an IPython console (inside my IDE, which is what I use mostly in general), and the progress bar doesn't print at all, i.e. only the "surroundings", but no bar is filled. Same with the example you provided.

So with this route, we have to choose whether the progress bars look nice in an actual web-based notebook, or look nice in an IPython console. I would say they should look good in an actual notebook.

FWIW, I'm also fine with switching the progress bars off by default and leaving a config option to switch them on.

Perhaps we should do that? Have four options for the configuration of the progress bars None (default), tqdm voor the console version, notebook for the notebook version, or autonotebook for the automatic one.

I'll make it configurable, we can decide on the default with that in place.

Also on the position and leave arguments: I specifically put them in to deal with nested progress bars. I worked nicely like this on my end, and didn't without IIRC. Mostly an issue when the inner bar takes a while (as in METIS spec modes).

I assumed as much, and that would be fine, but I couldn't quickly find an example where it really looked better. Do you have an example? Maybe I should try one of the METIS LSS modes.

[hugobuddel]

What IDE are you using?

autonotebook works fine in PyCharm:

it also works fine in just an ipython terminal:

[teutoburg]

What IDE are you using?

Spyder, and I'm now starting to suspect it might be an issue with that. It works much better when I run it from an IPython console started from the terminal, or directly running python in the terminal. I'll include some screenshots to illustrate a bit later. But I see that Spyder isn't very common, so I'd be fine with having non-ideal support for it (at least by default), if it means we have better support for notebooks (and "direct IPython consoles").

[teutoburg]

One more thing we might want to investigate before committing to e.g. autonotebook is the connection to logging. I have solved that before, and did put some comments in some places already, but I'd like to try how this plays out with the different variants...