Choose which type of progress bar you want in a notebook

[astrojarred]

Sometimes one doesn't always have control of the environment of the Jupyter instance where one's working (think Jupyterhub) and can't install the necessary extensions for progress bars. In this case it might be nice to have the option to manually request the simple progress bar so that it doesn't just display a widget error.

Is there already a way to do so that I've missed? Otherwise, if it's something you'd consider adding, I'd also be happy to try and draft a PR.

[till-m]

Hi @astrojarred,

thanks for your interest in this package.

I agree with the sentiment behind your idea. On the other hand, I am somewhat sceptical of maintaining a custom implementation of progressbars in general, especially when there are so many ready-to-use tools available (e.g. tqdm, progressbar2, etc.). The custom progressbars seems to have caused issues already, see e.g. #184. In that sense I would ideally like to replace the existing progressbars with one of the above.

That being said, considering that neither @nalepae nor me currently have the bandwidth to do anything but provide life support for this package, I don't see this transition being completed anytime soon. Therefore, if you were to draft a PR I would happily review and merge it.

[nalepae]

especially when there are so many ready-to-use tools available (e.g. tqdm, progressbar2, etc.)

First implementations of pandarallel used tqdm.
The main issue was tqdm slowed (a lot) computation when function to apply is very simple.
(Overhead added was relatively important in this case)

Progress bars was the most complicated to implement feature of pandarallel, since one of its main requirement is to add as low overhead as possible.

I'm not against replacing custom progress bars with more standards one, if benchmarks show there is no performance drop.

[astrojarred]

Hi both, thanks for the insightful replies. That's too bad about tqdm slowing things down.

I noticed that get_progress_bars() detects whether or not one is in an ipython environment or not. Perhaps a simple short-term solution could be to let the user override this and force ProgressBarsConsole from a notebook.

It might also be interesting to see if newer versions of the standard progress bar packages have any better performance. Even if not, one could let the user decide if they are okay sacrificing some speed for the convenience of fully-featured progress bars.

[mmoraschini]

I would also really love to see this one. I work on a notebook where widgets don't work therefore I cannot see the progress bar. thread_map from from tqdm.contrib.concurrent has an option to choose the progress bar to use, something like:

thread_map(functools.partial(fcn), inputs, max_workers=MAX_WORKERS, tqdm_class=tqdm)

This way I can override the default behaviour of showing the bar from tqdm.notebook and show the one from the base tqdm.

Maybe an easy solution could be to set it in pandarallel.initialize()?

Thanks a lot for what you built!

[nalepae]

Pandaral·lel is looking for a maintainer!
If you are interested, please open an GitHub issue.