-
Notifications
You must be signed in to change notification settings - Fork 231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Display One-Line Progress Bar #1080
Comments
Hi @OscarIntellico - thanks for your feedback! I don't think this will be possible with the current default settings (ploomber uses papermill for executing notebooks by default); however, it might be possible with ploomber-engine (our papermill replacement). It's possible to use ploomber with ploomber-engine, so if we can get this working, you could switch from papermill to ploomber-engine in your ploomber pipeline. So I think we have two items here:
Please check this out @mehtamohit013 |
hey @OscarIntellico, we added support for notebook progress bars to ploomber-engine, can you give it a try and let us know if this is what you're expecting? context: currently, ploomber executes notebooks with papermill, but you can also switch to use ploomber-engine, which is our drop-in replacement for papermill. If this solves your issue, we can integrate this into ploomber. to test it:
then execute a notebook that displays an inline progress bar:
|
Hi @edublancas , thank you for your very fast implementation.
The output is what i expected. PlatformRight now i'm using VSCode on a windows machine. VSCode is remotely connected to an ubuntu machine on which the code is run. |
excellent, thanks for your feedback. sounds like this works as expected (besides minor graphical glitches) I just made a new release, you can integrate it in ploomber like this: # to get the latest version
pip install ploomber-engine --upgrade Then in your pipeline.yaml: - source: fit.py
name: fit
product:
nb: output/nb.html
papermill_params:
# this will tell ploomber to use ploomber-engine
engine_name: embedded I believe this should work but let me know if you encounter any issues! we're working on a better integration of ploomber-engine into ploomber, so keep an eye on this: #1082 |
Hi.
I'm using ploomber in a Machine Learning Pipeline, in particular i'm using Neural Network built in pytorch. The training, validation and test loop print a progress bar with tqdm, since it can give a good estimate of the training time and can also add some granular informations such as current validation loss and so on.
In order to see the outputs, i added to my training task the papermill_params tag, with log_output: True.
While this allows me to see what i'm printing in my task, the progress bar is not interactive, but every time a new update comes, a new line is printed, as you can see in the image.
My desired behaviour would be to just have one single updating line, I don't know if right now is possible to implement this, since my understanding is that ploomber is redirecting the cell output on stdout. Also I think possible problems may arise with parallel task that both use the progress bar, but it would be nice for at least single-process pipelines to not loose the interactivity of tqdm.
The text was updated successfully, but these errors were encountered: