Making a progress bar work in IPython Notebook #1527

Closed
fonnesbeck opened this Issue Mar 26, 2012 · 7 comments

Comments

Projects
None yet
4 participants
@fonnesbeck

I have modified a simple progressbar that I was hoping would work with IPy notebooks. It works fine in a standard console:

working progress bar

however, in a notebook it does not:

broken progress bar

Here are the guts of the progressbar code:

import sys, time
try:
    import IPython
    clear_output = IPython.core.display.clear_output
except:
    IPython = None

class ProgressBar:
    def __init__(self, iterations):
        self.iterations = iterations
        self.prog_bar = '[]'
        self.fill_char = '*'
        self.width = 40
        self.__update_amount(0)
        if IPython:
            self.animate = self.animate_ipython
        else:
            self.animate = self.animate_noipython

    def animate_ipython(self, iter):
        if sys.platform.lower().startswith('win'):
            print self, '\r',
        else:

            print self, chr(27),
        self.update_iteration(iter + 1)
        # time.sleep(0.5)
        clear_output()

    def update_iteration(self, elapsed_iter):
        self.__update_amount((elapsed_iter / float(self.iterations)) * 100.0)
        self.prog_bar += '  %d of %s complete' % (elapsed_iter, self.iterations)

    def __update_amount(self, new_amount):
        percent_done = int(round((new_amount / 100.0) * 100.0))
        all_full = self.width - 2
        num_hashes = int(round((percent_done / 100.0) * all_full))
        self.prog_bar = '[' + self.fill_char * num_hashes + ' ' * (all_full - num_hashes) + ']'
        pct_place = (len(self.prog_bar) / 2) - len(str(percent_done))
        pct_string = '%d%%' % percent_done
        self.prog_bar = self.prog_bar[0:pct_place] + \
            (pct_string + self.prog_bar[pct_place + len(pct_string):])

    def __str__(self):
        return str(self.prog_bar)

So, at every iteration of my code (an MCMC algorithm), the sampler calls animate(tier) to update the progress bar with the current iteration. I am using IPython.core.display.clear_output to clear the cell output at each iteration, but for some reason ti does not work here. Any help would be appreciated. I would love to use PyMC in IPython Notebooks!

@minrk

This comment has been minimized.

Show comment Hide comment
@minrk

minrk Mar 26, 2012

Owner

The main issues, as far as I can tell:

  • you should probably clear output before drawing the latest edition, otherwise you are drawing output, then immediately clearing it. \r is non-destructive, but clear_output blanks the output immediately.
  • make sure you call sys.stdout.flush(), to push events to the frontend, otherwise they will all fire together at the end of execution.
  • import IPython does not imply import IPython.core.display, so your code always gives IPython=None.
  • I don't know why you would not use \r on non-Windows, because it seems to work perfectly well.

This seems to work just fine in the notebook, qtconsole, and terminal IPython:

import sys, time
try:
    from IPython.core.display import clear_output
    have_ipython = True
except ImportError:
    have_ipython = False

class ProgressBar:
    def __init__(self, iterations):
        self.iterations = iterations
        self.prog_bar = '[]'
        self.fill_char = '*'
        self.width = 40
        self.__update_amount(0)
        if have_ipython:
            self.animate = self.animate_ipython
        else:
            self.animate = self.animate_noipython

    def animate_ipython(self, iter):
        try:
            clear_output()
        except Exception:
            # terminal IPython has no clear_output
            pass
        print '\r', self,
        sys.stdout.flush()
        self.update_iteration(iter + 1)

    ...
Owner

minrk commented Mar 26, 2012

The main issues, as far as I can tell:

  • you should probably clear output before drawing the latest edition, otherwise you are drawing output, then immediately clearing it. \r is non-destructive, but clear_output blanks the output immediately.
  • make sure you call sys.stdout.flush(), to push events to the frontend, otherwise they will all fire together at the end of execution.
  • import IPython does not imply import IPython.core.display, so your code always gives IPython=None.
  • I don't know why you would not use \r on non-Windows, because it seems to work perfectly well.

This seems to work just fine in the notebook, qtconsole, and terminal IPython:

import sys, time
try:
    from IPython.core.display import clear_output
    have_ipython = True
except ImportError:
    have_ipython = False

class ProgressBar:
    def __init__(self, iterations):
        self.iterations = iterations
        self.prog_bar = '[]'
        self.fill_char = '*'
        self.width = 40
        self.__update_amount(0)
        if have_ipython:
            self.animate = self.animate_ipython
        else:
            self.animate = self.animate_noipython

    def animate_ipython(self, iter):
        try:
            clear_output()
        except Exception:
            # terminal IPython has no clear_output
            pass
        print '\r', self,
        sys.stdout.flush()
        self.update_iteration(iter + 1)

    ...
@fonnesbeck

This comment has been minimized.

Show comment Hide comment
@fonnesbeck

fonnesbeck Mar 26, 2012

Thanks, that is an improvement, but it still does not seem to work. For some reason, the problem arises when animate() is called externally. For example, even if I ignore my PyMC code altogether and call the ProgressBar's animate method from a simple function, it does not work -- it displays nothing until the last iteration, when it dumps everything to the screen:

bad progress bar

However, if I add a run() method to the ProgressBar class itself:

def run(self):
    for i in range(1000):
        self.animate(i)

it works ... any reason why this should be?

Thanks, that is an improvement, but it still does not seem to work. For some reason, the problem arises when animate() is called externally. For example, even if I ignore my PyMC code altogether and call the ProgressBar's animate method from a simple function, it does not work -- it displays nothing until the last iteration, when it dumps everything to the screen:

bad progress bar

However, if I add a run() method to the ProgressBar class itself:

def run(self):
    for i in range(1000):
        self.animate(i)

it works ... any reason why this should be?

@minrk

This comment has been minimized.

Show comment Hide comment
@minrk

minrk Mar 26, 2012

Owner

Hm, that's working just fine for me. I don't know why the same code wouldn't work for you. Can you confirm that clear_output is actually being called?

Owner

minrk commented Mar 26, 2012

Hm, that's working just fine for me. I don't know why the same code wouldn't work for you. Can you confirm that clear_output is actually being called?

@minrk

This comment has been minimized.

Show comment Hide comment
@minrk

minrk Mar 27, 2012

Owner

Here's the standalone progress bar as a working notebook: https://gist.github.com/2211026

Owner

minrk commented Mar 27, 2012

Here's the standalone progress bar as a working notebook: https://gist.github.com/2211026

@fonnesbeck

This comment has been minimized.

Show comment Hide comment
@fonnesbeck

fonnesbeck Mar 27, 2012

Yeah, this appears to work. I must not have properly restarted my kernel or something. Thanks for the help!

Yeah, this appears to work. I must not have properly restarted my kernel or something. Thanks for the help!

@fonnesbeck fonnesbeck closed this Mar 27, 2012

@jsalvatier

This comment has been minimized.

Show comment Hide comment
@jsalvatier

jsalvatier Apr 14, 2013

I'm trying to make a progress bar that has a nice javascript animation, but I only want to display that if it's going to be displayed properly. I also still want to do nice text animation in the regular console. Thus I'd like to know if my program is running from inside IPython Notebook. I gather that I'm supposed to let IPython notebook choose what to display by using http://nbviewer.ipython.org/url/github.com/ipython/ipython/raw/master/examples/notebooks/Part%205%20-%20Rich%20Display%20System.ipynb . Is there a clean way to set an HTML or svg element so that it animates?

I'm trying to make a progress bar that has a nice javascript animation, but I only want to display that if it's going to be displayed properly. I also still want to do nice text animation in the regular console. Thus I'd like to know if my program is running from inside IPython Notebook. I gather that I'm supposed to let IPython notebook choose what to display by using http://nbviewer.ipython.org/url/github.com/ipython/ipython/raw/master/examples/notebooks/Part%205%20-%20Rich%20Display%20System.ipynb . Is there a clean way to set an HTML or svg element so that it animates?

@ameerabsalam

This comment has been minimized.

Show comment Hide comment
@ameerabsalam

ameerabsalam Mar 20, 2017

Can't think of a way to make 1 overall - progress bar for ALL cells. ie the time it takes to run the whole notebook. Any thoughts ?

Can't think of a way to make 1 overall - progress bar for ALL cells. ie the time it takes to run the whole notebook. Any thoughts ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment