Custom bar symbols formatting + bar_format callback and dict return

[lrq3000]

Implements also #181 (add callback support for bar_format) + custom bar symbols formatting as a submodule.

Not the most urgent PR, but we missed some more customization (pretty sure someone will ask someday!). So here it is: the ability to set custom bar symbols, just like our fellow progressbar2:

import time
from tqdm import tcrange, tqdm_custom

bar_format = {'template': '{l_bar}{bar}{r_bar}',
                     'symbols': {'ascii': list('ABCDEO'),
                                      'loop': False},
                     }

bar_format_loop = {
                     'symbols': {'ascii': list('/-\|'),
                                      'loop': True},
                     }

bar_format_ind_loop = {
                    'symbols_indeterminate': {'ascii': list('/-\|'),
                                      'loop': True},
                      }

bar_format_ind = {
                    'symbols_indeterminate': {'ascii': ['----{,_,">', '----{,_,*>'],
                                      'loop': False},
                      }

for i in tcrange(100, ascii=True, bar_format=bar_format):
    time.sleep(0.1)

for i in tcrange(100, ascii=True, bar_format=bar_format_loop):
    time.sleep(0.1)

with tqdm_custom(ascii=True, bar_format=bar_format_ind_loop) as tt1:
    for i in xrange(100):
        time.sleep(0.1)
        tt1.update()

with tqdm_custom(ascii=True, bar_format=bar_format_ind) as tt2:
    for i in xrange(100):
        time.sleep(0.1)
        tt2.update()

Respective outputs:

  9%|OOOE                                      | 9/100 [00:00<00:09,  9.95it/s]

 12%|/| 12/100 [00:01<00:08, 10.00it/s]

100it|/| 100/? [00:10,  9.91it/s]

100it|                          ----{,_,">           | 100/? [00:10, 10.00it/s]

For multiline (work in progress, needs bugfixes):

import time
from tqdm import tcmrange, tqdm_custommulti
from copy import deepcopy

bar_format_ind_multi = {
                    'symbols_indeterminate': {'ascii': [
"""
     _
\. _(9>
 \==_)
  -'=
""",
"""
     _
\. _(6>
 \==_)
  -'=
"""
],
                                      'loop': False,
                                      'multiline': True},
                      }


bar_format_ind_loop_multi = {
                    'symbols_indeterminate': {'ascii': [
"""
     _
\. _(9>
 \==_)
  -'=
""",
"""
     _
\. _(6>
 \==_)
  -'=
"""
],
                                      'loop': True,
                                      'multiline': True},
                      }

bar_format_loop_multi = {
                    'symbols': {'ascii': [
"""
     _
\. _(9>
 \==_)
  -'=
""",
"""
     _
\. _(6>
 \==_)
  -'=
"""
],
                                      'loop': True,
                                      'multiline': True},
                      }

bar_format_multi = {
                    'symbols': {'ascii': [
"""
. . . 
 . .
. . .
""",
"""
* * * 
 * *
* * *
""",
"""
= = = 
 = =
= = =
""",
"""
0 0 0 
 0 0
0 0 0
""",
"""
O O O 
 O O
O O O
""",
                                    ],
                                      'loop': False,
                                      'multiline': True},
                      }

bar_format_multi_anim = {
                    'symbols': {'ascii': [
["""
. . . 
 . .
. . .
""",
"""
 . .
. . . 
 . .
"""],
["""
* * * 
 * *
* * *
""",
"""
 * *
* * * 
 * *
"""],
["""
= = = 
 = =
= = =
""",
"""
 = =
= = = 
 = =
"""],
["""
0 0 0 
 0 0
0 0 0
""",
"""
 0 0
0 0 0 
 0 0
"""],
["""
O O O 
 O O
O O O
""",
"""
 O O
O O O 
 O O
"""],
                                    ],
                                      'loop': False,
                                      'multiline': True,
                                      'random': False},
                      }

bar_format_multi_anim_rand = deepcopy(bar_format_multi_anim)
bar_format_multi_anim_rand['symbols']['random'] = True

sleep_step = 0.05
anim_interval = 0.1

for i in tcmrange(100, mininterval=anim_interval, ascii=True, bar_format=bar_format_loop_multi):
    time.sleep(sleep_step)

with tqdm_custommulti(mininterval=anim_interval, ascii=True, bar_format=bar_format_ind_loop_multi) as ttm1:
    for i in xrange(100):
        time.sleep(sleep_step)
        ttm1.update()

with tqdm_custommulti(mininterval=anim_interval, ascii=True, bar_format=bar_format_ind_multi) as ttm1:
    for i in xrange(300):
        time.sleep(sleep_step)
        ttm1.update()

for i in tcmrange(100, mininterval=anim_interval, ascii=True, bar_format=bar_format_multi):
    time.sleep(sleep_step)

for i in tcmrange(100, mininterval=anim_interval, ascii=True, bar_format=bar_format_multi_anim):
    time.sleep(sleep_step)

for i in tcmrange(100, mininterval=anim_interval, ascii=True, bar_format=bar_format_multi_anim_rand):
    time.sleep(sleep_step)

User can specify both ascii and unicode versions of the bar, if one isn't provided, then it will fallback to the default tqdm bar.

TODO:

• Add indeterminate progress customization
• Optimize reversing symbol by precomputing in __init__ and then loading it at the same time as c_symbol (eg, c_symbol_reverse = self.bar_format[key].get('ascii_reverse', None) ? Would allow manual user reversing BTW).
• Rebase on Add callable support for bar_format argument #181
• Remove copy/pasted code thank's to Add callable support for bar_format argument #181
• Multiline symbols support (fish/nyanbar-like, see below)
  • Multiline indeterminate bar
  • Multiline loop (both indeterminate and determinate)
  • Multiline animated progress bar
  • Fix leave=True (there are more than the required line returns sometimes, I guess it's Windows terminal's fault, it seems it doubles the number of line returns when reaching the terminal's end)
  • Fix multiline progress bar overflowing ncols when 100%
  • Fix multiline progress bar frac not reaching filler symbol when 100%
  • Fix multiple parallel/nested multiline bars using self.last_print_height (works with leave=False, need to fix leave=True bug)
• Fix Flake8
• Add unit tests
• Update docstrings and readme

About multiline symbols:
We should try to implement multiline animated ascii-art bars like nyanbar or fish. This shouldn't be too hard to implement, let me explain:

Basically, we have currently 4 different bar displays:

• progress bar : standard progress bar with numbers 1 to 9 and then # when enough progress, and it gets filled up until it reaches 100% completion.
• progress loop : animated symbol that loops in-place, takes minimal space on screen.
• indeterminate bar : infinite progress bar with a symbol that goes from left to right, then right to left (because we don't know when it ends).
• indeterminate loop : like progress loop.

Then for all these cases, you can add multiline support. We can then assimilate fish and nyanbar to the following cases:

• fish single line = indeterminate bar
• fish multi line = indeterminate bar multiline
• nyanbar = progress bar multiline

So if we can support multiline, we can essentially support what these two libraries do (somewhat, because for example nyanbar would only be simulated since the real one generates random colored blocks, our version would have fixed sprites but if you make enough of them, then you can give the illusion it's random).

Only issue is how to handle multiple concurrent tqdm bars, but this can be managed I think (we can memorize the number of lines of the last printed symbol and get back to original position, just like we currently do with automated nesting, then when we move_to() we would compute the correct position to print using sum([t.s_height for t in tqdm._instances if t.pos < self.pos])).

---

The rest of the text here is for the old version of this PR.

For reference, the old way with bar string templating was like this (but it was scraped to give more flexibility, but maybe it will be reintroduced in the future if people like it):

import time
from tqdm import tcrange

for i in tcrange(100, ascii=True, bar_format='{l_bar}{bar}{bar_symbols_ascii},A,B,C,D,E,O{/bar_symbols_ascii}{r_bar}'):
    time.sleep(0.1)

for i in tcrange(100, ascii=True, bar_format='{l_bar}{bar}{bar_symbols_loop_ascii},/,-,\,|{/bar_symbols_loop_ascii}{r_bar}'):
    time.sleep(0.1)

There are four different new tags for bar_format:

• {bar_symbols} and {bar_symbols_ascii} set custom progress bar symbols (unicode or ascii respectively)
• {bar_symbols_loop} and {bar_symbols_loop_ascii} set animated looping symbols (unicode or ascii respectively)

As you can see, I added both unicode and ascii tags, so that the user can provide either one or both. If the user didn't provide custom symbols for the target environment (eg, user provided {bar_symbols} but the environment is ascii), then the bar will fallback to the default bar. Why? This is to ensure robustness, so the user has to explicitly provide both a unicode and an ascii bars to work in all envs.

Formatting follow a rule similar to egrep: the first character is the separator, then the rest is compiled in a list of symbols. Eg:

• {bar_symbols}|1|2|3|#{/bar_symbols} --> | is the separator
• {bar_symbols_loop},|,/,-,\{/bar_symbols_loop} --> , is the separator

Note that symbols are not necessarily only one character long, they can be longer (can be interesting for animated loops). /EDIT: tested and it works.

[lrq3000]

PS: I forgot to say that I tried to minimize the performance impact of this PR on standard core tqdm (without bar_format supplied): there are two variables instanciations at the top and then one additional if statement. That's all. The rest is entirely done in bar_format if branch and inside the new c_symbols if branch.

[casperdcl]

I was just working on this! going to merge into this PR... thanks, Stephen.

On 4 August 2016 at 22:58, Stephen L. notifications@github.com wrote:

Not the most urgent PR, but we missed some more customization (pretty sure
someone will ask someday!). So here it is: the ability to set custom bar
symbols, just like our fellow progressbar2
https://github.com/WoLpH/python-progressbar:

import time
from tqdm import trange

for i in trange(100, ascii=True, bar_format='{l_bar}{bar}{bar_symbols_ascii},A,B,C,D,E,O{/bar_symbols_ascii}{r_bar}'):
time.sleep(0.1)

for i in trange(100, ascii=True, bar_format='{l_bar}{bar}{bar_symbols_loop_ascii},/,-,,|{/bar_symbols_loop_ascii}{r_bar}'):
time.sleep(0.1)

Respective outputs:

9%|OOOE | 9/100 [00:00<00:09, 9.95it/s]

12%|/| 12/100 [00:01<00:08, 10.00it/s]

There are four different new tags for bar_format:

• {bar_symbols} and {bar_symbols_ascii} set custom progress bar
  symbols (unicode or ascii respectively)
• {bar_symbols_loop} and {bar_symbols_loop_ascii} set animated looping
  symbols (unicode or ascii respectively)

As you can see, I added both unicode and ascii tags, so that the user can
provide either one or both. If the user didn't provide custom symbols for
the target environment (eg, user provided {bar_symbols} but the
environment is ascii), then the bar will fallback to the default bar. Why?
This is to ensure robustness, so the user has to explicitly provide both a
unicode and an ascii bars to work in all envs.

Formatting follow a rule similar to egrep: the first character is the
separator, then the rest is compiled in a list of symbols. Eg:

• {bar_symbols}|1|2|3|#{/bar_symbols} --> | is the separator
• {bar_symbols_loop},|,/,-,{/bar_symbols_loop} --> , is the separator

Note that symbols are not necessarily only one character long, they can be
longer (can be interesting for animated loops), but I didn't try that.

TODO:

• Fix Flake8
• Add unit tests
• Update docstrings and readme

FOR A FUTURE PR:

• Multiline animated ascii-art bars like nyanbar
  https://github.com/apg/nyanbar or fish
  https://github.com/lericson/fish? Maybe we can get some inspiration
  from fish's docstring2line function? (we can memorize the number of lines
  we print and get back to original position, just like we currently do with
  automated nesting). Then we would have to figure out how to handle multiple
  progress bars spanning multiple lines... Probably as a separate submodule?

---

You can view, comment on, or merge this pull request online at:

#223
Commit Summary

• add custom bar symbols formatting

File Changes

• M tqdm/_tqdm.py
  https://github.com/tqdm/tqdm/pull/223/files#diff-0 (66)

Patch Links:

• https://github.com/tqdm/tqdm/pull/223.patch
• https://github.com/tqdm/tqdm/pull/223.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#223, or mute the thread
https://github.com/notifications/unsubscribe-auth/AKR9m-vahdnBaC1vt5efo1Y1VLnzE-WWks5qcmCAgaJpZM4JdKc1
.

[coveralls]

Coverage decreased (-3.3%) to 87.5% when pulling 2360e45 on custom_symbols into 50fc2d8 on master.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@1104d07). Click here to learn what that means.

@@            Coverage Diff            @@
##             master     #223   +/-   ##
=========================================
  Coverage          ?   78.48%           
=========================================
  Files             ?        8           
  Lines             ?      660           
  Branches          ?      135           
=========================================
  Hits              ?      518           
  Misses            ?      141           
  Partials          ?        1

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1104d07...a70a553. Read the comment docs.

[lrq3000]

Ah lol @casperdcl, maybe some kind of telepathy was at work here? XD

[lrq3000]

I didn't plan to work on this at all, it just popped into my mind a few days ago, so that's indeed quite a coincidence we were working on the same feature at the same time :)

[coveralls]

Coverage decreased (-3.4%) to 87.327% when pulling 23ebf76 on custom_symbols into 50fc2d8 on master.

[lrq3000]

Just for reference and for fun:

• https://github.com/iarna/gauge/issues/5#issuecomment-146386489
• http://changaco.oy.lc/unicode-progress-bars/ (from jitter in IPython notebook #50 (comment))
• Include a smoother unicode progress bar #38 (comment)
• http://stackoverflow.com/questions/2685435/cooler-ascii-spinners

[CrazyPython]

@lrq3000 to minimize performance strain, let's make a branch that internally changes the code, or even make a new class. This small feature shouldn't slow everybody down.

[lrq3000]

For standard tqdm, it's an increase of 3 opcodes per printing (not per loop), this doesn't impact much the performance.

However it can probably be implemented in a separate submodule. It would incur repetition of some code (calculations of the bar length and current frac) and thus violate DRY somewhat and the change of the tags from {bar_format} to [bar_format] which is a little bit weird because it will be different than other tags, but necessary else the formatting will choke.

[lrq3000]

Fixed the custom_symbols tag string extraction, now it's preprocessed once in tqdm.__init__(). The performance impact is 1 additional if statement in format_meter() with bar_format=None, and it's roughly the same as before custom_symbols for users specifying a template in bar_format (the calculations about the bar size have a few more operations but it's constant time so it's negligible).

[coveralls]

Coverage decreased (-4.8%) to 85.962% when pulling 96c782c on custom_symbols into 50fc2d8 on master.

[lrq3000]

Moved all the code into its own module in tqdm_custom, so we can freely experiment and implement things like #228 without fearing that it will slow things down. If later we find a way to implement these features without overhead, we can move them back to the core.

Please do NOT squash all the commits, because the original commit can be useful to reimplement back to the core.

[coveralls]

Coverage decreased (-14.2%) to 76.541% when pulling a433ef4 on custom_symbols into 17bb11d on master.

[coveralls]

Coverage decreased (-11.0%) to 79.821% when pulling c35629e on custom_symbols into 17bb11d on master.

[coveralls]

Coverage decreased (-14.2%) to 76.581% when pulling f0231e6 on custom_symbols into 17bb11d on master.

[lrq3000]

Added two new features:

• customization of indeterminate progress status (two ways: looping symbol or looping bar similarly to fish).
• looping speed is not dependent on n anymore but on mininterval. So this means that mininterval sets the maximum speed, not the minimum speed (if some iterations take longer than usual, if the bar is not refreshed then the animation won't move, that's normal because tqdm refresh display only when necessary, there's no threading, and it allows for great efficiency).

The downside of the last modification (looping speed not dependent on n) is that I had to make format_meter a standard instance method, so it will be a bit less efficient. I tried to use a closure but the closure is tied to the class method so n_anim gets shared across multiple tqdm instances and that's not good.

[coveralls]

Coverage decreased (-14.8%) to 76.02% when pulling 5256dfc on custom_symbols into 17bb11d on master.

[lrq3000]

The bar_format callback should be separated back into its own PR and merged with milestone >5.

[rth]

@lrq3000 Thanks for this PR (and tqdm in general )!

The bar_format callback should be separated back into its own PR and merged with milestone >5.

Is anyone planning to work on this part in the near future ? If not, I could give it a try..

[casperdcl]

Might merge in #227 (which is simple) and put this PR into a submodule as suggested there as it's quite large