-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
what's the decent way to update desc
and postfix
#1570
Comments
You may want to have a look at the from math import sqrt
from tqdm import tqdm
from joblib import Parallel, delayed
bar = "{desc}: Element number {n_fmt}... | {bar} | [{elapsed}<{remaining}, {rate_fmt}{postfix}]"
a = Parallel(n_jobs=2)(delayed(sqrt)(i ** 2) for i in tqdm(range(10), bar_format=bar, desc="Process A"))
# Process A: Element number 10... | ██████████████████████████████ | [00:00<00:00, 101.86s/it] The variables are, per documentation: Per documentation, the default bar is l_bar='{desc}: {percentage:3.0f}%|'
r_bar='| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}{postfix}]' Hope this helps. |
Thanks @CopperEagle , but what I want is a dynamic desc and postfix in realtime. For example, on data list the str |
Oh, I see @yantaozhao In this case, we can use # Process a single argument
def do_stuff(arg):
return len(arg)
# Process subroutine
def process(arg, pbar):
sleep(1)
pbar.set_description(f"Processing {arg}")
pbar.set_postfix_str(f"The almightiy {arg} is here...")
return delayed(do_stuff)(arg)
# ... elsewhere
pbar = tqdm([ 'a', 'b', 'c', 'd', 'e' ])
data = Parallel(n_jobs=2)(process(i, pbar) for i in pbar)
# Processing e: 100%|█████████████████████| 5/5 [00:19<00:00, 3.84s/it, The almighty e is here...] The notable difference is that the tqdm iterator can no longer be anonymous, as it needs to be passed to the pbar = tqdm([ 'a', 'b', 'c', 'd', 'e' ])
for i in pbar:
process(i, pbar) Cheers |
Good idea @CopperEagle . what I want is something like below (not real runnable code): data = ['a', 'b', 'c', 'd', 'e']
Parallel(n_jobs=2)(delayed(foo)(x) for x in tqdm(data, desc=lambda i: f'hello {data[i]}', postfix=lambda i: f'current value {data[i]}')) where |
Sure @yantaozhao, we can make it a wrapper function. Here's how: def updater(pbar):
iter = pbar.__iter__()
class AnonymousPbar:
def __iter__(self):
return self
def __next__(self):
arg = iter.__next__()
pbar.set_description(f"Processing {arg}")
pbar.set_postfix_str(f"The almightiy {arg} is here...")
return arg
return AnonymousPbar()
## Then you can use this anywhere:
for i in updater(tqdm(range(10))):
process(i)
# Processing 9: 100%|███████████████████| 10/10 [00:10<00:00, 1.00s/it, The almightiy 9 is here...] However, this may look a bit convoluted when using this with a The fallowing is a drop-in replacement for def progress(iterable, desc=None, postfix=None, **kwargs):
pbar = tqdm(iterable, **kwargs)
class AnonymousPbar:
def __init__(self, proc):
self.iter = proc.__iter__()
self.pbar = proc
def __iter__(self):
return self
def __next__(self):
arg = self.iter.__next__()
if desc is None:
desc_str = ""
elif isinstance(desc, str): # compliance with tqdm
desc_str = desc # want simple string? you get it.
else:
desc_str = desc(arg)
if postfix is None:
postfix_str = ""
elif isinstance(postfix, str): # compliance with tqdm
postfix_str = postfix # want simple string? you get it.
else:
postfix_str = postfix(arg)
self.pbar.set_description(desc_str)
self.pbar.set_postfix_str(postfix_str)
return arg
return AnonymousPbar(pbar)
## Then, anywhere in the code:
for i in progress(range(10)):
sleep(1)
# 100%|█████████████████████████████| 10/10 [00:10<00:00, 1.00s/it]
## progress accepts any keyword argument that tqdm does
for i in progress(range(10), desc=lambda arg: f"Process {arg}", ncols=80):
sleep(1)
# Process 9: 100%|█████████████████████████████| 10/10 [00:10<00:00, 1.00s/it]
## progress can be used with joblib as expected
data = [6,3,8,2]
Parallel(n_jobs=2)(delayed(sqrt)(i ** 2) for i in progress(data, ncols=80, postfix=lambda arg:f"Process {arg}"))
# 100%|█████████████████████████████| 4/4 [00:00<00:00, 1126.59it/s, Process 2]
# [6.0, 3.0, 8.0, 2.0] The length of the bars ( Any argument by |
smart wrapper function @CopperEagle |
Given below code which is from website of joblib:
What's the decent way to update the desc and postfix, if I want to trace and show the current processing number in the desc or postfix field?
Is there any way to avoid manually creating a new pbar object?
The text was updated successfully, but these errors were encountered: